Command-line rendering with custom plug-ins

system · 2015-11-25 22:55:05 UTC

Hi all

I’m looking to batch render a scene that contains non-standard plug-ins. Problem is, Maya doesn’t necessarily load any of them automatically when rendering so any failing nodes that depend on my plug-in are causing my frames to render completely black.

My situation:
I’m a student who is using my school’s render farm. In short, the plug-ins I want to use are render/lighting related so I can’t simply cache out my geo/shaders/animation. That and also our render farm uses the command-line Render.exe in order to render the scene.

I’ve tried:

On Open/Close script nodes to use MEL and load the plugin
A few scriptJob ideas
The Pre/Post render MEL

My faux-example scene file:
I made a basic scene file - it’s a camera and a poly primitive that’s made using the the fePrimitives 0.9.1 plugin. If you’re using Maya 2016, I provided the plugin/Maya scene file in the attachments. Otherwise that, the download link to the plugin is below:

http://www.creativecrash.com/maya/plugin/feprimitives

This is a minimalist representation of my problem where:

In Maya, with the plug-in loaded, I can render the star from the Render View
In Maya, with the plug-in loaded, I can render the star from the Batch Render command
In command-line Render.exe, the frame renders black

Again, I know that I could just cache out the geo before rendering and avoid using the fePrimitive plugin all-together (it’s just an example! )

I feel like this error is fairly basic but for whatever reason I haven’t solved it yet. Did I overthink this? How do you command-line render with custom plugins?

haggi · 2015-11-26 01:01:57 UTC

The problem seems not to be related to loading the plugin but finding the plugin.
If you can render the scene if you start a batchrender, then it basically works because the render.exe does nothing else but starting a mayabatch.exe.

If you have nodes which require a plugin in your scene, this is saved in the requirements in the scene file. This means the plugin loads automatically if it can be found.
To me this sounds as if the renderfarm uses another evironment than you and simply does not find the plugin. If you have access to the logfiles of the rendering it could be helpful to have a look and search for problems during scene load.

system · 2015-11-27 03:44:38 UTC

I thought the same thing at first. I first tested what the farm does when I intentionally don’t provide the plug-in. As expected, it errs out with absolutely no frames (not even black).

I still haven’t gotten my example file to run locally using the command-line but I thought I’d take advantage of the log files from the farm and made a python script to add plugins. I placed the .mll/.so files in the “scenes” directory for the test.

import os

currentScene = os.path.abspath(cmds.file(q=True, sn=True))
polyPlug = os.path.join(os.path.dirname(currentScene), 'fePolyPrimitives2016.so')
curvesPlug = os.path.join(os.path.dirname(currentScene), 'feCurvePrimitives2016.so')

os.environ['MAYA_PLUG_IN_PATH'] = polyPlug + os.pathsep + curvesPlug + os.pathsep + os.environ['MAYA_PLUG_IN_PATH']
cmds.loadPlugin(polyPlug)
cmds.loadPlugin(curvesPlug)

I then used MEL to wrap the command so that I could place it in the Pre render MEL:

python "import os; currentScene = os.path.abspath(cmds.file(q=True, sn=True)); polyPlug = os.path.join(os.path.dirname(currentScene), 'fePolyPrimitives2016.so'); curvesPlug = os.path.join(os.path.dirname(currentScene), 'feCurvePrimitives2016.so'); os.environ['MAYA_PLUG_IN_PATH'] = polyPlug + os.pathsep + curvesPlug + os.pathsep + os.environ['MAYA_PLUG_IN_PATH']; cmds.loadPlugin(polyPlug); cmds.loadPlugin(curvesPlug)"

Here’s an example output of the farm:

Log File:

/home/csadmin/storage is not a mountpoint
/home/csadmin/storage is a mountpoint
-----------------------------------------------------------------------------------------------------------
Setting Environment Variables...
PROJ_PATH=/home/csadmin/storage/302405
PROJPATH=/home/csadmin/storage/302405
-----------------------------------------------------------------------------------------------------------
Rendering plugin_load_pre_render_003.ma from project...
/usr/autodesk/maya2016/bin/Render -s 1 -e 1 -of jpg -proj /home/csadmin/storage/302405/ -rd /tmp/farm/Job_302405 /home/csadmin/storage/302405/scenes/plugin_load_pre_render_003.ma

Starting "/usr/autodesk/maya2016/bin/maya"

Error: file: /home/csadmin/storage/302405/scenes/plugin_load_pre_render_003.ma line 6: RuntimeError: file  line 1: Plug-in, "fePolyPrimitives", was not found on MAYA_PLUG_IN_PATH.
Warning: file: /home/csadmin/storage/302405/scenes/plugin_load_pre_render_003.ma line 119: Unrecognized node type 'makeFePolyStar'; preserving node information during this session.
Warning: line 1: Errors have occurred while reading this scene that may result in data loss.
File read in  0.96 seconds.
Result: /home/csadmin/storage/302405/scenes/plugin_load_pre_render_003.ma
Total Elapsed Time Since Start Of Maya (hh:mm:ss): 00:00:09
====================================
Resource Usage At Start Of Rendering
====================================
       2	Page faults
 223.457 Mb	Max resident size
   0.000 Mb	Peak total size(Estimated)
   0.000 Mb	Peak arena size
====================================
  61.145 Mb	Heap
====================================
Rendering using 16 render processes.
A default light has been created, modify defaultRenderGlobals.enableDefaultLight to change this behavior.
Automatic near/far clipping values: 0.1, 1.5332e+08.
Starting Rendering /tmp/farm/Job_302405/plugin_load_pre_render_003.0001.jpg.
Constructing shading groups

Warning: line 0: TrenderPolyShape::sample: mesh 'fePolyStarShape' delivered an empty surface.
Rendering current frame.

Frame triangle count: 0
====================================
Resource Usage At End Of Frame
====================================
 264.297 Mb	Max resident size
   0.000 Mb	Peak total size(Estimated)
   0.000 Mb	Peak arena size
====================================
  61.233 Mb	Heap
====================================
Postprocessing rendering result.
Time For Tessellation (hh:mm:ss): 00:00:00
Time For Shadow Map   (hh:mm:ss): 00:00:00
Time For Post Process (hh:mm:ss): 00:00:00
Time For Frame Render (hh:mm:ss): 00:00:00
Finished Rendering /tmp/farm/Job_302405/plugin_load_pre_render_003.0001.jpg.

------------------------------------------------------------


====================================
Resource Usage At End Of Rendering
====================================
   0.000 Mb	Peak total size(Estimated)
   0.000 Mb	Peak arena size
====================================
  61.180 Mb	Heap
====================================
Total Time For Render (hh:mm:ss): 00:00:00
Total Elapsed Time For Maya (hh:mm:ss): 00:00:09
Scene /home/csadmin/storage/302405/scenes/plugin_load_pre_render_003.ma completed.
-----------------------------------------------------------------------------------------------------------
Frame copied successfully to render attempt 1.
-----------------------------------------------------------------------------------------------------------

The error comes up pretty early on - “Plug-in, “fePolyPrimitives”, was not found on MAYA_PLUG_IN_PATH”, even though I actually had sourced that plugin properly using the Python command. The frame rendered black, rather than nothing when I didn’t include the plugin, so I can infer that the node is finding and sourcing the plugin that I provided.

That error is strange but I thought “why not” and added just the string “fePolyPrimitives” to os.environ[‘MAYA_PLUG_IN_PATH’] but the error persists. The log files were enlightening but I’ll do a few more experiments locally and see if I can get a working example

haggi · 2015-11-27 04:13:30 UTC

The error occurs during file load, and you set the environement not until the rendering starts. Even if you load the plugin at this time the necessary connections and nodes have already failed and are not automatically updated.

The simplest way to solve your problem is to define a global plugin directory somethin like:
/usr/data/mayaplugins
and let the renderfarm set the environment variable before it starts rendering.
Of course you can try to build whatever geometry you need after loading the plugins by yourself in the script, but even if this is possible it makes life quite complicated.