PDA

View Full Version : Optimize Renderman for Maya's ray tracing?


jasonhuang1115
01-29-2008, 09:47 PM
Hi, everyone,

I use RFM 2.0 and bake the point-based brickmap for a mechanical character scene(0.3 million polys only, 100+ 2k tiff textures, subD scheme applied on all geos). But when I ray trace the scene trying to ray-traced reflection or to make 3d brickmap (using makeGlobalDiffuse3d), the render keep failing on my duo core PC with 2 gigs memory. (Maya was closed and using batch render) I can only finish a 720x480 rendering with. But when rendering 1440x960 size, the physical memory consumed up to 1.3 gigs, then the rendering suddenly finished with half-size file which is unable to open in fcheck or photoshop.

I read a little documentations and "Ray-tracing for the movie Cars". They mentioned new implementations such as "ray differential", "multi-threading".....for RFM 2.0 and RMS I am wondering where I can find related settings to optimize ray-tracing my scene in RFM.

Thanks.

lightcache
01-30-2008, 08:35 AM
Do the frames render correctly within maya?
there are some rendering "issues" within Renderman still, which are due to be fixed in the next patch (ive heard that before ;) ) mainly kicking alfred up the arse.
also,

try tweaking your bucket size and memory allocation.
and what does the script editor say?
if the renders hanging it'll tell you why there and in the ini. file.

ACamacho
01-30-2008, 10:42 AM
All the things lightcache said...

In the Advanced tab of the renderglobals, under "Statistics" there is an option to save out an XML statistics file. Enable it and point to a directory to save it to. It has alot of info on the different steps of rendering and memory usage.

Do you have raytracing enabled in any of the "make3d" passes? Don't think you would need it enabled. You didn't say if it renders fine without raytracing. Also, and may not be an issue, do you have the Sampling Type in the environment light set to "Baked" to use the baked map instead of other methods? I assume it would use the baked map anyway.

If it doesn't raytrace even with no textures, would you be able to post the scene to see in detail?

jasonhuang1115
01-30-2008, 04:41 PM
Thank you, lightcache and Angel, :)

The frame renders correctly with baked point-based brickmap and when not ray tracing the scene. But when I simply turn ray-tracing on in the render global, it won't be able to render correctly. When I batch render it, here below is the last couple lines in the script editor.

---------------------------------------------------------------------------
// Percentage of rendering done: 38 (K:/projects_backup/formula_91/renderman/f91_beta_29/data/job/tRender012608_beauty_rmanRenderRadiosityPass.job) //
// Percentage of rendering done: 39 (K:/projects_backup/formula_91/renderman/f91_beta_29/data/job/tRender012608_beauty_rmanRenderRadiosityPass.job) //
// Rendering Completed. See mayaRenderLog.txt for information. //
-------------------------------------------------------------------------

It took about 40 mins to get there, suddenly finish the rendering and leave renders, which I can view in fCheck, in the renderman's image subfolder.

And here below is last couple messages fromthe mayaRenderLog.txt.

------------------------------------------------------------------
Shader:'K:/projects_backup/formula_91/renderman/f91_beta_29/shaders/LFootRedLyr_']
Error: (01/30 07:45) rman Error: R52003 Can not allocate space for grid (57756 bytes) (mem: 1123582249/1123582249)
Error: (01/30 07:45) rman Fatal: P00003 Out of memory (shading related). (mem: 1123582249/1123582249) [<Shading> Object:'polySurface312Shape' Shader:'K:/projects_backup/formula_91/renderman/f91_beta_29/shaders/MetalDarkMat_']
Error: (01/30 07:45) rman Fatal: P00003 Out of memory (shading related). (mem: 1123581362/1123581362) [<Shading> Object:'polySurface312Shape' Shader:'K:/projects_backup/formula_91/renderman/f91_beta_29/shaders/MetalDarkMat_']
Error: (01/30 07:45) rman Error: R52003 Can not allocate space for grid (43520 bytes) (mem: 1123580830/1123580830) [<Shading> Object:'pCube852Shape' Shader:'K:/projects_backup/formula_91/renderman/f91_beta_29/shaders/ChestGridMat2k_']
Result: K:/projects_backup/formula_91/tRen
-----------------------------------------------------------------

The bucket size was 16,16. Should I go higher or lower to have better memory control? Also, I can't find the "memory allocation" in Maya render global's renderman menu. I have found there is another renderman's render globals in "Renderman" drop-down menu which I only used once a while back.

Angel,
I tried to specified the path for both "Statistic File" and "Statistics XML File" (I didn't type in file extension for both of then), in this particular render job, it saved out 1 kb and 0 kb unrecognizable files respectively. Is that because I should specify a certain file format for them or that is due the render failure?

When I tried to use "MakeGlobalDiffuse3d" node to create 3d brickmap, I did turn on ray-tracing in both renderglobals and inside "RenderGlobalDiffuse3d" node. Because I recalled when I use a simple sphere scene to test making 3d brickmap, I had to turn ray-tracing on to see the reflection of environment, but maybe I got it wrong. I will test that again. :)
BTW, I will try to ray trace the scene without textures and see how it goes.

Thanks a lot. :thumbsup:

Bitter
01-31-2008, 07:23 PM
Yeah check the texture size. Also, lowering the bucket size saves some memory at the expense of speed. It looks at smaller chunks of the scene at once. It also shades surfaces in chunks, might need to shrink that as well.

I do not believe ray differential settings are implemented in Renderman for Maya. mental ray has a shader to do that from ctrl studio but I haven't seen one for Renderman as it doesn't really have an explicit control for tessellation.

jasonhuang1115
01-31-2008, 11:43 PM
Thanks, bitter,

I ray trace the scene with default lambert assigned and RFM correctly renders the scene (1600x1066 resolution) without too much hassle (roughly about 10 to 15 mins I guess). Then I tried lowering the bucket and grid size to 8,8,64 or 4,4,16, do batch render, and close maya. It was pretty stable at the beginning with 16 to 50 mb usable memory left, but suddenly dropped dramatically around 35% rendering finished and the render failed. :sad:

Buexe
01-31-2008, 11:47 PM
Does it render without the brickmap? I would also suggest checking if you can use reflection sets and reflection distance, so that you can lower the total amount of ray-tracing calculations.

jasonhuang1115
01-31-2008, 11:54 PM
Does it render without the brickmap? I would also suggest checking if you can use reflection sets and reflection distance, so that you can lower the total amount of ray-tracing calculations.

Thanks, Buexe,

It's rendering with baked point-based brickmap. Would you point me where the setting of reflection distance is? I can only find a Max dist. in the environment light setting, but I set it to 100, according to my scene size, already when I bake the brickmap. :)

Jason

jasonhuang1115
02-01-2008, 12:48 AM
I tried rendering on a faster PC with 3 gigs ram and started wondering if the problem is from "out of memory". The machine still got maybe 500 MB memory left and the rendering suddenly stopped. In the mayaRenderLog (I also attach the file), it shows:

Error: (01/31 18:39) rman Error: R52003 Can not allocate space for grid (63696 bytes) (mem: 1122831027/1122831027) [<Shading> Object:'polySurfaceShape594' Shader:'C:/Documents and Settings/cxh045000/Desktop/1/projects/formula_91/renderman/f91_beta_29/shaders/ChestBlueLyr_']
Result: C:/Documents and Settings/cxh045000/Desktop/1/projects/formula_91/tRender012608_beauty__4008.ma
Fatal Error. Attempting to save in C:/DOCUME~1/CXH045~1/LOCALS~1/Temp/cxh045000.20080131.1839.ma
Error: (01/31 18:39) rman Error: R52003 Can not allocate space for grid (63696 bytes) (mem: 1122831607/1122831607) [<Shading> Object:'polySurface318Shape' Shader:'C:/Documents and Settings/cxh045000/Desktop/1/projects/formula_91/renderman/f91_beta_29/shaders/ChestGadgetsMat2k_']
Error: (01/31 18:39) rman Error: R52003 Can not allocate space for grid (59736 bytes) (mem: 1122831607/1122831607) [<Shading> Object:'polySurface318Shape' Shader:'C:/Documents and Settings/cxh045000/Desktop/1/projects/formula_91/renderman/f91_beta_29/shaders/ChestGadgetsMat2k_']

Buexe
02-01-2008, 09:05 AM
Thanks, Buexe, Would you point me where the setting of reflection distance is?
Jason
Select your material and open the Attribute Editor (AE). From the menu of the AE select Attributes>Renderman>Add Reflection Controls.
Four extra Attributes should show up now under "Extra Renderman Attributes" in the AE. Check the docs how to use them ( although they are pretty much self-explaining). I really think that you have a memory problem. Since RfM exists only as 32bit, there is a limited amount of RAM it ( or Maya 32bit) can handle. Maybe you have to split your image into passes? Generally you don`t need raytracing to get reflections with RfM, so maybe you can do an extra reflection pass without raytracing? Also if there are shapes that don`t need to be included in the brickmap, I think you just need to turn off "visible in refelctions" on the shape node. This tells RfM not to do raytracing on the given shape node ( not very user-friendly IMHO).
Hope this helps
buexe

jasonhuang1115
02-04-2008, 08:37 PM
Thank you, Buexe for the reflection control tip. :thumbsup:
If I lower the reflection distance, is that going to cut down the memory consumed during the rendering?
Also, when you mention that I don't need to ray-trace the scene in RFM to get reflection, do you mean by just using a reflection map to "fake" the reflection?

Buexe
02-04-2008, 10:03 PM
Whoops, I mixed something. I was using HDR image reflections on my last project. There I didn`t need to turn on ray-tracing to get reflections. Sorry, for the confusion. Lowering the reflection distance should lower the memory consumption, since less reflections = less memory. But there are probably more advanced users who could explain that better. I just heard that on the Digital Tutors DVD.

Bitter
02-04-2008, 10:47 PM
Raytracing usually builds geometry on demand. Lowering the reflection distance means some objects that don't contribute meaningfully to the reflection (far away objects) get left out and therefore don't need to be built. That means it saves memory because fewer objects have to be constructed to finish the current bucket.

The HDR acted as an environment map for reflections and doesn't need to be traced to show up. In Cars, pixar traced nearby objects (cars passing next to one another) but called a texture beyond that distance to save time and memory. A lot of shaders have that option.

jasonhuang1115
02-11-2008, 11:03 PM
Thanks Bitter for the explanation of reflection distance.

I change some settings as below:
1) cranking up the custom shading rate of some geometries with displacement from 0.1 to 0.5.
2) add reflection control to all of the shaders. Set the distance to 50.
3) lower the bucket and grid size to 8,8, 64 or even 4,4,16 to sacrifice the time for memory.

The scene renders out 800x533 resolution images, but when I tried rendering out 1600x1066 (the res I actually need). It still failed in the middle.

I attached the statistics xml files of both 1600x1066 non-raytracing and 800x533 raytracing scene (can't get the 1600x1066 raytracing statistic because it's can't be open with browser). Could someone take a quick look and point out if there is still possibility to render out the resolution I need with raytracing? :)

Thanks.

Bitter
02-12-2008, 12:53 AM
I just rendered 12660 x 7200 in mental ray with a premade smaller final gather map. I had the image sliced into different tiles. It managed the render fine. I haven't looked but it should be possible to slice your image into smaller chunks to render at one time. Or render region very tediously.

ACamacho
02-12-2008, 10:27 AM
If your able to share your file, can you post it here for us to take a look? (or if too large email me @ excaliber2013@yahoo.com (http://forums.cgsociety.org/excaliber2013@yahoo.com)).

It's very strange because I am working with a set as I write this that is close to 3M polygons and a mix of 1K-4K textures (200-300 unique objects....instanced goes much higher). And I am able to use raytraced shadows (no reflections/refractions however).

Although, does most of your geometry have custom shading rate around the 0.1-0.5 range? Does it raytrace with everything at a rate of 1.0 or higher? Remember that having the shading rate so low eats up memory quickly. Which I know you are using to make sure your texture remains sharp at a distance.

*edit: I don't think your attachment made it. Don't see it. :(

jasonhuang1115
02-12-2008, 09:24 PM
My bad. Here below is the statistics.

ACamacho
02-13-2008, 10:16 AM
I am not at work to view it again (had to modify the xml file to view it on my browser and feel lazy to do it again for home :) ), but one big red flag I saw is that your "Catmull-Clark subdivision memory" was around 300mb for the low-res raytracing render. That is really high and probably because of your high shading rates for geometry (0.1-0.5 from what you said). So when you rendered at higher res it must have taken some ungodly amount and didn't leave much room for anything else. Does the scene render at higher res with everything at a shading rate of 1?

Also, check your displacement bounds for your objects that have displacement. Seems like the bound is too high for the amount of displacement done. For example one of your objects has the bound at 1.0000 while the actual displacement is 0.004. That can eat up some memory too.

jasonhuang1115
02-14-2008, 06:46 AM
I just rendered 12660 x 7200 in mental ray with a premade smaller final gather map. I had the image sliced into different tiles. It managed the render fine. I haven't looked but it should be possible to slice your image into smaller chunks to render at one time. Or render region very tediously.

Thanks, Bitter. Would you mind shade a light how to render the image into different tiles? Or, point me to somewhere to dig in. What would be the pros and cons to render into tiles? Actually, I am not looking for that high-res render. A standard HD-res render, like 1920x1080, will make me very happy. :) Also, I am using RFM due to its AOV feature. I need those passes in the post for compositing.

jasonhuang1115
02-14-2008, 07:06 AM
I am not at work to view it again (had to modify the xml file to view it on my browser and feel lazy to do it again for home :) ), but one big red flag I saw is that your "Catmull-Clark subdivision memory" was around 300mb for the low-res raytracing render. That is really high and probably because of your high shading rates for geometry (0.1-0.5 from what you said). So when you rendered at higher res it must have taken some ungodly amount and didn't leave much room for anything else. Does the scene render at higher res with everything at a shading rate of 1?

Also, check your displacement bounds for your objects that have displacement. Seems like the bound is too high for the amount of displacement done. For example one of your objects has the bound at 1.0000 while the actual displacement is 0.004. That can eat up some memory too.

Hi, Angel. Thanks again as always. :bounce:
Sorry, I have been working all day long and unable to take a look at the scene. Basically, I set the global shading rate to 1 to remain the sharpness of textures in the render. And I set custom shading rate as 0.1 for objects with displacement, but later on crank up that to 0.5 to save some memory. If there is a way, in RFM, to let a certain geometry to look up to another UV set (say default UV set with other geometries for color map, and a 2nd UV set of its own just for displacement map), then I can get away with using custom 0.1 shading rate for those geometries. All poly geometries are attached with subD scheme. Regarding the displacement bound, should I add a rman displacement bound attribute to those displaced geometries (I haven't done this before)? Everything about displacement is set as default in render globals.

Thanks. :thumbsup:

Bitter
02-18-2008, 05:45 AM
I haven't had a chance to check on tiling in RMAN4Maya yet. The renderman forums should have the answer to that. If it follows the regular Maya convention it's not very hard to do. If worse comes to worse (and it did for me because the installation of Maya is hosed) I had to render region by hand and save the image. (Couldn't use cmdline rendering because the file was 2008 and 8.5 was installed by IT after 2008. It wanted to render with the wrong version of Maya and abort. I.T. won't give me Admin to fix that. . .I hate.)

Craziness. But it worked. Just don't get greedy and render region something huge.

Buexe
02-18-2008, 08:48 AM
The Render command has a region ( -reg ) flag, which can be used to specify exact tiles.

Buexe
02-18-2008, 09:58 AM
I have uploaded my jbRenderTiles script to highend3D.com. It allows you to specify how many horizontal and vertical tiles it shall render and optionally creates a javascript to put those together in Photoshop. I have not used it with RfM yet ( because I didn`t run into memory problems with it yet :) ). You can get it here:

http://highend3d.com/maya/downloads/mel_scripts/rendering/misc/jbRenderTiles-5111.html

If you put in "file" in the renderer option box it should use the renderer stored in your scene file. This is an early version, so please let me know, when you encounter any issues.
I hope some of you may find it helpful.
http://img206.imageshack.us/img206/8067/jbrendertilesrc3.jpg?s=1

Cheers
buexe

jasonhuang1115
02-21-2008, 03:06 AM
Thanks so much guys. For some reason, I didn't get updated post through google reader and thus missed some precious ideas and the definitely-should-try script. I was about the check the RenderPal for split up the render into regions.

Luckily, before I jump on rendering into tiles, I spent time tuning the setting according to Angel's suggestions and actually get the rendering to work. :applause:

The custom shading rate for displaced geometries is still pretty low as 0.2 (global shading rate 1). I set the displacement bound to a very low value, just about a bit larger than the actual displacement that I read from render statistics (some value like 0.002). Also, lower the reflection distance, again, from 50 to 10. At the first couple trials, the rendering still failed. I found it crashed while writing out the multi-channel exr (all other passes were 0k in size, and the exr was several mb). I delete that multi-channel exr output, and amazingly, the rendering finished with the correct render that I want.

I am not sure if the way I set the displacement bound is correct. The displacement ratio was some value way below than 1, but now is 83 or value way bigger than 1. I don't know how much this is affecting the memory consuming??

Also, the writing of multi-channel exr seems actually crashing my rendering. I tried shading rate of 1 on all geos (Subsystem memory total 758 MB, Catmull-Clark subD memory 306 MB) with writing multi-channel exr, and the rendering failed. However, with custom 0.2 shading rate ( Subsystem memory total 800 MB, Catmull-Clark subD memory 437 MB), not writing .exr, the rendering finished. :eek: Any thought on this?

Again, thanks guys. I have been learning a lot through this discussion.

jasonhuang1115
02-21-2008, 07:06 AM
Buexe,

I install the script, set the project folder, and browse to the file that I want to render. But not matter I put "rman" or "file" in the "Renderer" slot, the batch render won't kick off whereas I put "mr", then the maya batch render started rendering with mental ray??
(I have maya 2008 on the machine, but the renderman is installed on maya 8.5)


I have uploaded my jbRenderTiles script to highend3D.com. It allows you to specify how many horizontal and vertical tiles it shall render and optionally creates a javascript to put those together in Photoshop. I have not used it with RfM yet ( because I didn`t run into memory problems with it yet :) ). You can get it here:

http://highend3d.com/maya/downloads/mel_scripts/rendering/misc/jbRenderTiles-5111.html

If you put in "file" in the renderer option box it should use the renderer stored in your scene file. This is an early version, so please let me know, when you encounter any issues.
I hope some of you may find it helpful.
http://img206.imageshack.us/img206/8067/jbrendertilesrc3.jpg?s=1

Cheers
buexe

Buexe
02-29-2008, 01:44 AM
Sorry for the late reply, I was busy and the Renderman flags are a little bit different to the mr flags, so I had to adjust the MEL script a little bit. Here is a new version that should work with "rman" as Renderman renderer. Please check if it works for you. It does now work with my RfM renders.

jasonhuang1115
02-29-2008, 02:39 AM
Sorry for the late reply, I was busy and the Renderman flags are a little bit different to the mr flags, so I had to adjust the MEL script a little bit. Here is a new version that should work with "rman" as Renderman renderer. Please check if it works for you. It does now work with my RfM renders.

Thanks, Buexe, for updating the script. However, I spent about 10 mins to test it, but now I can't even use the mental ray to render (was able to with the ver. 0.1).

I even just create a sphere in the scene, set the renderer to MR, save the scene, load the script, browse to saved the sphere scene, and "go tiger". Maya then freeze with "no response" in the task manager?? I am using Maya 8.5 sp1.

Buexe
02-29-2008, 08:57 PM
Hey jasonHuang,
Strange it`s not even working with a sphere. I spend some time to optimize the Script today and changed the progressBar to a progressWindow, so now you can interrupt the process by pressing Esc. In the zip file is my test scene, Maybe you have some time to check it out, would be great to hear some feedback. So far it is working flawless here. Please note currently the list of supported renderers is:
sw - Maya Softwre
mr - mentalRay
rman - Renderman
so "file" as an argument is not supported!!!
I will make something nice in the near future that will allow only "valid" renderers to be selected, since the syntax/output for the command line rendering is a little different for each renderer. Note also that if this doesn`t work for you, you can add a "crop" attribute in you final pass tab, which lets you render only a specific region of your image. By having multiple passes with different crops, you can create your tiles manually.
Hope this helps
Cheers
buexe

jasonhuang1115
02-29-2008, 10:46 PM
Thanks, Buexe, for the quick reply and first of all, it's a really handy script and I am really looking forward to it's developing.

Actually, I believe it was working last night just as now I finish rendering your provided test scene. Couple concerns as below (using your test scene):

1. After hitting "Go, Tiger", Maya acts like rendering with the render view(sandglass as my mouse tip) and it kept showing that Maya is "no response" in my window task manager until the whole 9 tiles finished rendering (the reason I thought it wasn't working last night). So, I guess I can't close maya to free up more memory for rendering, which is part of the intention of this script)

2. It automatically activates Photoshop (I checked that option), but I have to manually open/drag that generated .jsx file in Photoshop to see the process of combining tiles. I love it generates the jsx file, but kind of prefer manually open that jsx file if I come back to check the render job, say, 24 hours later. :)

3. There are seams along the edges of each tile in the final combined image. It's a problem of render settings or it's normal that one has to manually clone out the seams.

Thanks a lot.

Buexe
02-29-2008, 11:38 PM
Thanks, Buexe, for the quick reply and first of all, it's a really handy script and I am really looking forward to it's developing.

Actually, I believe it was working last night just as now I finish rendering your provided test scene. Couple concerns as below (using your test scene):

1. After hitting "Go, Tiger", Maya acts like rendering with the render view(sandglass as my mouse tip) and it kept showing that Maya is "no response" in my window task manager until the whole 9 tiles finished rendering (the reason I thought it wasn't working last night). So, I guess I can't close maya to free up more memory for rendering, which is part of the intention of this script).
Maya is "locked" because the external call to the render command "locks" it. I thought about it,too, an alternative would be to write out a .bat file that would make it completely independent of the Maya GUI after they were generated. But I thought it would be nice to see the overall process and since the renders can be started without the scenes being open, I found it okay the way it is. If you really need to start it from the command line without the GUI, you can create a bat file by copying the render commands from the Script editor into a text file ( beware of the (back)slashes though.


2. It automatically activates Photoshop (I checked that option), but I have to manually open/drag that generated .jsx file in Photoshop to see the process of combining tiles. I love it generates the jsx file, but kind of prefer manually open that jsx file if I come back to check the render job, say, 24 hours later. :)

Haver you seen that the javascript file is in your images directory? you can start it anytime again, I jsut have to double click on it and it starts photoshop and the process.


3. There are seams along the edges of each tile in the final combined image. It's a problem of render settings or it's normal that one has to manually clone out the seams.

Yeah, that`s one thing I found too when having transparent layers. Using > Matting > Remove black Matte in Photoshop will remove some of it, but not all. I`m still looking for a solution, if there is some photoshop expert out there, I `d really like to get some input on this.

Thanks a lot.
Glad you like it, it is in an early stage and probably only for those Windows folks, but it does a nice job and I thought others might also find it useful. I was really surprised that there wasn`t a solution/tool for this common task/problem.
Cheers
buexe

jasonhuang1115
03-28-2008, 04:14 PM
Hi, Buexe

I tested your script again yesterday and had some problem.
I first started out rendering a super simple sphere with one light using rman as renderer. Everything went smoothly.
But when I turned to try to script with the scene I am working on, I got washed-out(too bright) render. I compare the secondary outputs generated from regular batch render and notice that, when using your script, the diffuseIndirect and diffuseEnvironment output are incorrect (too bright or I don't know how to properly describe it).
There is a brickmap which was baked once per job (my character is not moving). Have you tried this script with baked brickmap?

Thanks.

jasonhuang1115
03-28-2008, 04:38 PM
Hi, Buexe

I tested your script again yesterday and had some problem.
I first started out rendering a super simple sphere with one light using rman as renderer. Everything went smoothly.
But when I turned to try to script with the scene I am working on, I got washed-out(too bright) render. I compare the secondary outputs generated from regular batch render and notice that, when using your script, the diffuseIndirect and diffuseEnvironment output are incorrect (too bright or I don't know how to properly describe it).
There is a brickmap which was baked once per job (my character is not moving). Have you tried this script with baked brickmap?

Thanks.

Buexe
03-30-2008, 09:43 PM
hey jason, sorry I have no clue what could be the problem from what you are describing. My script uses the "crop" flag from renderman, which you can also set manually in the render globals. So there is no "secret magic" what could influence the rendered image from my perspective. Sorry, I have no clue : (

jasonhuang1115
03-30-2008, 10:22 PM
That's totally cool, Buexe. I will check out the crop flag in the render globals and see what I can do about it. Thanks anyway. :)

CGTalk Moderation
03-30-2008, 10:22 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.