Forests in maya mental ray.


Has anyone done it? Hundreds, thousands of 50,000 poly + trees?

Is it even possible? How would you do this?

I’m running into an issue where it seems the BSP data for a tree is nearly as much as the actual mesh data itself. A 60 mb mesh is making 40 mb of BSP2 data, per instance. So 80 instances are taking up 3.2 gb of ram.

I’ve been trying to use large BSP but it just does not seem to work. It crashes, the only thing that seems to be able to take so many polys is BSP2, but BSP2 does not flush like large BSP. So would I then render the image in strips? I rendered a 40x640 strip in a dense forest with BSP2 and it still ate up all 9gb of my memory. So I’d imagine strip rendering would have to be in ridiculously small chunks like 8x32 or something. Which could get nuts real quick.

BTW anyone get the automated strip rendering in backburner to work for maya?


I wouldn’t dare even try with this with Maya 2009 but haven’t really played around with mental ray in 2010, however this thread may be of use to you even though they talk of Max it’s practically identical.

There are some really good tips like not to use cutout opacity on leaves, and generally how proxies work.

Hope it is of some use:

One of the guys in there posted a render of a scene that has over 7k trees and 6k bushes in it and it’s looks pretty dense.


After more searching I found a site that does plugins for forests and 3DS Max. I’m posting the link because they have galleries where renders where done in mental ray:

Vray with 500 billion polys and the mental ray picture has many millions of polys too, so you should be able to do this.

Edit: Just see this in their known problems list, some good tips you might find useful too.

  • To get acceptable rendering times in large scenes, it’s necessary to use BSP2 as raytrace method. It is available only in Max 2009 or greater. For previous versions, or small-to-medium size scenes, you can use BSP instead.

  • Always disable “Scanline” checkbox

  • BSP2 is not fully stable yet. It uses a dynamic memory management engine, able to flush memory when necessary, but sadly, under heavy load it crashes sometimes. From our tests results, stability depends of the scene complexity, including number of trees and triangles. We have found problems in scenes using above 100.000 trees of 500.000 triangles each one (that really is a lot). These are some problems that you can find:

[li] Max 2009: Fatal errors after completing a render as “accessing unknown tag” or “accessing null tag”. You can minimize this error increasing “Memory Limit” parameter at “Rendering Setup->Translator Options”. The default value is 650Mb. Try using different values above 1500Mb. (of course, only if you have enough free memory available).[/li][li] Max 2009: Fatal error cancelling a render when Final Gather is being processed. Previous fix may be useful here as well.[/li][li] Max 2009: Other errors while rendering.[/li][li] Max 2010: Crashing error if material editor is opened after completing a render. This issue is already reported to Autodesk.[/li][/ul]You must understand that these errors are not caused by Forest, and we cannot fix them. In most of situations, Mental Ray is stable because common scenes don’t include hundreds of thousands of high poly meshes as Forest is able to handle. These problems are known by Autodesk and Mental Images (Mental Ray creators), because latest versions (as Max 2010 or Maya 2009 ) are more stable, although there are bugs to fix yet.

  • Rendering times with more than 500.000 trees grows exponentially. This seems to be the limit of Mental Ray handling large number of instances.

And remember that Forest allows you to combine 3D trees with billboards to reduce memory footprint.

If you really need to use millions of complex trees in your scenes, we suggest you switch to VRay. This renderer is much more stable and fast, and using it together Forest is an amazing experience.

I get sometimes the same errors in Maya (where it talks about accessing unknown tag)


in maya 2010

use mip_binaryproxy instead of mr_proxy, use BSP2 and Raytracing in the Features Primary renderer, and it works, 9.000 trees, each tree about 1000k polys.

memory ussage before render (maya interface and a lot of softwaare opened) 4.5 gb.
maximun memory usage during rendering 4.72 gb.

rendering time for the first image, 4:32 min. for the second one: 2:54.
resolution: 1200 x 900, the rest of parameters as it comes from preview finalgather.

i hope this can help you,


Did you think about the LoD (Level of Detail) system which comes with maya?


the LOD didn’t work with MR as far as i know in 2008


so with mip_binaryproxy + bsp2 + raytrace, you were able to render almost 1 000 000 000 triangles? not bad
could you give more detail on your machine?


Ohh, I didn´t know that.


Why wouldn’t it? It’s just a clever way of swapping geometry. Should work fine.


yes it would have to do it, but it only works at the maya interface and maya renderer, when you render with mr no LOD is rendered. and so on, if you are ussing instances, all the instances changes at the same time deppending on the distance of the first intance to the camera, it is not a solution


two quad cores with 20 Gb of ram, but the ram usage is under 5gb
and a quadro fx 5600 with 1.5 gb.

the total amount of triangles is over 9 000 000 000. and i will try to go up to 500 000 000 000.


In 2009 MR supports LOD(I see you’re using 2008 so not an option for you). I’m not sure how you would handle the instancing with LOD though. Looks like the BSP settings will be the best way to go.


i think the LOD are managed by maya and not Mr


but ussing mip_proxies there is not a problem, you only have two cubes (one for the leaves and another for the trunk) for each tree, and the memory ussage is very low with 9 000 000 000 triangles just about 4.72 gb interface included, 256 mb more than just before to start the render, the problem is the stack management of maya, if you use a high number of nodes the interface becomes very slow.

i think that in maya 2010 instances and mr_proxies (assemblies) woks also.

i will do some tests


Sir. Can I call you Jesus.

I’ve been banging my head on the desk for a month trying to get mass amounts of mib_proxies to render, thinking it’s the BSP data that shoots it through the roof. But it’s the scanline rendering that shoots it through the roof.

Turning off scanline my render went from 5 gb to 500 mb !

I had no idea, why doesn’t autodesk document this shit!?

Thank you very much.


Could someone tell me aporiximatly how many triangles could i render by using mr proxys/BSP2/raytrace on a 64 bit machine with 12 gb of Ram (regradless of the quad core processor) ?


3000 500k trees were only using up about 300-400 mb of RAM.

So as a very rough approximation you could say maybe 80,000 500k trees. Just one tree. Nothing else

But getting to that level, I think you might run into a mental ray bug before you run into an actual memory limit.


yep in my previous post the people from itoosoft have tested mental ray

We have found problems in scenes using above 100.000k trees of 500.000k triangles each one (that really is a lot).


not at all rygoody, it has been a headache for me too during the last 3 months.

thanks for do the test with assemblies, i didn’t like the assemblies, they only can handle the original shaders that where assigned during creation, with instances you can change individually the shader of each trunk or leaves.

i have done a test with diferent models of trees and i have notice that the total number don’t grow so much the memory ussage, the mater is the number of diferent trees, here you have a test with 5.000 trees (instances) from 5 models, the total polycount was about 4.500.000.000 and the memory ussage, interface included, was always under 8gb.


Well that’s why we have Jeff :wink: