PDA

View Full Version : help- mental ray disributed rendering workflow


yondon
06-01-2009, 03:15 PM
Hello People,
First excuse my poor English. I'm from Israel so English is not my native language..

The situation in short:
For the last two years I'm working on a short film with another friend.
We're going to render it in HD720, Using Mental Ray for maya.
we're using pretty large texture files and shave and haircut for our characters, so the render process is pretty much demanding.

We have 4 pcs to render- 2 which we already working on - let's call them masters
and 2 more the we getting in few days. all 4 of them equipped with quad core CPU and 8GB of ram and connected to a 100MB network.

we want to know how to organize the project before start rendering. how to take advantage the best possible way of our hardware.

1.should we use simple batch render from within Maya or should we get a render manager?

2. Is it better to send render from both master computers or should we have only one master that is sending renders to all other machines?

3. about texture files-
we're using textures that most of them size between 4096X4096 Pxls to 2048X2048 Pxls in Jpeg format. Do I need to convert them all to MAP files? will this be a major benefit?

4.About networking rendering and those texture files-

One of the master machine is hosting all the project and the texture but there are so many files.
There won't be a problem with all this network/reading traffic?
Is it better to copy the textures to each slave? and if so, than how do i tell them to use those local files instead of the files in the original project on the master machine?

any inputs and tips on this process will be a great help. I'ts the first time we're doing it, and we almost don't have time..

thank you all
Yonni

chronic
06-01-2009, 04:00 PM
invest in Gigabit Ethernet its really worth it.

the easiest option is to host all the files form one machine, with gigabit ethernet the traffic load of pulling textures and sending rendered frames across the network is not a problem. getting an automated network rendering software is great, and pretty easy to set up.

chronic
06-01-2009, 04:04 PM
for an estimation of network traffic an load, calcuate the total size in MB of textures used by each scene. if each scene only needs to pull 10MB of textures for each frame, 100 ethernet is not too bad, but if you are pulling hundreds of MB each frame across 4 machines you are limited by the network and disk access speed on the host machine.

.map files are not necessary unless you are bumping up against your physical ram limit (8gb on each machine?)

yondon
06-01-2009, 04:30 PM
I guess we have hundreds of MB's in textures for each scene.
So if i understand, you say 100MB network is not going to help us?

Is Gigabit Ethernet the same as router?
What exactly do we need to purchase? can you share some links? do we also need to buy another network card or different cables for each machine or the one that comes on board will work?
or again, is it better to copy all the textures to each slave?


and yes, 8gb on each machine. what do you think about converting to map files?

InfernalDarkness
06-01-2009, 09:10 PM
For your personal reference:

You keep referring to 100MB networking, and I understand that you mean 100Mb, which is mega-bits (not bytes). A byte is 8 bits, of course, so a megabit (Mb) is 1/8th of a megabyte (MB). The distinction is that 100Mb per-second, or what your network speed tops out currently, is NOT 100MB (megabytes) per-second, but 1/8th as fast as 100MB/s would be. So keep this in mind when doing your math.

Gigabit ethernet will require a new router and likely new NIC cards for your computers, but if they're new enough then it's likely they'll support Gigabit speeds already. Just get a new router and you should be fine. A gigabit/s is of course one billion bits per-second, or one eight of a gigabyte per-second top speed transfer rate. This will help vastly for you if you're moving that many textures across the network.

Does your host (master)render the scene well? Do you run into RAM limits or crashes from mental ray? If not, then I'd say you don't need to convert to .map files just yet. If you're using all 8GB of your master machine to render one frame, then definitely switch them to .map textures.

yondon
06-01-2009, 09:27 PM
Hey Darkness,

Thank you for your kind explanation about the MB\Mb thing/
Always good to learn new stuff, specially that basic :)

Can i just purchase a Gigabit Switch for the networking, while keeping the 100 Mb router for internet?

Cheers

nfrancisj
06-04-2009, 07:56 AM
Hello People,
First excuse my poor English. I'm from Israel so English is not my native language..

The situation in short:
For the last two years I'm working on a short film with another friend.
We're going to render it in HD720, Using Mental Ray for maya.
we're using pretty large texture files and shave and haircut for our characters, so the render process is pretty much demanding.

We have 4 pcs to render- 2 which we already working on - let's call them masters
and 2 more the we getting in few days. all 4 of them equipped with quad core CPU and 8GB of ram and connected to a 100MB network.

we want to know how to organize the project before start rendering. how to take advantage the best possible way of our hardware.

1.should we use simple batch render from within Maya or should we get a render manager?

2. Is it better to send render from both master computers or should we have only one master that is sending renders to all other machines?

3. about texture files-
we're using textures that most of them size between 4096X4096 Pxls to 2048X2048 Pxls in Jpeg format. Do I need to convert them all to MAP files? will this be a major benefit?

4.About networking rendering and those texture files-

One of the master machine is hosting all the project and the texture but there are so many files.
There won't be a problem with all this network/reading traffic?
Is it better to copy the textures to each slave? and if so, than how do i tell them to use those local files instead of the files in the original project on the master machine?

any inputs and tips on this process will be a great help. I'ts the first time we're doing it, and we almost don't have time..

thank you all
Yonni


all that really matters is memory!! MentalRay is a memory hog. it seems to me that having such large maps in addition to shave&haircut will cause slow downs. keep an eye on the sampling also. i reccomend outputing in layers and compositing them in nuke or shake. you should get a render manger...batch script is ok..until it crashes..(no log to look at to see why it crashed) Set only 1 master. in my experience...shave uses only 1 core to render...which means you can render 4 frames at once. Map files are good also...saves a little time..mR won't need to covert them for use... I think your net traffic will be fine. I did something similar with 12 IBM blades on a crappy network. everything gets loaded into memory anyway...so it won't be too much for your network...well...until the memory is full and it starts writting to disk

heres a tip for you... If it renders on 1 machine in 1min.. it will do about the same on the network.( not taking into account whats being rendered and how much of it is visible)

let me know how things go.. and when you release your short. I wanna see... :)

Cheers

-N!K

InfernalDarkness
06-04-2009, 07:06 PM
I'm not sure if I understand what you meant by this?


heres a tip for you... If it renders on 1 machine in 1min.. it will do about the same on the network.( not taking into account whats being rendered and how much of it is visible)

If it took an entire network the same time to render a scene as one single workstation, people wouldn't use renderfarms... Perhaps I am misunderstanding?

yondon
06-04-2009, 07:41 PM
Hi again,
I have another disturbing issue with my renders -
At the moment, I am rendering with Mental-Ray for Maya using Batch renders.
For now, my renders are local and use a quad core carrying 8 giga ram.

Unfortunately,My computer does not use more than 2\3 giga's
(only the toughest Shave & Haircut scenes exceeds the 3 giga's...)
and my renders are around 20 minutes a frame.......

is there any way I can get my renders to use more of the unused memory ?
will Map texture files help ?
any suggestions for the Mental ray settings maybe ?

Thanx !

Yonni.

yondon
06-04-2009, 07:45 PM
let me know how things go.. and when you release your short. I wanna see... :)

Cheers

-N!K

Thanks Nik :)
I really hope to make it by mid July :curious:, Hoping for the best!
Definitely I'll be happy to share it with you all !

Cheers

InfernalDarkness
06-04-2009, 10:29 PM
is there any way I can get my renders to use more of the unused memory ?
will Map texture files help ?

If you're already using a 64-bit OS, then you should be fine. It simply means your scene doesn't cap out or require as much memory as you have. That's about the best "problem" you could have, in this industry!

But if you're not using XP 64-bit, Vista 64-bit, or Windows 7 64-bit, then you'll be stuck below 4GB of RAM usage no matter what. A 32-bit Operating System can only address up to 4GB of RAM. To access more, you'll need to switch to x64.

yondon
06-04-2009, 10:52 PM
If you're already using a 64-bit OS, then you should be fine. It simply means your scene doesn't cap out or require as much memory as you have. That's about the best "problem" you could have, in this industry!

But if you're not using XP 64-bit, Vista 64-bit, or Windows 7 64-bit, then you'll be stuck below 4GB of RAM usage no matter what. A 32-bit Operating System can only address up to 4GB of RAM. To access more, you'll need to switch to x64.


Thanks darkness,
Well we do use Vista x64 so I guess we don't have a problem?

It's just weird.
I thought we will need much more memory rendering this film so we upgraded each machine to 8gb. luckily Ram are so cheap right now :)
And I was certain that there is something we can do to make MR render faster by using more memory..

thanks so much

CGTalk Moderation
06-04-2009, 10:52 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.