View Full Version : Render farm - same builds, different render times?
07 July 2012, 05:01 AM
I've put together a set of 5 maya render nodes. They are All core i7s with 8gb RAM and the same cheapie video card and 500gb sata hdd. All were built and configured at the same time.
My setup, I have 2 wired into a router which my network storage is wired into - ill call them render 1 and render 2. The other three are connected through an 8 port switch which connects to the router. I'll call them render 3, 4 and 5.
I've checked all 5 and they have gigabit connections. I can copy files off of my network storage at roughly the same speed.
So the problem I'm having ... I am running my renders with backburner, render 1 is managing and rendering, all the others are simply nodes. They are rendering off of the same set of project files, which are on my network storage, and are rendering back to the network storage. The render times on render 2 are running about half as long as on my other machines ... By half I mean 1, 3, 4 and 5 take 16 min for a frame, and 2 takes 8 min.
I've Tried to isolate why so I can pass those speed gains on to the other machines but I keep coming up short. Where can I look? How would you troubleshoot this?
Thanks in advance.
07 July 2012, 09:01 AM
have you monitored that all active cores and threads are active and none of them are running out of memory - in case something else is running on the slower machines?
I'm assuming all settings in the bios is identical?
Are any of them running significantly hotter than the others - maybe the cpu is throttling to become slower?
07 July 2012, 12:54 PM
There are many factors that could affect the performance. It could be a hardware issue like cooling or a BIOS setting (like Hyper-Threading or extra cores disabled). Running a CPU intensive benchmark like Cinebench could shed some light on that.
It could be a networking issue that only shows up when all of the nodes are hitting the share at once. Maybe try copying large amounts of data back and forth on all nodes at once without rendering anything. Depending on the queue manager you use this could be run as a job.
Or it could be a software thing like the queue manager checks in only so often and the nodes are idle for a while between renders. So they might render all at the same performance but they don't return the results right away.
The key is to isolate the different factors as much as possible while troubleshooting. The more factors you eliminate the more apparent the issue will be come. By the way, I work for Tribune as well, private message me if you want my work email. :thumbsup:
07 July 2012, 10:52 AM
Run cinebench 11.5 on each one and see how the times compare.
07 July 2012, 01:32 PM
Just copy the same MAx files on each machine and test rendering times separately.
They should be identical.
- If one is faster than the others then check the cpu use from task manager whether some other application is consuming the cpu.
- If on all machines, 3dsmax is using %100 of cpu and one machine still more than %5 faster then check the bios options, or any overclocking tools present in windows.
Speedstep technology or similar settings may slowdown your cpu to a lower frequency.
It has hppened to me - lucky like you- I have the exact same machine next to the slow one and figured out dumb Asus overclocking tool slowing down the frequency and multiplier, though it reports a higher value. You have to click apply to set frequency settings everytime you enter windows. (I understand the slow speed from fan sound as well)
07 July 2012, 11:49 AM
one other thing to consider is bios upgrade.
I had similar thing couple years ago on couple identical HP workstations (xw6600, xeons).
Everything identical but 2 of them where rendering twice as long as the other ones.
After bios upgrade everything went fine and they were fast as others as well so that can be one thing to check out too, bios versions.
07 July 2012, 11:22 AM
Your issue may be caused by several factors.
If I understand you right, you have the problem using Maya. Here some possibilities from lowest level upwards.
1. It may be the cards, even if they are the same model, they may not have the same bios version and so they use the CPU different.
2. The processors AMD Opteron from series 6000 have an energy saving routine which sets them to 500 Mhz when the system considers that they do nothing. You should deactivate that because it may slow down render. (It has to be done in bios).
3. When you render in Maya through Backburner, it is done through Maya.exe using its command -batch which, in certain circumstances, uses the configuration from the file C:\Users\root\Documents\maya\2012-x64\prefs\userPrefs.mel. Between line 1845 and 1847 you find this:
-iv "v_mrBatchRenderNumThread" 2;
-iv "v_mrBatchRenderNumThreadAuto" 1
Those two lines have influence over the use of the machine under certain conditions.
4. In case that you have more than 32 cores, in your machine you can set restrictions concerning nš of cores collaborating in a process, at OS level. You can determine this using the Task Manager during the render process.
Donīt know if I could help you or if I tangled you even more.
07 July 2012, 11:22 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.