OT: Painless RenderFarm Deployment?

Become a member of the CGSociety

Connect, Share, and Learn with our Large Growing CG Art Community. It's Free!

THREAD CLOSED
 
Thread Tools Search this Thread Display Modes
  12 December 2012
OT: Painless RenderFarm Deployment?

Let's say you have 20 computers in your render farm.

On PC1, you install the OS, MAX, Adobe products, & loads of other animation/compositing/graphic apps. Now this system is totally setup & functional. You use disk image software to make an image of this functioning system.

Is it possible to then burn that system image onto the disks of PC2 thru PC20 and then do some authorization magic and have the OS and MAX and other apps be properly authorized and function properly thereby making the setup of the farm painless & joyful?

Or is the only option to painfully have to spend days installing all OS & apps on all systems? (As I have had to do in the past when upgrading the OS on the render farm).

Thanks for any ideas!
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  12 December 2012
I guess it depends on if you have 20 licenses of some of those things. Max provides you with render satellites, so you only need one license... Assuming you don't have 20 licenses of the other stuff, (well, you're going to have to have 20 Windows keys), I would do it this way:

Comp 1, workstation, would have the full versions of everything installed.

Comp 2, you would install your 3dsmax render satellite, and all the plugins/items it will need.

Then, disc image Comp 2, and install it on Comps 3 - 20. Then you'd have to activate Windows on comps 3 - 20 with your keys. Plus you'd need to do this for any other software that requires a key per machine.

Does that help? Good luck!
__________________
www.zephyrstar.com
 
  12 December 2012
Yes it does help... thanks for the info!

My question really relates to if it is possible and others have definitely done this and what to expect during the process.

I would have 20 Windows7 licenses (keys) and whatever other app licenses needed to be legal, etc.

Let's separate this out a bit.

OS
When I have burned the image onto all 19 other systems, what happens when I then go and start one of the 19 systems? The system starts but Windows7 complains that the system MAC # doesn't match the OS key installed? It then asks me to input a new key? I do so and then it updates the registry with all the proper MAC/KEY info and then Windows7 starts and runs properly on this system and I do the same for all other systems? This is what I would hope.

Apps
Now Windows7 is running fine on all systems. The systems will be primarily render slaves so in the case of MAX, it will just run fine in net-render mode? I thought that there would be some MAX stuff in the registry that would need correcting to match the new system MAC#? Seems like when you enter the license key during normal install it would be tied to the MAC# in the registry and this would not match up to the new system therefore MAX would not function fine.

Likewise with Composite for net-rendering?

Combustion for net-rendering?

Adobe After Effects net-rendering?

Even though these systems are primarily net-render slaves, some of the systems run apps in workstation mode such as encoding software or whatever. Any software that I have a dongle for, the dongle will be plugged into the appropriate system. But (even with the dongle apps) still it seems like there would still be incorrect stuff in the registry that would need correcting so things would be tied to the new system's MAC#?

The real point of all this is, if I can simply re-authorize and be good-to-go... that's what I want. If I have to painfully uninstall/reinstall apps in order for them to be properly functional on the new systems then burning disk images is pointless. (Same if the re-authorization/registry repair is painful-time consuming).

Hopefully someone has done this exact process before and can say it does work for sure.

Thanks again.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  12 December 2012
I would install, but not license any of the OS tools, etc. Most should provide you with 30 or so days to license. I would think it is much easier to license after the fact then try and get licenses sorted out, etc. Some tools will require proper presetup before being distributed. For example with Backburner you will need to clear the MAC Address entry from the Backburner.xml to prevent the MAC Address you described earlier. Or place a clean version of the file on a network location and copy it to each machine before starting the server/managers.

You should invest some time into planning out your deployment. How does each machine need to be setup and configured? Will the machine be a slave or a workstation?

Windows 7 Deployment Guide

-Eric
__________________
"The Evil Monkey hiding in your closet."
 
  12 December 2012
Thanks for the ideas Eric. After reading the OS deployment guide, for just 20 systems it might be easier in my case to just install manually. I know that installing MAX would allow a trial installation but am not sure about all the other apps. If I was smarter about IT stuff I'm sure it would work but my ultimate goal is getting the pipeline updated as fast as possible without losing the functionality of the current pipeline. Guess I'll have to update in phases to make that happen.

Thanks again.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  12 December 2012
I think you'll be fine doing windows 7 that way, but as far as having a full install of max on the render sats, it will probably be easier to just install the render sat on each one of those nodes. You can also run them headless after initial setup and use RDP to log into them and administer them.
__________________
www.zephyrstar.com
 
  12 December 2012
We've done this with our 10 node render farm without problems.

First we installed one system completely... Win 7, 3dsmax, Backburner, V-Ray... and I added another network-based folder of plugins to the plugin.ini of 3dsmax, so that any free plugins that don't need any installation could just be added to that one folder on the network-server and all the nodes could load the plugs from there. So there's no need to copy them to each node individually. V-Ray's license server settings were set up so that they pointed to the machine in the network that got the dongle, plus Backburner Server was set up to point to the correct Manager machine in the network. Also we set up the IP address of the machine to be fixed, and the machine's name to make sense (numbering your nodes along with your IP address is probably a good idea).

Then we cloned the hard drive to each of the nodes (which hardware-wise were all identical). Now the important part:

First you have to change the IP addresses of all the machines to be unique. Not only in Windows' network settings, but also in the settings of Backburner's Server.exe. ALSO you have to change the MAC address in the backburner.xml file (which you find somewhere in your Users-directory), to the MAC address of your new machine. (Because we cloned the first machine, the MAC address in the backurner.xml is identical to that machine). You can find your machine's MAC address in your network status window.

That should be about it. Probably you will have to confirm some firewall requests during your first accesses to the nodes.
 
  12 December 2012
You could use VBS or Batch Files to Automate the installation process. That is what I personally have done. Place all the installs on a Network Location and have the Batch File or script copy and install the files as needed. With 3ds Max you can create a deployment image and run that on the machines that should automate that install at least.

You could always try searching for deployment related stuff to find items like Adobe Deployment Guide and Network Deployment using VBScript.

-Eric
__________________
"The Evil Monkey hiding in your closet."
 
  12 December 2012
Thanks to all for the ideas on this.

Laserschwert: Glad to hear that it is doable regarding Windows7 & the MAX stuff. Following your description, I think I could get this part to work properly. The other apps might take a little more looking into or have to be installed manually.

Each of the slaves are not identical but are pretty similar in that they all currently have 2 HDDs, DVD drive, 2 network cards and are all Dell servers but they probably have different MBs and controllers & network cards so this might be an issue for Windows7 to work properly immediately.

Also wondering how I would move the disk image around? Burn to a HDD and then attach it in place of the #2 drive on each system and copy onto the #1 drive?

Oh, and all the slaves currently use fixed IP addresses and matching numbered names.

Thanks again.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  12 December 2012
Originally Posted by Mike Truly: Also wondering how I would move the disk image around? Burn to a HDD and then attach it in place of the #2 drive on each system and copy onto the #1 drive?

Not sure what you mean with "burning to a HDD", but most backup tools should allow you to mirror your harddisk to any other harddisk that's attached to the computer. So just setup node 1 and plug in the HDD of node 2 and mirror to that. And when node 2 is up and running as well, you use that one to mirror its HDD to the HDD of node 3, while node 1 will mirror to node 4. After that you have nodes 1 - 4 done. Use those to create nodes 5 - 8 and so on. I assume there are tools out there which allow you to mirror to more than one HDD simultaneously, so that might save some time.

But: With your nodes all having different hardware, I'm not sure how well mirroring works. You probably have to install different drivers to all of them, so that might be problematic.
 
  12 December 2012
Laserschwert: Thanks again for the ideas. Sorry, just meant copying the disk image to the HDD. Although the systems are not identical, they are very similar. They were purchased at different times in batches. So 4 are identical, then the next 4 are identical to each other but different than the original 4 and so on. But for example, even though the MBs and controllers are slightly different on the different batches, they all use eSATA HDDs (hot swappable).

I just wonder what would happen on the differences? Would Windows7 would fire up and say 'theres a problem with the controller driver... please insert your Windows7 install disk' then it would look for the correct driver? I could provide the proper one from Dell if it wasn't a standard Windows driver.

Not sure.

Oh and one of the systems is a bit different than the others in that it runs Windows Server 2003 R2 and serves texture maps during net-renders and receives the rendered frames from the slaves. It has 25 CALs so as to enable that many network connections simultaneously. In order to update that system to be on par with Windows7 (this machine is also a render slave) I would need to upgrade it to Server 2008 or something. (But my CALs would be wiped out by a new OS install and I would have to re-locate the CAL software to re-install them).

The point of all this is to upgrade the OSs on all these systems so that future software installs don't refuse to install because the OS is XPx64... like the Adobe products and soon MAX.

Thanks again.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  12 December 2012
to copy the images I would recommend using clonezilla.

It's a bootable standalone open source software that lets you mirror your harddrive using images / partitions.

Very useful for backups too.
 
  12 December 2012
Alexander,

Thanks very much for the recommendation! Will check it out.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
  05 May 2013
Mike, I just did this over the weekend and it worked great.
There were some hiccups as I did not research it far enough to realize I would need to edit the backburner.xml files on each node after starting them up.

One that was take care of, things rolled along nicely.

I didn't realize you had a render farm.... you continue to surprise me
David
__________________
Nothing to see here

www.dustinforensics.com
 
  05 May 2013
Hi David,

Glad to hear you've been successful setting up the farm via the disc clone method! I'll definitely have to pick your brain before I update mine. Yeah, I've had a render farm for many years now and it's been working great... that's why I hate to disturb it. I think I'll buy some new HDDs and setup the new OS farm on them so I can re-insert the old drives if I have problems and I need to quickly get back to a functioning XPx64 farm.

But these upgrades are on hold for the moment as I am buying a winter home down your way (actually down in Florida) and am dealing with all the monetary/logistical hassles of transporting my computers, etc. back and forth between homes. At some point I'll be upgrading my systems but not just yet.
__________________
Sincerely,

Mike Truly
-----------------------
Truly Media
970.349.5651
www.trulymedia.com
 
Thread Closed share thread



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 06:15 AM.


Powered by vBulletin
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.