Need advice about Xeon render blade please

#10

What software will you be using on the farm? Cinema 4D, Maya, After Effects, Nuke, etc.

#11

yeah, if you’re only getting 6 blades (which are going to be expensive), I wouldn’t bother getting blades. Surely you have enough room in your studio for 6 mid tower cases. With the money saved, you could easily get probably 8-10 non-blade machines of equal performance. Xeons are expensive either way, but you pay a small premium for them being shrunk down so small into blade rackmounts.

#12

Do you have space problems?
Or do you already have a climate controlled area for things such as servers with the racks already set up?

High density is rarely the way to go unless you tick both the above boxes.

A taller and easier to climatize set up with more and more straight forward networking might see you better off.

Also what engines and requirements?
Once you get a wider array (many CPUs and cores) and a lot of it is dual or quad procs, you also have to bear in mind they share memory, and, guaranteed, you will at some point want to split jobs more granular so you can have multiple running per CPU.

If your average job now can cap 24GB on a workstation, for a farm with duals seriously consider at the very least 32, if not more.

Wattage is also important.
How expensive is power there?
High density means lower power consumption per cycle, but higher power consumption in cooling per watt produced (residual heat means cooling can’t ever be lazy, which means it hardly ever cycles off).
If you have expensive power bills, but an already under capacity climate controlled area, they are great. If it’s the opposite, then you might want to go lower density with a taller and easier to cool rack.

Just keep these things in mind.

Other things like cost of licenses and licensing schemes also contribute.

A farm doesn’t finish costing you money once it’s set up, it just starts there, and people often under-estimate how impactful a poorly chosen one can be on your bills for power and software.

#13

Oh so RAM are shared? Do they share power supply and NIC cards too?

@sentry66, @Jaco thanks for your reply, really got me thinking. Apart from saving space, are there any major pros for going blades? We have no space problem. Actually, we have lots of space. And since our country is really hot, our room is already air-conditioned.

Would an air-conditioned room considered climate controlled? Is there more to it than that?

I’m thinking the farm would be used primarily for Vray which I believe that Vray for Maya comes with 10 distributed/standalone license. Some of them may also run 3Delight and PRM, it all depends on our investor.

#14

What they share depends on the build, type and so on.

PSU is usually shared as in you only need one plug per blade, and internally it will draw and power what it needs to, but there are many offers where you have two, and good blades often have redundant power supplies in case one fails (which is something you can put in any server tailored case too, if you need to).

NICs depends, some have multiple regardless of the number of CPUs and mobos (plates) hosted, some have one plug and deal with splitting and offering multiple IDs in a managed way, some will have fiberchannel too.

Climate control is about the inside of the chassis being at reasonable temperatures, so if you have an air conditioned room hosting forty workstations comfortably, it won’t be a problem to add 10, whether you use them for distributed rendering only or seat someone in front of it it matters absolutely nothing, if they don’t overheat, they will keep churning frames out. That’s the whole extent about climate control.
With racks you have to be more specific and careful because it’s a lot of heat in a small space, but the principles don’t change.

Workstations are, from a power and running costs point of view, very rarely advantageous over more compact solutions.
They will be cheaper in terms of casing and management, but they usually aren’t as optimized heat and power draw wise as blades can be. It doesn’t mean they aren’t an option though. Again, space and power being the difference between a computational centre and a bunch of workstations.

There is nothing magic about hardware inside blades. If it can fit in one, the equivalent can fit in a case if you prefer that.

That’s why I was stressing those points.
People think of a renderfarm as if it’s some sort of magical, abstract entity… It’s not, it’s just a bunch of computers, end of story.

Power, space and computational needs and constraints dictate whether you need one in racks, or you can pile up some cases on a desk.
It’s all about logistics, end of story.

#15

Thanks Jaco. Yes you’re right, at first everyone at my studio (me too) thought a render farm is something specialized. I’m relieved to know that each blades are just like regular PC :slight_smile:

How much faster would a Xeon 2690 have over 2650? The CPU price alone is almost twice more expensive?

#16

It depends at what tasks.
Some you could see something close to time and half, others you will hardly notice the difference.

Whether the 1k per CPU difference is justified, as for other things before, depends on many factors.

If your CPUless blade cost is (for the sake of argument) 800$, and you have no space or racking or network limits, it’s not worth it, because a configured blade costs 5k with the 90s, and 3k with the 50s, making an additional blade better ROI.

If a CPUless blade costs you 2k, and you are license capped with some expensive software that brings in license per CPU as a factor, adding, say, 500$ per CPU, then the 90s will be better value for money.

If you have licenses in excess, space in excess, cooling in excess, and power doesn’t cost a fortune, then it boils down to naked blade costs, and in that case many cheaper CPUs will usually win over few top of the line ones you pay a ridiculous premium for.

Is it worth the money? It depends, as I said before a farm is about balancing all things out from a purely logistical and financial point of view, if someone who doesn’t know your studio and constraints gives you a straight yes or no answer to that question, don’t trust them :stuck_out_tongue:

#17

A a xeon 2690 is at most 35% faster than 2650 at single-threaded tasks and at most 45% faster at multithreaded tasks. In the real world though, performance doesn’t scale perfectly linearly and will likely be lower than what I just said.

Don’t compare CPU price vs CPU price. Compare system price vs system price

#18

Hm? Did you think about pure i7-2600k/ - 3820/ 3770/3930k CPUs?
They have THE BEST money/ speed ratio…

#19

Only if you’re not using an expensive renderer (for example Renderman). Doesn’t make sense to spend $2,000 per render license and put it on a $800 machine because you run up a huge bill in licenses. Since in this case there are 10 complimentary licenses (according to the OP) the single processor nodes would be a good bang for the buck.

#20

Do each node need full Maya license if I’m doing primarily Maya batch? Is there a cheaper license for render node?

#21

This is all stuff you can find on AD’s website, or from your reseller.
From their doco for 2013 (and AFAIK it hasn’t changed for 2014) assuming you mean MRay:

Network rendering using the render command line utility
Each Maya license allows you to render in Maya interactively on one machine and run batch rendering on five machines. You can therefore perform mental ray for Maya rendering on up to 6 machines.

Other engines will be a different deal, and how maya seats are used is up to how things are set-up over there.

#22

Thanks Jaco. I guess the 5 batch machine also applies to Vray. 1 Vray every 10 nodes, 1 Maya every 5 nodes. Doesn’t sound too bad.

Including the software price, building 16x3930k actually comes really close to an 8 blade twin Xeon 2660. Now it’s just a matter of space I guess :open_mouth: I have this impression that Xeon will probably have longer life-span than I7 with more endurance, we might go that route for longevity.

#23

how does the batch vray licensing work anyway?

It still needs maya to be installed in some form right?

I’m just trying to figure out if you want say 20 render nodes and have 2 vray batch licenses so you can render on 20 computers, do you need 4 maya batch licences still, or can you render on all 20 machines with 2 vray licenses with just 1 maya batch license?

Do you need to pay for a full suite of maya batch licenses regardless of rendering engine?

#24

That’s my guess. 4 Maya license and 2 Vray license for 20 nodes (plus their workstations counter part.) We might already have that covered, I don’t think we’re using any batch license right now.

#25

damn, that was the major reason I was considering vray - i thought Vray render licenses would be cheaper than having to pay such a high premium to get more maya MR licenses.

I guess we’re screwed paying for multiple maya batch renderers either way.

#26

When you render on a farm (I don’t know vray in the specific) you usually do so with interim files and referencing correlated to the rendering engine.
ASS for Arnold, RIB for RMan engines, mi for MRay and so on.2

If VRay has a container file, you don’t need batch seats for anything other than generating them. If it doesn’t, well kinda sucks.

@Panupat
Consider depleting your license pool. That might mean 6 dual Xeons and 6 single CPU units (for the sake of example) will already be covered, and, assuming they will cost the same, will give you a better coverage of single CPU tasks (again, assuming you have any).
Just some food for thought.

Also, check that licensing isn’t per CPU with whatever secondary software you use, because in that case the 8 xeons will still count as 16 nodes.

#27

V-Ray can render from an archive (similar to RIB, IFD, MI, etc.). The archive for V-Ray is called vrscene and the officially supported applications like Maya can export them. This means you don’t need a Maya license for every render node but keep in mind it will take time to export the vrscene prior to the render instead of just submitting the Maya scene file to the render farm. Additional V-Ray licenses can be purchased for rendering, they’re something like $100 each if you buy a lot of them at once (20+), or if you buy just one or two they are like $500 each. When purchased in bulk they’re a pretty good deal.

#28

Great information, thanks for sharing. Still have to discuss with the pipeline team if we can come up with vrscene workflow.

Btw, one of the resellers confirmed to me that Vray has no CPU limit per node :beer:

We are checking the price and get back to you soon.  List price is around
US$1400.00 per license with USB dongle.
The connection is ONE Master can link with 10-slave to do rendering, there
are totally 11 nodes.  No. of CPU does not be limited per computer.

That’s from Hongkong resellers… even just 1 license, the price + plane ticket is still cheaper than buying from Thai reseller locally. I might just do that.

#29

There you go, problem solved :slight_smile:

And yes, you might pay an overhead in time taken to write and read and transfer those files, and one in online storage for the life of the rendering files (and if you’re smart and have a decent load balancing you’ll pay the price for those files to be distributed to the blades too), but except for very rare cases (client and engine have truly aligned scene data, which I doubt is the case for maya and VRay) you normally get your money back in ease of management, peace of mind, lower memory footprint, and of course one less licensing headache.