Octane computer... 4 gpu on z270 mobo - 7700K... possible?


#1

Hi, the title say it all.

I got a budget (5000$) to build a Octane/redshift workstation. However I use After Effect a lot and I dont want to compromise Ghz for more cores.

Most people go the x99 way, thinking that more cores is best but if I use the GPUs for rendering, more cores will not help me. I use a 24-cores z820 right now. All the renders are done on cpu.

I’m watching this motherboard:
https://www.newegg.ca/Product/Product.aspx?Item=N82E16813132953

As you can see, it have 4 gpu slots. However in the z270 specs, they say 1x16 or 2x8 or 1x8+2x4.
Nothing about 4x4.

I dont think there is any cpu on x99 that can beat the 7700k for after effects without overclocking.

Anyone can advice me??
thanks


#2

The socket 1151 platform is the wrong way to go for a GPU rendering workstation. There are not enough PCI Express lanes for the cards to communicate. That processor can accommodate only 16 lanes (regardless of the number of slots on the board).

https://ark.intel.com/products/97129/Intel-Core-i7-7700K-Processor-8M-Cache-up-to-4_50-GHz

So all of the GPU would be sharing the available lanes that would otherwise normally be used for a single GPU. Something like the i7-6800K on the socket 2011 platform has 40 PCI Express lanes.

https://ark.intel.com/products/94196/Intel-Core-i7-6900K-Processor-20M-Cache-up-to-3_70-GHz

If you’re wanting to save some money the Ryzen 7 series processors have 24 PCI Express lanes and have respectable single core performance for things like After Effects.


#3

I was thinking Z270 mobo had 24 lanes…

‘‘Useful for storage fans, however, is the addition of four extra PCIe lanes to the top-end Z270 and Q270 chipsets, for a total of 24. With high performance storage having moved to the PCIe interface over SATA (see the NVMe-based Samsung 950 Pro), PCIe lanes are quickly becoming scarce. Combined with the 16 PCIe lanes on the processor, a Kaby Lake system now sports a total of 40 PCIe lanes.’’

I’m I mixing things up??

The other options would be the i7 6850K but it would be 20-25% slower in AE…


#4

I think that a bandwidth of 4 pcie lanes (~4GB/s) per gpu should be enough for rendering. Isn’t that right?
This motherboard should do the job https://pcpartpicker.com/product/fN98TW/gigabyte-ga-z270x-gaming-9-eatx-lga1151-motherboard-ga-z270x-gaming-9

Of course, with a budget of 5K$, I think that the X99 platform is the only choice for this use right now.

Something like this PCPartPicker part list
with 4 of these https://pcpartpicker.com/product/gwJkcf/evga-geforce-gtx-1080-ti-11gb-sc-black-edition-video-card-11g-p4-6393-kr.

Don’t worry about the single core performance. You just need to oc the 6850K at a reasonable frequency (a 4.4GHz should be easy to get with this AIO cooler), and you’ll have a single threaded performance relatively close to a 7700K running at stock speeds (~10% slower).


#5

Thanks FatherN for the detailed answer.

1 - I cannot just overclock one core? I just need 1 bloody core for AE!! This will be my workstation for few years. I need to take care of the cpu.

2 - can I stack Founder Edition 1080 Ti without them being Hybrid? because the TI hybrid are not out yet… Waiting will be difficult as the boss is exited right now. He might substract the budget if I wait.

3 - If I go x99, the goal would be to add even more cards in the future with raisers. I guess I should take a crazy PSU. The mobo you suggested can run 6 cards I see.

4 - I was thinking about this case: Thermaltake Core X9 - people say good things about it. I need a case that can take all the hybrid cooler fans.


#6

FatherN, if I want to overclock the CPU, I need special RAM?

I wanted to take my z820 workstation memory but its DDR3. I need to add 64gb in my budget. Is the G.SKILL Aegis ok?

https://ca.pcpartpicker.com/product/rntWGX/gskill-memory-f42400c15q64gis

Another question: With the big Core X9 case, lets say I stack 4 1080 TI. They would run very hot stacked like that. Could I not use risers and take 2 of them somewhere else in the case? I would rather have 1080 TI cards than Hybrid 1080.


#7

Risers and ribbon cables for PCI Express are very finicky. The cables from 3M are the only third party ones I’ve ever seen run reliably at full speed. They are around $100 each. Don’t waste your time with $10 cables on eBay, they don’t work reliably.

https://www.digikey.com/products/en?mpart=8KC3-0726-0500&v=19

You might want to consider specialized hardware designed for this. Especially if you want to run six (or more) GPU in a single machine. I’ve used the first one listed below but not the second one.

Up to 4 GPU - https://www.supermicro.com/products/system/4U/7048/SYS-7048GR-TR.cfm
Up to 8 GPU - https://www.supermicro.com/products/system/4U/4028/SYS-4028GR-TRT2.cfm


#8

Here a link to someone running 4 1080TI founder edition without any water cooling:
https://render.otoy.com/forum/viewtopic.php?f=9&t=59887&start=60

And here a link about temp and noise of this model. They say it does not matter where the card is. Inside or outside the case: http://www.tomshardware.com/reviews/nvidia-geforce-gtx-1080-ti,4972-7.html

However, no information on stacking them on full load!


#9

Ok, here the built:

Intel Core i7-6850K 3.6GHz 6-Core Processor
NZXT Kraken X62 Liquid CPU Cooler
ASRock X99 WS EATX LGA2011-3 Motherboard
G.Skill Aegis 64GB (4 x 16GB) DDR4-2400 Memory
Samsung 850 EVO-Series 250GB 2.5" Solid State Drive
EVGA GeForce GTX 1080 Ti 11GB Founder Edition Video Card (4X)
EVGA SuperNOVA P2 1600W 80+ Platinum Certified Fully-Modular ATX Power Supply
Thermaltake Core X9 ATX Desktop Case

Correct? I wonder at the smaller parts I will need… (fans, cables…)


#10

The guy in Otoy forums sais: "Air cooled, turn the fan up at 75% and the max temperature is 70C for continuous Octane rendering. It could be lower if you crank the fan up. "
The highest temperature are the first two cards, the last two are in 50-60 something.
As I noticed, he runs his gpus slightly oc’ed too. So, imo, if you leave your gpus at stock speeds, you should have very decent temps while rendering.

The build you posted is like the one I suggested, with the exception of the chassis and RAM. If you want this Thermaltake, go for it. As long as it fits your needs and aesthetics, it’s fine. If you chose it because of it’s liquid cooling capabilities, then you already have the answer. No need for hybrid gpus. The FE 1080Ti’s would run cool enough, even if you install 4 of them stacked upon each other. So, my opinion is that you should get a chassis smaller in size, but airy enough to cool your system.

Why do you want the G.Skill RAM? This kit is more expensive and runs at lower speeds than the one I’ve posted before. Any reason for this particular choice?

Why a 250gb ssd? With your budget a 500gb should be a minimum imo.

As for your previous questions, yes, you can overclock each core at different speeds, but let me start by saying that you shouldn’t be so afraid of oc. I have my cpu (5960X) at 4.2GHz and with the “Per Core” frequency feature of the BIOS, I run the first 2 cores at 4.4GHz for better single core performance. The Vcore (which is the main Voltage you should care about when oc) is set to 1.22V and everything runs smoothly for 24/7. As long as you feed your cpu with the correct voltages and keep it at low temps, it will be stable for a long time.


#11

I admit the case is too huge for my taste, however I was thinking about adding more cards. Where could I put them in a normal sized case??

I’ll start with 4 cards. That mean all the back slots will be used. How can I add 2 more cards in the future?? Is there some kind of brackets for that?

This RAM was the cheapest on PCpartpicker Canada… I guess in the US you have better deals!

For the SSD, I already have a 500gb Evo Pro on my currant built that I will use as my C drive in the new one. But maybe I should get a M2 drive for the C drive and keep the 500gb for HD footage and .Tiff renders.
I use a lot of footage, often uncompressed, in After Effects. I work at a TV station.

Yep, a M2 drive would speed up things nicely I think… but for the C drive or my footage drive?!

Going back to the case, I need something that will give enough place for 6 cards. If you have any suggestions, please let me know! I will check again at your first case.

Overclocking!! Excellent. 2 cores at 4.4ghz would be amazing.

A big thanks for the feedback, it really help.


#12

Ok, the parts list is gone to the technical director. I’ve changed the case for this one:

https://www.newegg.com/Product/Product.aspx?item=N82E16811139060

And the ram for Corsair Vengeance. Indeed the Aegis RAM have bad reviews.

Any fans or add-on I should take into account? I think the cooler come with thermal paste already?..


#13

If a business is footing the bill I would go with this.

https://www.supermicro.com/products/system/4U/7048/SYS-7048GR-TR.cfm

There’s no point in mess around with gaming stuff and consumer stuff when it’s for business. If you run into issues with that many GPU or BIOS issues or whatever ASRock isn’t going to give a shit.


#14

Thanks Olson, it’s for a business but they are really cheap. They can go spend 25 000$ on a shooting that will not even pass at TV but when we talk to them about upgrading a station, they look at us like spoiled babies.

Already I’ve made a mistake… I though I only needed the C4D plugin for Octane. But I need the stand alone too… I have to go tell them now.