CGTalk > Development and Hardware > Technical and Hardware
To minimize the ads you see on this page create a CGTalk account and log in HERE
Thread Closed share thread « Previous Thread | Next Thread »
 
Thread Tools Search this Thread Display Modes
Old 02-05-2013, 08:59 AM   #1
cdenny
Veteran
portfolio
cd
USA
 
Join Date: Feb 2009
Posts: 37
[Amazon EC2] for only 45c/hr you get this!

I was researching Amazons EC2 program, and had stumbled upon something interesting. With their 'Spot Request' system you can get an instance of cc2.8xlarge (88 compute units) for only 45cents/hr. It stays that price for the most part, spiking to 50cents once a week if that.
I tried using hamachi to create a vpn with my own workstation, and it works with one instance running, but trying to add another from the same AMI doesnt work (still trying to figure out a fix). Important part is creating it as a Windows Server 2008 r2 for clusters. After you configure it you can actually use it on any of their instance types. I am lacking in the whole backburner department so I haven't played with that, but running .bat files is super fast and slick! There are of course data transfer fees but they are negligible for an individual.



As far as I know, this is WAY cheaper and just as fast as the point and click renderfarms. Need it done in an hour? Make a dozen of them, it's literally two clicks once youve created the AMI.
Ask any questions if you want, I'm still learning as I go.
 
Old 02-05-2013, 12:11 PM   #2
imashination
Expert
 
imashination's Avatar
portfolio
Matthew ONeill
3D Fluff
United Kingdom
 
Join Date: May 2002
Posts: 8,741
Nice, Ive been pondering setting up a machine or two for rendering, let us know how you get on.
__________________
Matthew O'Neill
www.3dfluff.com
 
Old 02-05-2013, 12:55 PM   #3
tswalk
Lord of the posts
 
tswalk's Avatar
portfolio
Troy Walker
USA
 
Join Date: Jan 2012
Posts: 708
this might benefit you some... this is to part 3 of his series on setting up EC2 and backburner.

http://vimeo.com/27994567
__________________
-- LinkedIn Profile --
-- Blog --
-- Portfolio --
 
Old 02-05-2013, 04:37 PM   #4
pringleman
New Member
portfolio
Jamie Gilbert
London, United Kingdom
 
Join Date: Feb 2008
Posts: 26
interested in your results...will be following this.
 
Old 02-05-2013, 04:50 PM   #5
sentry66
Expert
 
sentry66's Avatar
portfolio
node crazy
USA
 
Join Date: May 2008
Posts: 1,980
I'm assuming software that requires a license per render machine is incapable of utilizing cloud services like this?

Makes me almost want to jump from maya to 3dsmax so I get unlimited MR licenses....stupid maya
 
Old 02-07-2013, 10:39 PM   #6
NoxLupi
Veteran
Martin Hinrichsen
Copenhagen, Denmark
 
Join Date: Apr 2007
Posts: 59
Unhappy

Okay, for the last 4 hours I have been fighting through buggy non-responsive web-interfaces, failed connection, webpages that never loads and password decryption that doesn't work. And so far, I haven't had the change to see a single instance desktop. That's laughable(in a sad and disappointing way) in my opinion.
 
Old 02-08-2013, 04:15 AM   #7
cdenny
Veteran
portfolio
cd
USA
 
Join Date: Feb 2009
Posts: 37
Quote:
Originally Posted by NoxLupi
Okay, for the last 4 hours I have been fighting through buggy non-responsive web-interfaces, failed connection, webpages that never loads and password decryption that doesn't work. And so far, I haven't had the change to see a single instance desktop. That's laughable(in a sad and disappointing way) in my opinion.


Sorry to hear that, read this and go through it step by step and if it still isn't working I dont know what to say. They provide an image you can use, without having to worry about password decryption and such. It's for C4D but if you get it working you can just install Maya or whatever, change the password to your own and save it as a new image.
http://cosmocyte.com/animation-techniques/
 
Old 02-08-2013, 04:54 AM   #8
AJ1
User
 
AJ1's Avatar
CGTalk Forum Leader
portfolio
A J
Indiana, USA
 
Join Date: Jan 2012
Posts: 1,810
Hey man,

Wow! This looks super cool. I've read a lot about this for rendering, but I've never seen an example of someone who got it to work.

How does it compare to RR or Rebus price wise? Do the licences install ok? Can you think of this like your renting a beastly dual xeon by the hour?

Thanks,
AJ
__________________
 
Old 02-08-2013, 12:20 PM   #9
NoxLupi
Veteran
Martin Hinrichsen
Copenhagen, Denmark
 
Join Date: Apr 2007
Posts: 59
I got around it. had to use internet explorer, apparently they don't support Chrome or Firefox.

Chrome works until you press a link "instances" and then the script dies. and you cannot browse to other functions. and it cannot decrypt passwords.

Firefox just returns HTTP Status 400 - BadRequest. upon login.

Internet Explorer works, however you constantly get a warning box suggesting to close a faulty script.Very very shaky interface!

Thanx for the links cdenny!

I think I know a way to automate the IP setup and login to render manager.
so you only have to prepare the manager, launch the render instances and press render.
 
Old 02-08-2013, 08:12 PM   #10
cdenny
Veteran
portfolio
cd
USA
 
Join Date: Feb 2009
Posts: 37
Quote:
Originally Posted by NoxLupi
I got around it. had to use internet explorer, apparently they don't support Chrome or Firefox.

Chrome works until you press a link "instances" and then the script dies. and you cannot browse to other functions. and it cannot decrypt passwords.

Firefox just returns HTTP Status 400 - BadRequest. upon login.

Internet Explorer works, however you constantly get a warning box suggesting to close a faulty script.Very very shaky interface!

Thanx for the links cdenny!

I think I know a way to automate the IP setup and login to render manager.
so you only have to prepare the manager, launch the render instances and press render.


Good to hear! Interesting tidbit about the browser, I've been using Chrome and haven't had issues with it, I've had no 'script' stuff popup. I'm using the latest version on windows 7 pro x64. As for the ip dealio, I'd love to hear your solution, the only thing I know of, to keep a static ip for the manager, is to create and associate an elastic ip to the manager so it stays the same. this costs $.005 for every hour it isnt associated with an instance, and data out costs 1c/gb of data to the other instances. It works out to be only $3.50 just sitting there.

I was messing with license stuff last night, and I'm not sure how its working so far. I've been using the trial for licensing, but it counts down just like any other (I tried turning off internet time update thinking it would just revert, but it doesn't). I'm still looking into how that all works; the only issue is with creating the job and using the manager, the other instances dont need to mess with it because Maya allows up to 6-8 nodes with mental ray. I'm going to try to create a vpn and use my actual workstation as the manager because I have a static ip, but then the flow of data will be limited to around 300kb/s which is just too low in my opinion. If you have good internet this may not be an issue. One other annoying issue is the naming, I checked the box that says 'use ip <xxx>' for dynamic naming, but the computer name is still just amazon-xxxx or whatever.

Compared to rebusfarm, it's much more complicated, may take more time just to set it up and stuff, but its much cheaper. Based on 3.9c/hr/ghz, a single cc2.8xlarge instance with maya's 45c/hr rate would be equivalent to paying 3.45/hr (about ten times the cost). There are other costs with data usage, etc, but that is nearly negligible at 12cents/gb. As a student this is awesome, but if your a busy professional or such, going with a dedicated service like rebusfarm may still be a better option, because the time you spend setting it up and making it work and dealing with issues might not be worth it. Zync render utilizes Amazon EC2 farm, and while I don't have access to it, I hope their rates are better than rebus farm. I was thinking about upgrading or building a better computer, but I think I'll stick with this EC2 method for now because there is no upfront cost ( which I cant afford anyway).
EDIT: One thing to note is that if you used a comparable machine locally you'd have to pay electricity for it, which would be about 1000watts on 100% load. Around here it's 11c/kwh, so just consider that a saving as well.
I plan on just setting up the renders and then creating .bat files to run on two instances, with the files being synced by dropbox. This may not work with the licensing, I'll find out soon enough.
Good luck!

Last edited by cdenny : 02-08-2013 at 08:56 PM.
 
Old 02-09-2013, 02:59 AM   #11
NoxLupi
Veteran
Martin Hinrichsen
Copenhagen, Denmark
 
Join Date: Apr 2007
Posts: 59
Oh dear... I found the issue! It was Avast webRep a browser plugin.
Now chrome works just fine... Sry, I never thought a simple thing like webRep would screw it up that bad.

In regards to the idea i had it has become quite a complicated mess with wbscripts, batch files and Unix commandline tools.. And I'm not quite there yet
I may look into writing a program/tool instead, but this will be some other time.

In regards to licenses: I have not setup licenses so far(no maya install). just playing with Backburner and the integration between the Instances. But I am working on a way for the Instances to derive there "Backburner" configurations from the manager server when they are launched. Might be able to port that function to the license part. At least for floating licenses. That i cannot test from home, since I only have single license here.
 
Old 02-10-2013, 10:18 AM   #12
cdenny
Veteran
portfolio
cd
USA
 
Join Date: Feb 2009
Posts: 37
Just a little update:
I built another one from scratch because, no matter what I did, changing the computer to 'dynamic naming' in the EC2Config Settings the name would stay the same. This caused problems. I ran into trouble with the elastic ip. Backburner connected, shared drives were visible etc, but I kept getting an "Error Exit on 211". This appeared to be the slave not 'finding' the files, which made no sense, I guess I just dont know how to use it or something.

One other problem, using \\servername\blah to connect wouldn't work on anything other than the host (where you could just do \\localhost anyway). The private ip worked, but that isn't static. The elastic ip worked on clients, but the host couldn't "map to network drive" it's own shared folder using the elastic ip, it would just pop up a login window where you couldn't log in. I ditched that and downloaded Deadline because you can have two free slaves, and like I said way earlier, these cc2.8xlarge instances are beastly. First ami I built, the name doesn't change (it's borked) so that is the 'manager', it can be its own slave so I just set the repository to \\servername\DeadlineRepository . I "released" the elastic ip because I can take the five minutes to manually network the shared drive and connect the second instance to the 'manager', it's literally just two numbers, it's nice that \\servername\blah works on the host and \\ip address\blah on the slave. If using the instance as both a manager and a slave is horrible I'll just get the elastic ip back and make the 'manager' instance a cheap low powered 5c/hr computer. I'm loving Deadline so far, even though I have no clue how to use it for the most part.

I can't stress how awesome and cheap this is, people were raving over how awesome it was when they were using the normal instances with 26cpu units, and not these compute clusters with 88.5 cpu units for 20c more. If I could only figure out the whole networking, folder access part of this, you could easily create a small personal, on demand, renderfarm that is 10x faster than an overclocked 3930k rig for $3.60/hr, but with no overhead. As a college student on a tight budget this makes me happy. Am I unclear on anything? I feel like I've explained how it works pretty well, a lot of it is just trying over and over to get it right. I've only spent $1.41 setting this up so far, ha.
 
Old 02-10-2013, 03:37 PM   #13
tswalk
Lord of the posts
 
tswalk's Avatar
portfolio
Troy Walker
USA
 
Join Date: Jan 2012
Posts: 708
did you go through Judd's entire vimeo series on setting up backburner in EC2?... i know its' a bit slow, but he does a pretty thorough explanation of everything.
__________________
-- LinkedIn Profile --
-- Blog --
-- Portfolio --
 
Old 02-11-2013, 03:31 AM   #14
QuakeMarine1
PRO
 
Join Date: Dec 2004
Posts: 141
Quote:
Originally Posted by cdenny
Compared to rebusfarm, it's much more complicated, may take more time just to set it up and stuff, but its much cheaper. Based on 3.9c/hr/ghz, a single cc2.8xlarge instance with maya's 45c/hr rate would be equivalent to paying 3.45/hr (about ten times the cost).


do me a favour
send a job to EC2 and send the same files to Rebusfarm
(register and you get 10 points free at Rebus automaticly)

lets compare the prices and speed Rebus vs EC2
I doubt your math "about 10 times more" is correct.
I am curriouse to see the result !

Andre
Rebusfarm.net
 
Old 02-11-2013, 05:24 AM   #15
cdenny
Veteran
portfolio
cd
USA
 
Join Date: Feb 2009
Posts: 37
Quote:
Originally Posted by QuakeMarine1
do me a favour
send a job to EC2 and send the same files to Rebusfarm
(register and you get 10 points free at Rebus automaticly)

lets compare the prices and speed Rebus vs EC2
I doubt your math "about 10 times more" is correct.
I am curriouse to see the result !

Andre
Rebusfarm.net


Well considering I used your calculator it's probably pretty accurate. (this is only true when you use the cc2.8xlarge instance, which cause problems because it's 'cluster edition')
BUT
Screw this, it just doesn't work, I've spent hours messing with this shit and it just isn't worth it. It was a fun-ish experiment that just failed, just bill your clients for using rebus or something. I got it all networking correctly using a VPC and EIP, BUT THERE'S ALWAYS A PROBLEM, like backburner telling me it rendered 20 frames in 6 seconds, but the frames don't exist, and believe me, I checked the settings a thousand times. Just buy another freaking computer or use rebus. I have wasted too much time for too little return. Also, with the .10cents/gb transfer rate I've realized those costs can start to add up when you have thousand of files and stuff.
Good luck if you try to make it work, but I'm done. The only thing I'm going to try is using Deadlines 2 node edition, or just use bat rendering, I'm done trying to make backburner work with amazon.
 
Thread Closed share thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 07:15 AM.


Powered by vBulletin
Copyright ©2000 - 2016, Jelsoft Enterprises Ltd.