CGTalk > Software > Compositing Software > Adobe After Effects
Login register
Thread Closed share thread « Previous Thread | Next Thread »  
 
Thread Tools Search this Thread Display Modes
Old 02-05-2013, 12:25 PM   #1
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
200% render speed using multiple AE instances

I stumbled upon an interesting aspect with After Effects the other day when I was looking for ways to start multiple instances of AE on my Mac. Might not be new, but I haven't seen any info about it before.

If I started several instances and let them render an image sequence simultaneously (with skip existing frames) it actually rendered quite faster if all of them were in single processing mode, compared to rendering the same sequence with just one instance of AE using multi processing mode.

This test computer I used is quite old, an eight core with only 8 gb ram. But increase in render speed was a little more than the double. The test project rendered was a relatively simple particle effect in Particular.

Here are the render times from various tests.
  • one instance multiprocessing - 4:11
  • two instances multiprocessing - 3:15
-----
  • one instance singleprocessing - 5:27
  • two instances singleprocessing - 3:04
  • three instances singleprocessing - 2:22
  • four instances singleprocessing - 1:59
  • six instances singleprocessing - 1:48

As you can see I get a 210% render boost with four or six simultaneous render instances compared to just one multiprocessing instance.

In the past there was a plugin from Gridiron that I believe automated this process in a good way. Unfortunately this plugin is no longer supported for CS6 and is no longer developed.

The process of manually getting multiple instances to render simultaneously is quite messy, at least on a Mac. It is something you want to do only once really. But it works perfectly on render slaves where you only set this up once using the AE Render Engine and watch folders. One benefit of this besides render speed is that you are allowed to run more instances of commercial plugins on each machine, as the plugin often doesn't need a unique serial number for each local instance you start.

And it seems multiple instances utilize CPU much more efficiently, as CPU usage stay more constant at the top when doing this.

Perhaps not all scenens will render faster using this method, but I guess one can build custom tools to exploit this more easily. Or why does not Adobe themselves fix their render system so that it uses system resources more efficiently?

Well, I'm curious what others might experience in terms of boosted render speeds using this approach.
 
Old 02-14-2013, 08:01 PM   #2
DrakenFX
New Member
 
DrakenFX's Avatar
portfolio
Isaac Borjas
personal, USA
 
Join Date: Mar 2012
Posts: 18
I'm Curios about this using Windows, do you have ay idea how to run multiple instances of AE on Windows base machine? so i can elaborate some test about it, that will be nice
 
Old 02-15-2013, 02:52 PM   #3
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
I believe you add -m to the end of your After Effects shortcut.

In other words, create or edit the shortcut to After Effects and simply add "-m" without the quotation marks to the end of the file path pointing to the After Effects executable.
 
Old 02-16-2013, 04:09 PM   #4
DrakenFX
New Member
 
DrakenFX's Avatar
portfolio
Isaac Borjas
personal, USA
 
Join Date: Mar 2012
Posts: 18
Thank you @Swahn , i'll check this out and see how can Render Times be Improve using AE on Windows.
 
Old 02-24-2013, 08:33 AM   #5
sentry66
Expert
 
sentry66's Avatar
portfolio
node crazy
USA
 
Join Date: May 2008
Posts: 2,076
I've been doing this for about a year now with Windows. I have a 6-core i7 and 64 gigs ram and launch 6 AE instances, each set for different time ranges. It renders way faster with solid reliability than using the multiprocessor mode that usually crashes 1/3 into your render. On my system, it's been something like 3-4x faster than using the standard multiprocessing mode.

The key to making it work well is having a fast SSD
When I do it on a slower regular hard drive, I get about 40-50% the performance compared to the SSD

Last edited by sentry66 : 02-24-2013 at 08:37 AM.
 
Old 02-24-2013, 12:06 PM   #6
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
Quote:
Originally Posted by sentry66
I've been doing this for about a year now with Windows. I have a 6-core i7 and 64 gigs ram and launch 6 AE instances, each set for different time ranges. It renders way faster with solid reliability than using the multiprocessor mode that usually crashes 1/3 into your render. On my system, it's been something like 3-4x faster than using the standard multiprocessing mode.

The key to making it work well is having a fast SSD
When I do it on a slower regular hard drive, I get about 40-50% the performance compared to the SSD


Good to know.
 
Old 04-05-2013, 01:05 PM   #7
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
I'm currently trying to construct a custom tool together with a couple of coding guys to create a more automated workflow to exploit this.

But I encountered a rather annoying issue with custom modules due to Adobes unsuitable solution to fix their licensing issues with codecs rendering on network slaves (which can be installed in an unlimited amount on each farm), explained here: http://blogs.adobe.com/aftereffects...ffects-cs6.html

The problem is that no custom output modules will work when using the new "non-royalty bearing mode" (which means you put a text file with a specific name in a specific folder to put it in "render engine only" mode). This works fine if you use the hard coded default render settings (such as "Multi-Machine Sequence"), but not if you render using any custom output templates.

Adobe still believe users find Watch Folders for network rendering to be a pleasant experience apparently. It works if you want to render using one instance of AE only. But if you want to exploit using several instances to gain a lot of render speed, Watch Folders will not only clog your entire interface with those TV-screens, it's quite likely that not all instances will join the render. When I did experiments with Watch Folder I noticed that most of the times about four instances joined the rendering, and the other four just did nothing. Seems like some of the instances will check the watch folder first after another instance has already begun, thus the render job is already flagged as "started" and the other instances won't render it. To me it looks like Watch Folder is clearly not designed to have several instances running on the same machine.

Rendering directly with aerender.exe works EACH TIME you need it for all instances, and you can start all of them in minimized windows that automatically close once rendering is done, making it a superior render approach for network rendering compared to Watch Folder.

Unless you want to register each and every slave with a serial there is no option but to stick with the default output modules.

But as suggested in this thread http://forums.adobe.com/message/4921115#4921115 it would have been decent of Adobe to let users render conventional image sequences in the non-royalty bearing mode, rather than the sleazy solution of just disabling custom output modules altogether regardless of what your output files are.
 
Old 04-05-2013, 04:49 PM   #8
sentry66
Expert
 
sentry66's Avatar
portfolio
node crazy
USA
 
Join Date: May 2008
Posts: 2,076
Networking is such a huge bottleneck these days. Gigabit ethernet can't keep up with a bunch of cores pulling in 1080p or 4k footage, especially if it's CG layers with 10-15 footage files per frame. You'd need fiber or 10gigabit ethernet to get enough bandwidth.
 
Old 04-05-2013, 07:41 PM   #9
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
I wouldn't use AE to comp loads of render passes in 4k anyway, as AE tend to be quite slow. Especially when it comes to compositing huge multichannel EXR files for example.

No, I'm not talking about network-heavy projects.
 
Old 04-06-2013, 12:00 AM   #10
sentry66
Expert
 
sentry66's Avatar
portfolio
node crazy
USA
 
Join Date: May 2008
Posts: 2,076
I don't know, I comp 4k multilayer projects a lot with AE and it's fine. I don't often set the viewport resolution to full, and instead mostly I run it at 1/4 or 1/2, or even 1/8.

It's relatively snappy on a 4.9ghz i7 and 64 gigs of ram and a decent SSD drive and caching SSD. I wish AE was better at being multithreaded when you're working, but at least you can trick it into utilizing all your cores at render time and get some pretty fast rendering performance.

I've seriously considered nuke, but as nice as it is, it also has a lot of disadvantages compared to AE for my line of work. If all someone did was comp single camera shots all day long, they'd be nuts to not use something like nuke or fusion.

Last edited by sentry66 : 04-06-2013 at 12:06 AM.
 
Old 04-08-2013, 09:11 AM   #11
Swahn
Expert
 
Swahn's Avatar
Daniel Swahn
Sweden
 
Join Date: May 2005
Posts: 344
Quote:
Originally Posted by sentry66
I don't know, I comp 4k multilayer projects a lot with AE and it's fine. I don't often set the viewport resolution to full, and instead mostly I run it at 1/4 or 1/2, or even 1/8.

It's relatively snappy on a 4.9ghz i7 and 64 gigs of ram and a decent SSD drive and caching SSD. I wish AE was better at being multithreaded when you're working, but at least you can trick it into utilizing all your cores at render time and get some pretty fast rendering performance.

I've seriously considered nuke, but as nice as it is, it also has a lot of disadvantages compared to AE for my line of work. If all someone did was comp single camera shots all day long, they'd be nuts to not use something like nuke or fusion.


Maybe it is due to my previous experience with EXR files in AE...
Honestly I have not had the opportunity to work that highres on my brand new BOXX, which probably will feel a lot less sluggish compared to my old rig.

Regardless, for lighter projects rendering using this exploit really decrease render times by more than 200%.
 
Old 04-08-2013, 09:11 AM   #12
CGTalk Moderation
Lord of the posts
CGTalk Forum Leader
 
Join Date: Sep 2003
Posts: 1,066,481
Thread automatically closed

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.
__________________
CGTalk Policy/Legalities
Note that as CGTalk Members, you agree to the terms and conditions of using this website.
 
Thread Closed share thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 12:36 AM.


Powered by vBulletin
Copyright ©2000 - 2016, Jelsoft Enterprises Ltd.