AE watch folder nodes get stuck

[Mylenium]

Hello,

What would cause AE nodes to get stuck on a job? They either get stuck after rendering or right after loading up the job. I'm guessing the nodes are trying to access the rcf file and for whatever reason they get stuck. Maybe the nodes are accessing the file at the same time?

I'm testing out two Mac G5 nodes and AE will sit there with the spinning rainbow icon. If I "force quit" one of the nodes then the other node continues with the job.

I think this used to happen on our old PC farm as well but we had so many nodes going that our jobs would still be rendering. I wanna make sure the Mac nodes/watch folder system is reliable before I online the other 20 nodes.

Has anyone else run into this and is there a fix?

Ernest

These errors mostly depend on the footage types used. Before rendering, AE will always scan all footages which can take a while if you are using layered PSDs or large video files in compressed formats and then concurrent access becomes an issue. Could of course also be simple network collision timeouts.. Not sure.

Mylenium

[ernpchan]

Hi Mylenium,

Is there a way to get around the network collision timeouts? Our projects consist mainly of image sequences and psd/ai files. I would hope that the watch folder system is more reliable and not requiring someone to force quit nodes multiple times during the day.

E

[Mylenium]

Hi Mylenium,

Is there a way to get around the network collision timeouts? Our projects consist mainly of image sequences and psd/ai files. I would hope that the watch folder system is more reliable and not requiring someone to force quit nodes multiple times during the day.

E

Network collisions occur due to non-ideal routing. Hence the preferred way to use AErender would be to have all machines on a separate sub-net connected all to the same router/ switch which in turn ideally has no other network traffic load. Many modern machines have 2 LAN controllers onboard, so I'd reserve one of them for this exclusive render-network using a fixed IP. Of course it shoudl all be full duplex Gigabit at least and you also need to make sure that on the machines the disks are fast enough to not cause a stutter.

Mylenium

[ernpchan]

Hi Mylenium,

So I had my IT guy move the two nodes I'm testing to the same switch and the nodes still got stuck.

They got through about 10 jobs fine and they both got stuck on the same one. They rendered the first job in the queue fine and then just sat there with the spinning rainbow icon. I had to force quit both of them and setup the watch folder again. Sometimes if I force quit one the other node will get unstuck and keep going.

Is there something else we can try?

E

[Mylenium]

Hi Mylenium,

So I had my IT guy move the two nodes I'm testing to the same switch and the nodes still got stuck.

They got through about 10 jobs fine and they both got stuck on the same one. They rendered the first job in the queue fine and then just sat there with the spinning rainbow icon. I had to force quit both of them and setup the watch folder again. Sometimes if I force quit one the other node will get unstuck and keep going.

Is there something else we can try?

E

Check the prefs file for the asynchronous read/ write disable option. Some output modules react extremely sensitive even to the slightest delay.

Mylenium

[ernpchan]

Thanks Mylenium,

I found this in the prefs on my PC workstation and the Mac node. What should the value be set to? Double zero?

"Asynchronous file reading and writing" = 01

[Mylenium]

What should the value be set to? Double zero?

Yes. It's a boolean on/off toggle. BTW, if the problem persists, you could write a command script that shuts down hanging instances of AErender and then restarts them... This just occured to me after my last post.

Mylenium

[ernpchan]

Well unfortunately I think the nodes still got stuck. One was on "waiting to open render control file" while the other one sat there with the spinning rainbow icon. I let it sit for awhile and then just did a force quit and the other node went ahead and opened up the project and started rendering.

I'll look into the script thing.

For our jobs we use the process of rendering out image sequences and then using dependency to re-import that sequence to generate a QT. If we skip the image sequence route and just render out straight to QT would that alleviate this problem?

Is the watch folder system normally this fussy?

E

[Mylenium]

Is the watch folder system normally this fussy?

Well, it is in some way.... For some users it works without problems, for others it doesn't.


For our jobs we use the process of rendering out image sequences and then using dependency to re-import that sequence to generate a QT. If we skip the image sequence route and just render out straight to QT would that alleviate this problem?

You can't render to video files, so your suggested workflow would kill the benefits of network rendering... The re-importing thing might be part of the problem, though. In that case AE would initiate an extra lookup every few seconds to see if the image sequence is finished. Perhaps you can approach this differently: Render with AE, assembel with a different program. Tools like ProCoder can do this just as well using a watch folder without interfering in AE's own stuff.

Mylenium

[ernpchan]

I know multiple nodes can't render a video file. What I meant was instead of a job doing the skip existing files image sequence option we just render straight to the QT file.

Some of our shows edit at 24 and our footage is at 24 so going straight to a QT would be fine.

The shows that edit at 2997 would be more problematic because expanding the 24 projects to 2997 via AE causes problems when the new frames are inserted. Sometimes animated masks or layers with translation keys don't show up on the correct frame. So the work around is rendering out the 24fps image sequence and then re-importing that to generate the 2997 QT.

Doing the image sequence job first is ideal though because then I know I have nodes all chipping in to help render a shot.

But if AE is getting stuck just trying to open up the rcf file then it's not even going to render out the image sequence job. Argh!

We have access to Butterfly Netrender and it supports AE, it just requires a few more steps to get BNR to render AE files. The process wouldn't be as simple as just going to collect files in AE.

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.