Its certainly is a reasonable request. Personally, I believe the current method of separating jobs out to different processors through Renderama is important and we should not be quick to dismiss it so easily, however, producing a multithreaded Camera would be nice.
We have two primary programs that weâre using:
Animator & Camera
Multiprocessor support for Camera sounds like it should be first priority over Animator because users want faster previews without having to setup a Rama batch job. The questions would be:
Whatâs involved in making an application multithreaded?
Can Cameraâs architecture support multithreading?
If yes, how much is involved and what would it cost?
If no, can anything be done to change that without a major overhaul of Camera?
How would multithreading be advantageous for Animator?
After the completion of the Intel port, EITG will most likely resume adding new features. Where would you like them to spend their time?
Perhaps someone with more experience with multithreading can share some insights?
I do remember a post by Matt Hoffman on the EITG forums mentioning the difficulties of multithreading Camera. Their attempts failed and they ended up with the solution weâre using at the moment.
Here is a seldom case when we agree with you Yes, we also think that a multithreaded Camera would be nice.
Mutithreading is a parallel code execution in bounds of same process (same address space). It can be implemented with a single processor as well, thatâs a problem of OS which processor takes care about which thread. Application is responsible for threadsâ initializing, running and killing.
IMO itâs better to ask: Is Cameraâs architecture suitable for multithreading? We think that yes, same as any render task. The problem is that multithreading is not a thing can be applied to anything to speed up.
"Iâve 180% speed up with my dual Mac!!!:applause:
"Iâve 115% speed up with my dual Mac :sad:
Both experiments/results are fully true, and percents are not even min/max values. Just a lot depends from what kind of work is processed. Not all can be multithreaded/parallelized. If, for example, youâve a large MrBlobby in your prj, then most probably it remains same slow no matter how many processors are involved. Cause implicit surface should be built step by step, sequentially, not parallel. Thatâs true for absolute most geometry generation actions.
However, the shading phase can be parallelized well. It would be especially effective for most RT aspects, count GI. But not for all: building BSP, creating photon maps and some other things should be performed sequentially.
Alas, itâs a very hard work. SDKâs of mutithreaded apps typically gives very short advices, like:
compile as mutithreaded DLL
avoid to write in static variables
thatâs absolute enough for your plug-in/shader to be mutithreaded
Thatâs true, but only until developer tries to make something not-trivial. And that is only âusageâ of hostâs pipeline. Difficulties/problems grow tremendously for large app and threadsâ âorganizationâ. All memory allocations, writing files and many others should be âsurroundedâ by semaphores/critical sections. Each thread should be equipped with its own âray-trace stackâ. There are tens and tens problems here
Using more processors is a method of âbrute forceâ (iron instead of brain). Look at RT soft shadows for example. They are poor and slow. One way is to increase a count of samples and add more processors. Another solution is to reorganize the algorithm of shadows calculation. We would like the second way
Weâve no ideas what could be parallelized in Animator. The preview speed? But it depends more from GPUâŠ
As already mentioned, how practical would it be to have previews rendered automatically on 2/ 4 Cameras?
That is to say have a âpreviewsâ renderama built into Animator?
So the workflow for the user is the same as now, but 2 Cameras start up and render.
Although, in all seriousness, this would only be practical if Camera could render vertical frame strips, for us, it would be useless otherwise.
not only CAs! my last project had many, many deformed objects in it. building the control files took around 10min/100frames! even when the deformed objects are turned off.
so, for some utility mask layer renders it looked like this:
writing control files: ~ 30min
rendering: ~ 10 min
Of course, all these things are reasonable, but⊠they have no any relation with multithreading Please understand: multithreading is not a âmagicâ can help for anything isnât enough fast.
Our opinion is similar (thought very blurred). Other apps have a âbigger materialâs ball (or cube, cylinder etc.) with all texturesâ. IMO itâs still ineffective, a ball is still far away from a real rendered object. Maybe a some kind of simplified render (pre-render) is the best/optimal solution. Not all should be copied
My work is likely different from others on this forum, so let me elaborate on this subject
As an industrial designer I am producing mainly stills for new products on extremely tight deadlines sometimes only 24 to 48 hrs.
I have many scenes in Animator with many millions of polygons, and often have into the thousands of objects (many, many duplicates) with lots of texture maps.
These projects are unweildy to say the least. Open GL chokes. Even software drawing mode is very slow in outline mode. I am always turning object groups on and off to improve screen redraw.
Rendering for me is pretty fast. I use GI for smaller projects and get render times of 5 to 15 minutes per still (I may produce as many as 30 stills for a project with multiple concepts so as a whole thatâs still a lot of render time!). My large projects would take over an hour with GI per still, so I usually fall back on Phong and get good quality in under 5 minutes.
Increasing render speed would be great as I could use more of the âadvancedâ render features like GI. But slowdowns and workflow in Animator are a legitimate performance bottleneck.
I can appreciate the limits of MP/ Multi-threading, but this is how hardware companies are going to give us more power in the forseeable future. Based on initial 3d benchmarks the latest 3Ghz Xeon from Intel/Apple is roughly equivalent to a 3.2 Ghz G5. Not bad, but thatâs only about a 60% performance improvement over my dual 2 Ghz G5 (weâre talking single thread performance) This is WAY off from Mooreâs law. Only a 60% performance improvement over a 3 year time frame. However, there are now FOUR of those processors in a single workstation. Thatâs a lot of untapped potential power.
My point is that the real challenge for software developers is to re-think thier apps, maybe even consider re-writing major parts of them to squeeze every ounce of performance from those extra processors. I understand that much of the calculation in 3D is linear, but being creative with how tasks are divided may yield some good performance improvements.
I will soon get back to the point of this forum with some more practical feature suggestions. Thanks to all for their input. I think this is a great discussion, and I hope Matt, Blair, Igors, and the rest of the EIAS Programming gang are following this and are inspired to make our favorite 3d App better and faster.
My work is likely different from others on this forum, so let me elaborate on this subject
As an industrial designer I am producing mainly stills for new products on extremely tight deadlines sometimes only 24 to 48 hrs.
I have many scenes in Animator with many millions of polygons, and often have into the thousands of objects (many, many duplicates) with lots of texture maps.
These projects are unweildy to say the least. Open GL chokes. Even software drawing mode is very slow in outline mode. I am always turning object groups on and off to improve screen redraw.
Rendering for me is pretty fast. I use GI for smaller projects and get render times of 5 to 15 minutes per still (I may produce as many as 30 stills for a project with multiple concepts so as a whole thatâs still a lot of render time!). My large projects would take over an hour with GI per still, so I usually fall back on Phong and get good quality in under 5 minutes.
Increasing render speed would be great as I could use more of the âadvancedâ render features like GI. But slowdowns and workflow in Animator are a legitimate performance bottleneck.
I can appreciate the limits of MP/ Multi-threading, but this is how hardware companies are going to give us more power in the forseeable future. Based on initial 3d benchmarks the latest 3Ghz Xeon from Intel/Apple is roughly equivalent to a 3.2 Ghz G5. Not bad, but thatâs only about a 60% performance improvement over my dual 2 Ghz G5 (weâre talking single thread performance) This is WAY off from Mooreâs law. Only a 60% performance improvement over a 3 year time frame. However, there are now FOUR of those processors in a single workstation. Thatâs a lot of untapped potential power.
My point is that the real challenge for software developers is to re-think thier apps, maybe even consider re-writing major parts of them to squeeze every ounce of performance from those extra processors. I understand that much of the calculation in 3D is linear, but being creative with how tasks are divided may yield some good performance improvements.
I will soon get back to the point of this forum with some more practical feature suggestions. Thanks to all for their input. I think this is a great discussion, and I hope Matt, Blair, Igors, and the rest of the EIAS Programming gang are following this and are inspired to make our favorite 3d App better and faster.
By preview I was meaning âSnapshotâ. However you bring up an interesting subject! I am very much in favour of this kind of feature.
But by pre-render do you mean a no frills snapshot? Perhaps add a render option to the linking editor 3d view? Then you could render one group at a time with no frillsâŠ
Well, Iâd like to see better material previews. A more consistent numerical data input. More feed back during rendering. Iâd like to see the product marketed- no-one knows about it. This isnât healthy.
We too We heard many times about âbetter material previewâ, but what is it? No one knows and no one app has an ideal preview. What we do now to setup material? Something like: edit - snapshot - edit - snapshot⊠many times. Often it needs to simplify scene: turns off other objects, lights, set additional cameras etc.
It would be nice if host does this routine work for us. For example: we are in Texture Window, press F7(?) and let host runs Camera with actual object only, simplified preview lights and actual orbit view. Add some âvariantsâ, for example:
render a whole material;
render actual texture only;
render textures stack up to actual one
Another one: re-render a selected/desired object only in previous screenshot screen. Yes, it will have a jagged edges etc. - thatâs only a draft sketch. But IMO it shows enough well how material looks in scene.
And who knows, maybe this set of modest features would give much more than a âsuperâ preview promises âreal-timeâ but ends with 50K of polys. Or with fractal noise with 5 and more octaves. Or with many other things.
If we could have a window (the snapshot preview window) be feeded by camera caches in a loop in the background without camera window lauching⊠without anti-aliases and using the best of all MP processors we could have a interesting and almost realtime preview.
Whatâs the reason Renderama has that big an impact in local multicamera rendering? The fact that it treats local cameras as networked ones? Or is it just slow managing things?
If producing a multithreaded Camera is so complicated, perhaps it would be better to create sort of an specialized Renderama-like app, say, âMetaCameraâ, able to talk more directly to several local Camera copies and do tricks such as dividing the frame into four stripes (would that route produce less overhead than assigning different frames to each pooled Camera? Would it be faster for Renderama to address several networked MetaCameras, each networked PC producing single frames instead of fours by using the striping technique?), or knowing when there is a pending preview job and having some preference for resolving it even if it is in the middle of an animation job (assigning one Camera to it, or pausing the main job to dedicate all Cameras to it)âŠ
For a striped image task division technique, perhaps MetaCamera could be smart enough to try pre-ordering things like precalculated shadows and cubic reflections so that they can be shared among the pooled Cameras: even if they have to wait at idle for one of them to build these, in the end a MetaCamera would be faster than every Camera doing it itself, I guess.
All is not very complicated was done in previous 10-15 years LOL
Yes, multithreaded render is hard to implement but same time it promises effective/attractive results. If Tomas and Jens run a serious RT/GI scene on their quads and have got x3 and more speed up - they can say only âwowâ Multithreaded technique does not âduplicateâ memory allocation same as modelsâ, plug-insâ and shadersâ loading for each render instance. The discussion sorta âwhatâs better: MP or network renderâ is obsolete IMO - now itâs time when 3d app should have both
Yes, we also think that a multithreaded Camera would be nice.
