How to handle Big files.?

[israelyang]

Hi
How long is the animation? If it's around a minute I'd say 30 minute per frame should be toleratable. Try turn off reflection on all shaders(note by default blinn has a 0.5 reflectivity in Maya) if you haven't done so. Use ray trace light on only objects that require it. Use low res depth map for farther objects, bake shadows into the shaders if the shadows won't move in the animation. How many lights do you have in the scene? How many are casting ray trace shadows and how many depth map?

[h2o]

Divide the scene in different distance. Such as near, mid, far - 3 layers. Optimize texture file size will also help.

[Mauritius]

All i want to know is how is this kind of scene is handled int he production level.. i ' m only using software render in maya. I avoided ray tracing but the shadow looks horrible despite highres depth map. its there any thing i can do to render faster in 2k.? or should i wait. becuse of the time constraint i may lose the project.

The short answer is that you chose the wrong renderer. The Maya renderer is a piece of crap. Literally. I honestly don't have the slightes bit of compassion for long term users of that renderer when they run into issues with it . If you must use it, you better know what you are doing. ;)

The long answer is that if you can't switch to another renderer, you need to break the scene's compexity down by rendering in depth layers and putting those back together at comp stage.
If you have complex camera moves, that can be a nightmare.

I would suggest to look at alternatives for future projects. E.g. there's RenderMan for Maya and there's 3Delight for Maya. The latter comes with a full blown free 1st license, so if you need just one seat you don't have to spend a penny on it.

.mm

[Als]

I have to comment on this. I've decided to go throught this root and try and use 3Delight.
Well, problem comes from two things: support, manuals and examples.
I found 3Delight pdf file to be very limited in explaining, and installation doesn't have enough examples.
Support means that you either have to buy support from 3Delight guys or you are out in a dry. There is very very few tutorials on 3Delight out there.
While you can find lot's of renderman shaders on internet, many of them don't work with 3Delight (many do to).
As shaders and renderman tutorials have same problem.
It should work with 3Delight, but if it doesn't, it's enourmous task finding out what's wrong.
Also I had to recompile all the shaders for the new version, but even though it's worth it, it's still painfull.
Liquid works fine, but again it doesn't do all things you need to work with.
I've tried using shaderman to make shaders for 3Delight, and this has many limitations too.
I would really like to see more examples in 3Delight, at least on basic things, like how to do production value - hdri, fog, depth of field, raytracing shadows, volume shaders, volume lights, deep shadows, etc.
The speed of 3Delight displacement is amazing, but there is lot I need to workout to be able to use it in production. Eyesplit is such a pain.
So I'm starting to read renderman books etc. which helps, but does everybody have the time to do it, I doubt it.
So, I agree, yes, this is the way to go, but there are still too many problems associated with this.
Power of 3Delight and Renderman is amazing, but you need to be able to use it, otherwise it doesn't help.

Thank you Mauritius for your help this far!


Als




The short answer is that you chose the wrong renderer. The Maya renderer is a piece of crap. Literally. I honestly don't have the slightes bit of compassion for long term users of that renderer when they run into issues with it . If you must use it, you better know what you are doing. ;)

The long answer is that if you can't switch to another renderer, you need to break the scene's compexity down by rendering in depth layers and putting those back together at comp stage.
If you have complex camera moves, that can be a nightmare.

I would suggest to look at alternatives for future projects. E.g. there's RenderMan for Maya and there's 3Delight for Maya. The latter comes with a full blown free 1st license, so if you need just one seat you don't have to spend a penny on it.

.mm

[Mauritius]

I have to comment on this. I've decided to go throught this root and try and use 3Delight.
Well, problem comes from two things: support, manuals and examples.
Sorry, but I have to disagree. The proble comes for a lack of willingness of the users to invest serious time time in learing the essences of rendering. Rendering is the most complex part of any production. Don't expect to get into it by doing a few tutorials.
I found 3Delight pdf file to be very limited in explaining, and installation doesn't have enough examples.
The PDF is a refernce dosumentation. It is not a tutorial on 3Delight. There is so much stuff out there. Have you read any of the SIGGRAPH course notes on RenderMan???
Support means that you either have to buy support from 3Delight guys or you are out in a dry. There is very very few tutorials on 3Delight out there.
Err, this is a professional high-end renderer. What do you expect? They already give you a license for free. You must be kidding??? Support for the free license is 250 US$/year. That is a bargain given the level of support you get from these people. In this week we received three new 3Delight release fixing one issue we had and implementing two features we asked for. Their support is worth every penny.

There's no tutorials needed about 3Delight as there is a shitload of documenetation on RenderMan and 3Deligt is RenderMan compliant. Apart from some very rare cases, all the stuff in the SIGGRAPH course notes works straight with 3Delight, all the stuff in "Advanced RenderMan" and "Essential RenderMan Fast" does. What are you missing???
While you can find lot's of renderman shaders on internet, many of them don't work with 3Delight (many do to).[quote]
Many??? Can you give an example of a shader the does not compile with 3Delight, please.
[quote]As shaders and renderman tutorials have same problem. It should work with 3Delight, but if it doesn't, it's enourmous task finding out what's wrong.
Again, please give an example. I have trouble finding any shader online that doesn"t compile with 3Delight these days.
Also I had to recompile all the shaders for the new version, but even though it's worth it, it's still painfull.
RTFM. Call shaderdl with --embed-source and 3Delight does it for you automagically.
Liquid works fine, but again it doesn't do all things you need to work with.
I've tried using shaderman to make shaders for 3Delight, and this has many limitations too.
What limitations are you talking about?
I would really like to see more examples in 3Delight, at least on basic things, like how to do production value -
[quote]hdri
Use an any HDR image (like a float tiff, radiance, .hdr etc.) and send it through tdlmake. There's your HDRI support.
fog
"fog" shader is a standard RMan shader and comes with 3Delight
depth of field/quote]
DepthOffField -- works a charm in Liquid, MtoR, MayaMan and Houdini -- with 3Delight. Or put it in the RIB yourself, the RI command uses actual focal distance and f-stop -- it doesn't get any easier.
[quote]raytracing shadows
Use "raytrace" as your shadowname in shadowspot or shadowdistant. Presto (also well documented in the manual).
volume shaders, volume lights
"noisysmoke" comes with 3Delight.
deep shadows
Just render a deep shadow map. Very same process as an ordinary shadow map, just a diffrent display driver. Liquid has e.g. autohadow and you can tick 'Deep Shadow' in Fluid on the light shader, MtoR has similar easy ways. If you want to hack a RIB you plain switch your display driver (see the manual on the DSM driver).
If that is seriously too hard for you to understand, you shouldn't be touching rendering at all. Sorry if I sound harsh.
The speed of 3Delight displacement is amazing, but there is lot I need to workout to be able to use it in production. Eyesplit is such a pain.
The workout is reading. The 3Delight manaul clearly states the RenderMan spec. as the main reference. The 3Delight docs just document were 3Delight extends the 3.2. spec or deviates from it. You can download it the 3.2 spec.at the Pixar website for free.
Eyesplits are not a pain if you know what you are doing. 99% of the time noobs have issues with eyesplits because they set insane displacement bounds (or noe at all and the app has a crazy high default). :)
So I'm starting to read renderman books etc. which helps, but does everybody have the time to do it, I doubt it.
Not everybody should be working in this industry.
In many Mediterranean countries, you can be a skipper w/o doing a real license. Imho that is bad and it makes it dangerous for everyone out there on the sea. If you grow up in a country by the Baltic sea, chances are you have to learn a lot first before you will be granted a license as a skipper of a yacht.
Same with rendering or any other sort of expert knowledge. You buy a piano, it takes years to master. You buy a real renderer it takes years to master. Rendering is a profession like animation or painting. If you are not willing to invest time into learing, you will have trouble getting good results and be constantly frustrated by the mistakes you make.

I studied e.g. typography. In any program you can instantly create perfect type today as this is just the font itself. However, to create well typeset type, you need to understand typography at many levels.

Rendering or RenderMan is no different in those regards.


Cheers,

Moritz

[Als]

Well, I'm really shocked by your reply.
I'm gratefull for your help so far, but I didn't expect this rude and arogant reply.
Well, if I dared asking question, or telling about problems I have when I started learning, you reply is that I should change the profession?
You were the inspiration for me to start learning 3Delight, so I really didn't expect you reaction to be so harsh.
I got a lot of references and material, but clearly I can read 150 pages a day, and understand them all. I just started learning it.
I'm not expecting to have amazing results without investing time in it.
I just said I wished there were more simple examples of how to do this and that which would make things much easier to learn.
There is a difference between renderman and 3Delight, which clearly is not problem for you since you know both, but I learned that if you have one line wrong in the rib file, the result will be totaly wrong.
I got the siggraph courses, which one would you recomend me to start with?
We are doing internal short movie, and I'm trying to use the time to learn 3Delight.
Otherwise I work very long hours since I'm most of the time only 3D person who is doing jobs.
I started reading renderman companion. Should I skip that and take essential renderman, or advanced renderman?
I'm trying to learn some shaders from renderman repository, and I found few I needed which didn't compile, but I'm starting to understand more about it, so it is easier.
I tried page about uberlight for example and none of the ribs worked. But I managed to get noisysmoke to work now. Still trying to understand how to make smoother raytracing shadow. Example from the manual makes just noisy edges, didn't find yet how to smooth them.
I will look up the shader compiler command, thanks for that. It would not hurt if they added this information in release notes.
I'm trying to use shaderman for making shaders, but I'm not sure how I can use 3Delight shaders which are not there. I guess I have to learn how to write bricks, or I'm missing something again.
I managed to render curves, and use Hdr as a lightsource, but still not sure if it's working properly. I realise you have to use light probe format to convert with tdlmake, is there any option you recomend using when doing so? Also I'm still not getting smooth shadows as I would like.
So, yes, I'm trying to learn it, and if you don't have patience to answer my questions - please don't.
I'm not looking for all the answers or easy tutorials to follow, just some directions.
And even you haven't been very polite, I am very gratefull to you anyway.

Thanks


Als

[3delight]

Hello There,

I would really recommand reading "The RenderMan Companion" first. Even if it is quite old now, it is still, IMHO, the book that best describes the "RenderMan spirit". All examples in the book - shaders and RIBs alike, will work with 3Delight .

Regarding shaders: next version of 3Delight For Maya will automatically use Hypershade networks, so this will make things easier for you.

If you need assistance, please e-mail us at info at 3delight and we will be glad to help when time permits.

Thanks for using 3Delight,

- Aghiles.

ps: RenderMan is not a software, it is a standard! You are confusing RenderMan and "PrMan".


I got a lot of references and material, but clearly I can read 150 pages a day, and understand them all. I just started learning it.
Als

[neuromancer1978]

Shaderman is actually very similar to Hypershade in Maya, just a bit more complex. I would recommend just toying around with it at first, then once you start to get a more understanding of RSL functions try more complex shaders. Trust me it's no easy task, I have been using Pixie, Aqsis, 3Delight and Shaderman for over 2 years now and only just now have gotten really good at understanding RiSpec and how to render with it.

Rendering for Beginners by Saty Raghavachary is an excellent read too, in fact it may be your best option right now before tackling Adv. Renderman.

Experiement as much as you can, and don't start big because you won't EVER get the exact result you want.

If you need any help you can always ask around, I may be able to help even though I am no where near as experienced as some people on this site.

[Als]

Hi Aghiles!

Thanks a lot for your reply.
I just have old habit calling PRman renderman, since when I first found out about it, I was not aware of other, or indeed that it was standard for shading language. Again the same name is not making things easier either, and today I realised there is also third renderman name - RenderMan Interface Specification. Now there is also Renderman for maya.
You have to admit it is a bit confusing.
There is link for Risc on your website, but it is broken because seems that Pixar changed policy, so it can be downloaded only after registering, which is free.

Thank you for making 3Delight available!
Now I can learn it, and use it in production as well.
You current version of 3Delight blew me away with speed.

I hope this emails help encourages other people to try 3Delight too.
I can't wait for new version!

Thank you !


Als

[Als]

Thanks for you kind words.
My only question regarding shaderman at the moment is how to use for example 3Delight shaders to quickly mix and combine them, or is this wrong aproach?
I'm using context http://www.context.cx/ to edit mel scripts, shell etc. so I'm also looking for highlighters for sl and rib. I also found slceditor, but it doesn't have load to load the shader, so you have to paste the text, or I also have wrong aproach there too?
I just looked at Rendering for Beginners, and it looks excellent too.

Thanks again


Als



Shaderman is actually very similar to Hypershade in Maya, just a bit more complex. I would recommend just toying around with it at first, then once you start to get a more understanding of RSL functions try more complex shaders. Trust me it's no easy task, I have been using Pixie, Aqsis, 3Delight and Shaderman for over 2 years now and only just now have gotten really good at understanding RiSpec and how to render with it.

Rendering for Beginners by Saty Raghavachary is an excellent read too, in fact it may be your best option right now before tackling Adv. Renderman.

Experiement as much as you can, and don't start big because you won't EVER get the exact result you want.

If you need any help you can always ask around, I may be able to help even though I am no where near as experienced as some people on this site.

[Mauritius]

Well, I'm really shocked by your reply.
I'm gratefull for your help so far, but I didn't expect this rude and arogant reply.
Well, it takes five years of applying RenderMan in production to feel like a fish in the water with it. Maybe you might consider that someone like me perceievs your post as arrogant because it staes so many shortcomings of 3Delight that to me are obviously problems you have using the product, not the product has (as e.g. lack of tutorials, shaders not compiling etc.).
I think if you re-read your original post you might find that it sounds like you are bashing 3Delight. That is fine if you make valid pioints but you haven't. The issues you have stem from a lack of knowledge on your side, and blaming that on a lack of turials lets me deduce that you haven't very toroughly looked at what is out there in terms of documentation.
Well, if I dared asking question, or telling about problems I have when I started learning, you reply is that I should change the profession?
I was not talking about you, I'm talking about people in 3D in general. I meet a lot of people in my job and I frankly think some people shoulcn't touch rendering (or riggining, or <insert any other specialization here>). I'm just a little bt sick of the whinging and bitching I hear from people that obviously lack the background to make such comments.

When I started learning RenderMan, I learned a lot by poking people in the RMan newsgroup. Since Google records it, you can still find all my posts by going through the archives and you can see how long it took me to learn all this. Nevertheless, I always found the answers to my questsions by reading what was out there. I never assumed that just because I didn't know how to get st. to work, it would'nt work and was a software (or documentation) problem in the first place.
I got a lot of references and material, but clearly I can read 150 pages a day, and understand them all. I just started learning it.[quote]
If you read 150 pages a month (which is quite archivealble, particularly if we're talking Siggraph course notes with their big line spacing and short line lengths, hehe), you would already be able to answer a lot of the questions you asked yourself. The oldests post I find from you with the word RenderMan in it on CGTalk is from one year ago. I think that even if you only read 50 pages a month since then, you would be able to answer most of the questions in the post I replied to yourself by now.
[quote]I got the siggraph courses, which one would you recomend me to start with?
Just read them in the order they came out. :)
I started reading renderman companion. Should I skip that and take essential renderman, or advanced renderman?
I suggest the "RenderMan Companion" as a reference while you are reading "Rendering for Beginners: Image synthesis using RenderMan" from Saty Raghavachary. Imho it's the best book currently out there for beginners. Or read "Essential RenderMan Fast" with the "RenderMan Companion" as a reference. "Advanced RenderMan" is slightly outdated by now since it covers state of the art solutions to common rendering problems through RMan, but it docunents the state of the art in 1999. The companion is a reference, so it is of merely timeless value.
I tried page about uberlight for example and none of the ribs worked.
Yeah, they are bad hand-hacked RIBs that violate the RI spec. I just looked at uberlight.rib. It has AttributeBegin..End commands in object definitions which is wrong but 3Delight just prints a warning and ignores this.

However, they lack (inline or explicit) declaration of all the parameters given to the shaders so 3Delight ignores them which leads to the RIB rendering crap.
Last but not least the RIB instances "smoke" but gives that shader a lot of parameters that only "noisysmoke" understands. Just adding the types to all the parameters and swapping "smoke" for "noisysmoke" makes the RIB render the correct image instantly. That is bad QA on the side of the RMan repository.
Line 538:
Atmosphere "noisysmoke" "float density" [.05] "float integstart" 1 "float integend" 50 "float stepsize" [.15] "color scatter" [1 1 1] "float octaves" [3] "float freq" [.5] "float smokevary" [3] "float use_noise" 0 "float lightscale" 100

Line 553:
LightSource "uberlight" 2 "float width" [.25] "float height" [.1] "float roundness" [0] "float intensity" 5 "float wedge" 0 "float hedge" 0 "float roundness" .5
But I managed to get noisysmoke to work now. Still trying to understand how to make smoother raytracing shadow. Example from the manual makes just noisy edges, didn't find yet how to smooth them.
Increase the "shadownsamps" parameter in the light source shader.
I will look up the shader compiler command, thanks for that. It would not hurt if they added this information in release notes.
Huh? The release notes contain information about what has changed in the latest version. I"m sure that four or so years ago or so when this feature got introduced, it was mentioned in the release notes of the resp. version. How about typing "shaderdl -h" on the command line? :)
I managed to render curves, and use Hdr as a lightsource, but still not sure if it's working properly. I realise you have to use light probe format to convert with tdlmake, is there any option you recomend using when doing so? Also I'm still not getting smooth shadows as I would like.
tdlmake -lightprobe -filter gaussian lightprobe.hdr lightprobe.tdl

Make sure your image is cropped to the probe and doesn't have a black border. Use a Gaussian filter, so high values don't result in negative crap in lower mip-map levels which you otherwise get with filters that have negative lobes.

I also think you are confusing HDR images with image-based lighting. They are not dependent on each other at all. Aka you can do image based lighting with a LDR image. You can also create a LDR cubic environment map from a LDR lightprobe. Or you can use a HDR image texture but not for lighting.

Beers! ;)

Moritz

[Als]

Well, again, thanks a lot for great help, this will give me some nice oportunity to learn more. Also I do have to say that english is not my first language, which might lead to the missunderstanding.

I was not bashing 3Delight at all, quite contrary. I was just stating the problems I'm having while learning. That's all.

In the start of this thread you said that the way to go is renderman for maya or 3delight for maya, and I agree on that too.

Easy to use version of 3Delight is just behind the corner, but not right not yet.
I can't wait to see it.

At the moment I'm using maya 5, liquid 1.60, with newest version of 3Delight.

There are many things which I was able to achive, and every invested minute in learning is giving fruit. It all started with the wish to be able to render proper particles from maya, and it became so sweet experience that I wanted to know more.
In short time between two versions of 3Delight I noticed big improvements too, which is really promising.
So again I state that the problems I have do come from luck of knowledge about renderman, shading language and 3Delight. I was just trying to point out the stubborn ones, in hope that you might help, and you did. Thanks again!
There is planty of tutorials about renderman, but as you noticed yourself they have subtle differences which makes ribs "useless" to someone who can't spot the errors.
Also some of them are very old. They are still all great as a learning tool.

To go back to subject of thread I found this shaders:
http://www.edit.ne.jp/~katsu/rms_tree.htm (http://www.edit.ne.jp/%7Ekatsu/rms_tree.htm)
They work with 3Delight great.
Next step would be to learn DSO and learn how to use it with those trees.

My main point was warning that it is not easy to use any renderman compliant renderer and that it takes a learning curve to get there.

But, as you said, the results are more then worth it!

Thanks again


Als


PS
I'd like to go back to the points you made and report the progress back here. I will try to do the trees scene...

[rendermaniac]

The good thing about RenderMan complient renderers is that they are very adapatable - this is also a bad thing from the point of view of someone writing the translation software.

This is why it is hard to use RenderMan - there aren't many good front ends and they take a lot of effort to create. Moritz should know all about that with his work on Liquid ;)

You really do need to know why you are turning that button on and what it will do for you. This is where a good rendering TD comes in. However saying that, once you do have a pipeline in place it should be pretty easy to crank out shots - getting a pipeline in place is the hard part and where all the little details come in.

I suspect for a really big scene you want to archive out your high res geometry into delayed read archives.

However there are a lot of details to get right - such as pivot points, shader assignment (group archives by material if you can - it makes life easier). Deforming geometry makes things a lot more complex, as does instancing (reusing archives - not what Maya calls instancing), passing through render/shader attributes etc.

Then there is setting up a low res proxy, level of detail - and keeping it all up to date!

Unfortunately RenderMan for Maya doesn't do RIB archives (not sure how Maya references are handled??) so it probably isn't a good option.

3delight or liquid will (probably) do, but I suspect they will need quite a bit of work to get working as you want.

Also do get Advanced RenderMan - it goes into the most detail of any RenderMan book. It could do with a new edition to update some areas (particularly GI stuff), but what is there will save your arse.

Simon

PS Katsu is a very clever guy, but his shaders aren't written for learning from and are a bit old now. There are some good introductory DSO's at http://www.renderman.org/RMR/DSOs/index.html which are very basic, but much easier to understand.

[Als]

Thanks a lot!
What can I improve tree/leaf shader?
Or at least can you explain what are the misadvantages of those shaders?
I realise that every film company is investing a lot in renderman shaders and they are well kept secrets. On the other hand there is a lot of shaders which are aither explaining principles, or they are great starting point.
I'm trying to figure out what is the best way to make shaders, sl editors, etc.
Shaderman looks promising, but I'm not sure if I need to build bricks to be able to combine shaders, and if so how to do that. Shaderman forums seems abandoned, so I'm thinking maybe people are using different tool.
Also I found cutter text editor, which looks promising, but it's again a new thing which needs investing time to use it... (http://www.fundza.com/index.html)
Also shade tree seems to be dead, I can't find it...
Dark tree shaders look great! I'm looking how can I use it with renderman...

Thanks again!


Als

[Mauritius]

Just as a follow up to this, a pre-release of LiquidMaya 1.8 is just out.
Thread is here (http://forums.cgsociety.org/showthread.php?t=375101).

.mm

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.