Modern cell shade technique


#1

I have played around alot with the mental ray toon shader look.
In general the results are not very descent, and this aspect of mental ray didn’t evolve much.
The renders are still a far cry from a fluid an elegant 2D cell shade look.

This animation was done 12 years ago in max by “Fortiche” (don’t know how they achieve the lines with the fluid movement):
https://www.youtube.com/watch?v=BkudnrTj_sI
I consider it to have high production value but it seems that even after a decade, it’s still not possible to achieve this in maya/mental ray. I hope I am wrong and that there are techniques and tool to achieve these results.


#2

If I remember correctly this was done using softimage XSI using Mentalray. Mental Ray is very powerful renderer but you need to work on with your shaders. To me the given clip the general shader arrangement is very simple even the outline is also not doing much as compared to most of the Japanese animations but most of the Japanese studio also depend heavily on composting and use mostly in house developed tools.


#3

So those toonlines are done in mental ray. I am surprised because they move in a very fluid way, as if they were done by hand in composting by an animator.

If that’s really mental ray, how can one achieve this look?
I studied the clip closely, there seem to be alot of light linking also, but that’s easy, what’s heard is the fluid movements of cell shade lines whether in shading or in the lines.


#4

It is a motion capture clip . and the fluidity is due to the fact the shader is reacting very good with the light. I don’t know if you have used softimage XSI but I must say try Todd Akita’s ink lining tutorials for better understanding.He did a whole series on toon shading while he was working for Psyop.I hope all your doubts will fade away about cell shading in CG.
It’s shame that XSI is no more but it was a great software.I don’t know which software you are using to achieve cell shading but I think basic remains same in all the softwares.


#5

http://download.autodesk.com/us/support/files/toon.pdf


#6

Are you sure about the motion capture thing? Is there any documentation regarding this production? I am sure there is also some modeling tricks while achieving this look specially with characters and the way the rendered lines reacts with the surface.

Who is Todd Akita, did he work on the clip?

Does anybody know other kind of resources regarding this clip?


#7

Great! actually I know most of the information in the PDF haven’t read it all and I am sure there must some tips there. But is there any reason to believe that the same high quality and defined look is doable in Maya/Mental Ray, has it been done before? is there any examples? I would like to know this before investing alot of time and effort on this.


#8

Send Duncan Brinsmead a PM.I believe he created it along with other Paint Effects.


#9

let’s say it clearly, mental ray toon in maya is a piece of crap…I recently had to do a pre production using it and I could never achieve it in maya, no way to map the line, no way to control the size of the line with a map or whatever, need 20 node in the render setting to be able to have a correct line that doesn’t look aweful or surimposed with one another, no seriously the all aspect of toon in maya mentalray is totally outdated.

Considering the pre production one friend had to finish the product in XSI doing the toon in it, and beleive it or not, but the line actually worked exactly as expected in there.

I don’t beleive Nvidia or autodesk will do anything about that cumberstone workflow in maya and toon line I have given hope in that…don’t expect any change in this area using maya and do it with something else, it’s not a priority for them, not at all.
As an exemple the toonline actually don’t work with the adaptive antialiasing, that says it all.

The clip I’ve work on.

https://www.youtube.com/watch?v=OHXf7wEpBPI


#10

|

couldn’t agree more.
I taught I was missing something.
The link does work from my place, is there any other link?


#11

not sure but try in youtube

“Skip The Use - Nameless World”


#12

It can totally be done…
maybe check out some of the stuff by Christophe Desse including tutorials:

http://www.cgchannel.com/2010/05/tutorial-cel-shading-in-maya-with-christophe-desse/

http://www.thegnomonworkshop.com/store/products/tutorials/cde_series/

http://xtrm3d.cgsociety.org/

https://area.autodesk.com/tutorials/how_to_render_cell_shader_and_outlines_in_mental_ray_for_maya

http://www.xtrm3d.com/gallery/npr/index.html


#13

Cool music video! Nice job!
It seems that you dropped the outline strokes.


#14

These are the most practical tutorials I have seen for cell shading in Maya (Mental ray and Maya software) and it’s totaly worth spending some time learning what christopher has to show. However the main issue is not addressed, how will the line behave in animation? when the character turn his head or when the camera moves? specially relys hevaily on Photoshop filters to finalise the look specially the lines.

I wonder what thematt thinksabout these tutorials? Do they have most of the info and steps to achieve a look like “je n’ai que deux pieds”?


#15

Well I would say google is your best friend if you really want to know or learn more about the clip ,I know about the clip as I have seen it at that time, sorry I am unable to recollect exact info.As per my memory Todd Akita didn’t worked on this clip but he was master of such an art form and I was simply referring his name because I thought it would be easier for you to find the relevant link , but as it seems you didn’t try it out ,Here is one such link of a whole series of tutorials on Cel shading in XSI I hope it would be of some help to your question https://www.youtube.com/watch?v=hBzX6uzwZl0.


#16

well the look in that clip is not that hard to do actually, it’s a quite simple outline and most of the work is done at the shader level, the trick is always to have a line on separate layer to be able to do masking.
Lines are getting hard to managed in mental ray in Maya once you have to do something other that a simple outline, like line only at specifics place, lines that should get thiner at other place, lines with noise etc…the workflow in mental ray is actually the worst you can imagine, having to deal with shader contrast, shader store and what’s not…I hate it it’s totally worthless and I’d like to punch the developper who though it was artist friendly.

Anyway line is always a bunch of tips and trick, but if you want a simple outline that is clean around a character, then yes mental ray contrast works good, just make sure you put enough sample 12-20 should do it.

I’ve been doing toon for over a decade and not much have evolve in that sense.
Maya Toon is actually quite cool, but unfortunatly is quite hard also to get nice line with eavy model, because it work only on non smooth object, or you have to subdivide your model manually and sample the line quite a lot which gets really heavy for maya.

To have a nice line where you want, you sometime have to have a model exclusively for the line with the hard edge where you want the line etc… it’s awfull to do it that way but it’s often the best solution.


#17

I invite you to join this thread:
https://forum.nvidia-arc.com/showthread.php?13414-Modern-cell-shade-technique

Your input might be super helpful there.


#18

thanks.;but nothing really new from what I read. Like he’s saying, XSi had a lot of programming behind the toon shader and that’s why it works good…in maya only mental ray does the toon and maya isn’t really build to support it correctly so the result are quite impredicatable.

Have you try to use the toon in comp…like a simple facing ratio from black to white on objects and a sobel filter. It happen that I got good result from that in the past, but the line stay consistant in size, which can lead to strange result while animating in the deaph.

edit:
there you can see the maya mental ray toon in action, but the shader was devellop in house (Mac guf ligne) to do that.;but it was metal ray. As you can see the outline is quite nice, but probelm arise once you have to artisticly direct the line when it’s facing camera and or put in specific places but do not want it always there, considering multiple angle of views. I think they did a good job nonetheless.
https://www.youtube.com/watch?v=_rY4iCqltS0

cheers


#19

I requested a modern cel shading workflow in the Maya ideas site:

http://mayafeedback.autodesk.com/forums/160514-ideas-for-maya-forum/suggestions/9291159-modern-cel-shading Vote and spread the word if you want to make it happen :slight_smile:


#20

It’s worthwhile giving this video a watch. It’s XSI and am not sure how well the techniques would translate to Maya but its interesting nonetheless…

https://www.youtube.com/watch?v=yhGjCzxJV3E