Referencing and material pipeline question

Become a member of the CGSociety

Connect, Share, and Learn with our Large Growing CG Art Community. It's Free!

THREAD CLOSED
 
Thread Tools Search this Thread Display Modes
  08 August 2013
Referencing and material pipeline question

As of late I'm running into this problem more and more

- Layout team referenced proxy character for working out their camera

- Animator replaced that reference with low res character, sometimes hi res to check overlapping.

- lighting/rendering team then replaced that reference again for rendering.

What I'm running into a lot lately is that when replacing reference, materials would be disconnected from the model. I discussed this with the rigging team and they told me it was to be expected with the way Maya references stuff.

Anyone ran into this problem before? Would appreciate any suggestion.

Thanks!
 
  08 August 2013
I haven't had any experience in swapping referencing like you guys, but 100% of material shading losses for me are due to either;

1) Per face shading assignments (clean your assets with this script or split objects by shader); http://www.toadstorm.com/blog/?p=26)

OR

2) Instancing references which is harder to reproduce, but for some reason even with clean assets, Maya prefers to add reference edits and completely kill shading assignments to references. I'm not sure why and haven't been able to figure it out.

The first one is most common and would be trivial for Autodesk to fix, but year after year the same problem remains.

In your case, if those two aren't applicable I hope someone else knows!

Maybe have you tried the new Asset system rather than referencing? Seems like they are heading down that path.
 
  08 August 2013
Hi Hamburger thanks for dropping in

Yes I think our main problem would be #2. I heard that word "reference edits" a lot from the riggers. One way the rigging team suggested was to have a script that would reference materials separately from characters and re-apply them to the models. So that all the connections would happen after reference edits.

I'm still trying to figure out the best way.

Last edited by Panupat : 08 August 2013 at 09:56 AM.
 
  08 August 2013
Originally Posted by Panupat: As of late I'm running into this problem more and more

- Layout team referenced proxy character for working out their camera

- Animator replaced that reference with low res character, sometimes hi res to check overlapping.

- lighting/rendering team then replaced that reference again for rendering.

What I'm running into a lot lately is that when replacing reference, materials would be disconnected from the model. I discussed this with the rigging team and they told me it was to be expected with the way Maya references stuff.

Anyone ran into this problem before? Would appreciate any suggestion.

Thanks!


Shading is often applied on the 'base' geo and when rigging you often end up with different end meshes (skinning creates a shape0rig).

Were I work shading people work on the ref'ed geometry, riggers worked on the same ref'ed, animators take the rigged scenes and during rendering we assemble the shaders (we have scripts that export shaders from the shading scene and apply it in the rendering scene, it never goes through referencing)
__________________
maya@reddit r/maya
 
  09 September 2013
bernie - thanks for sharing! So the script would remember all the geometries the material was assigned, and reassign them, too?
 
  09 September 2013
Originally Posted by Panupat: bernie - thanks for sharing! So the script would remember all the geometries the material was assigned, and reassign them, too?



technically I think it's the other way around. All of our rigged (deformed geo) objects are put in one set called bake_geo, all of the keyed objects in another bake_transform set.

When animators are done, we export the baked geo and in the render scene, open up shaded/lit objects and apply the caches to them. This way any extra stuff the lighters and shaderers added stays in.
__________________
maya@reddit r/maya
 
  09 September 2013
hmm so in the render scene, it would be static characters without rig but with all materials applied? Then you apply geometry cache to it?

Assuming I understand correctly... for the render scene, it should be safe to reference static characters with all materials applied right? Does applying geometry cache add any extra deform node?

Last edited by Panupat : 09 September 2013 at 02:25 PM.
 
  09 September 2013
Originally Posted by Panupat: hmm so in the render scene, it would be static characters without rig but with all materials applied? Then you apply geometry cache to it?

Assuming I understand correctly... for the render scene, it should be safe to reference static characters with all materials applied right? Does applying geometry cache add any extra deform node?


We don't reference the shading and fur and lights, we simply import the cache INTO the shading scene, save as a render scene and send to the farm
__________________
maya@reddit r/maya
 
  09 September 2013
Thanks for your insight Bernie. Really appreciate
 
  09 September 2013
As the TD at Colorbleed I also had to set up a pipeline for this part of a project (actually the pipeline from start to finish). I ended up with a similar solution as to what berniebernie described.

During 'publishing' of a model all the geo of the asset get what we call 'assetId'-data. This is used everywhere in our pipeline to keep track of what asset it is, and this stays in there even if it ends up being deformed or whenever references get imported, and also is still available after using alembic caching and importing the cache.

Then during look development of the asset the lighting artist can set up a shader network for it. Then when it's done this shader gets exported together with a 'shaderRelations' file used to re-apply the shader afterwards. (We also have support for multiple shader variations for a single asset, for example having a green box and a red box shader for a single asset.)

The lighting scene ends up containing only the alembic cache. Actually we make a maya scene that contains the alembic and reference that. So whenever the alembic cache gets updated the shaders and all changes stay on the geo based on reference edits. Then our pipeline tools allow us to assign all the shaders automatically, based on the assetId and shader variation it references the shader that was exported earlier in the pipeline and directly assigns it.

I've optimized the shader exports, the shader-relations file (saved as JSON-data) and the importing so that even for big scenes loading and assigning all shaders should stay below 5 seconds. (Before optimizing on scenes with tons of objects it sometimes took over 2.5 minutes.) For this optimization I used the new Maya Python API 2.0 and some caching stuff here and there. Speeds could be even higher if this would be getting data from a database (instead of seperate files) which could be a single query (all assets at the same time).
This is all running from our network server.

Stuff to keep in mind is:
- Instances (When using object names use long names and if using the API get DAG node names using the MDagPath's fullPathName method. I'm not a huge Pymel fan myself for these kind of Pipeline things. Optimization wise the implementation in Maya 2013 was way too slow for the big scenes we had, even maya.cmds and maya.OpenMaya was lacking the speed we needed. Luckily maya.api.OpenMaya really made a difference.)

- You can't assume names of nodes stay roughly the same throughout the pipeline. Tracking of the asset should be done with some sort of data stored on the nodes (preferrably in a way that remains unchanged after caching systems.

- Naming conventions (like suffices) are nice in general, but if a production is something that ends up being rushed then human error might slip in and go unnoticed until late in the pipeline. If it's used for checking everything within the pipeline this can introduce big errors. Therefore we mainly use it for 'node types' and differences in 'quality levels' (like proxy, normal and smooth). Tools to keep track of naming conventions do help a lot though.

Hope this helps!

Tiny note about some bug with renderLayers and shader connections (older maya version only?)
Also in the past I've seen renderLayers do some disconnections with shaders and referenced objects as well. If that's happening with you then maybe check out this post:
http://mayastation.typepad.com/maya...der-layers.html

Maya included a command to fix that issue. Though save your scene up-front as I had it crash occasionally or do exactly something that wasn't intended to be there. And I'm not sure if this is even a problem with newer versions of Maya. (We had the problem back when we used Maya 2011.) Since we switched to using V-ray instead of Mental Ray I think we didn't have the problem at all anymore, but that might be from switching to a new Maya version as well!

This is actually a different issue (it's actually a bug) than what berniebernie described with the origShapes. The problem he describes is related to how deformers make a new shape as the output and use the original as an hidden intermediate shape.

Last edited by BigRoyNL : 09 September 2013 at 08:34 PM.
 
  09 September 2013
@Roy - thank you very much for your lengthy explanation. Really appreciate.

I'm curious about what you mentioned 'data stored on the nodes.' Could this node be something like a group node (with geometry as child) with custom attributes storing the ID? Or do you have a specialized custom node for it?

Last edited by Panupat : 09 September 2013 at 12:49 AM.
 
  09 September 2013
I've seen multiple solutions for this. Currently we store our own attributes on the nodes. The pipeline had to be build pretty quickly (and I'm a one man team) and this was quickest to set up and easiest to debug.

Other solutions could be:
- Seperate nodes that act as the metadata (red9 studio pipeline tools have a system for that built-in.) I remember reading some pipeline related stuff by Crytek (I think it was them) that they also implemented a similar system.

- Maya API implemented some sort of metadata system which I think is exactly for things like this. It's fairly new and I think only implemented since Maya 2014. More info here: Add metadata to Maya nodes. This has support for components as well, so you can even add metadata to faces.

I don't think there's one single way to do this, one might be better than the other at something but the other might be way better at something else. Using attributes and looking up nodes in the scene based on that can become slow if you have tons and tons of objects in the scene (we tested our pipeline with a project where we had to fill a supermarket with many actual real products, optimization was built for the bigger stuff). To make it fast enough I had to use the new Maya python API 2.0. I could have done it as C++ but didn't want to rely on compiled code for this part of our pipeline. (I do have it in the to-do list to build it in C++ and have a fallback in the code to python code if it can't find the C++ plug-in. But it's already really fast, so there's no priority for that.)

If the new Metadata API is implemented well it should be the fastest solution, but you never know with new features how well they will hold up.
 
  09 September 2013
Are the animators just doing Rclick Replace Reference in the Reference Editor? Bad to do that, and its never worked well in the past. Replace Reference doesn't always work.

You can set up an attr (low/high) on the rig
 
  09 September 2013
@refract - sort of. We do have a tool for switching resolution, but it's essentially doing reference replaced the same way I think.

By having hi-low in the rig, would that mean the hi res will need to be loaded every time even when we're working low res (but hidden)? Or is there a way to load the geo when you switch the settings?

@Roy - from what I gathered from the metadata documentation, will you still need to loop through geometries/nodes to find them? Or is there a faster way to do it with API?
 
  09 September 2013
I've not used Maya's new Metadata API as of yet. I thought they would've implemented some sort of 'find nodes with metadata value' command, because that would really make sense. Though after quickly skimming the documentation I didn't find anything along those lines.

Our system worked on custom attributes. Then you also had to loop over the scene and find the nodes with a certain attribute and the value of the attribute. I found using the API (either Maya Python API 2.0 or C++ API) the only real solution for it to keep everything fast enough.

There's a topic on Tech-Artists that might get you started if you go the attribute route: Maya: List nodes by attribute value.
It contains example code for getting the nodes with certain attributes of a certain value.
 
Thread Closed share thread



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 03:10 AM.


Powered by vBulletin
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.