View Full Version : .ma file specifications


PosingMantis
07-03-2006, 02:02 PM
Hi Everyone,

I'm very keen to know how to interpret maya ascii file so that i can debug any potential problems with files. Using one of the age old techniques called "Trial and error" on the contents of the file i'm getting hints one by one. But I need to know everything about it. I gave a search here and read a couple of threads on .ma files but i didn't find links that would lead me to a kinda complete documentation or specification on .ma files.

So it'll be helpful if anyone shares any links/info on the same.

Bonedaddy
07-03-2006, 02:10 PM
There's a page in the Maya docs that describes how .ma files are set up. If I recall correctly, they only use 6 different MEL commands, and have some fancy stuff that has to be included at the beginning -- and that's it. It's really just knowing how to use those 6 MEL commands.

JamesPiechota
07-04-2006, 12:31 AM
Yup. It's under:

Developer Resources > File Formats > Maya ASCII File Format

PosingMantis
07-04-2006, 05:39 PM
Hi Guys,

Thanks a lot for the hints. There is indeed a whole lotta info out there regarding .ma files and their organization.

Jason, there r not 6 but 11 commands that are supported:

file
requires
createNode
setAttr
addAttr
parent
connectAttr
disconnectAttr
select
currentUnit
fileInfo

Cheers.

PosingMantis
07-05-2006, 05:25 PM
When going through the .ma file specs, I came accross a section that says that even mel scripts can be referenced into any maya scene. After discussions with one of my collegues about the same, we wondered what possiblilities would that feature lead to. I created a mel script that created a sphere and printed "hello world" :)). Then referenced it into a new maya scene and noticed that the script executes whenever I load the reference and on unloading, deletes the sphere. The sphere has got the same restrictions that any object would get when referenced in the form of a file. The only difference is that when it is referenced from a file, the namespace is prefixed with the object and when created from a referenced script, the prefix is missing.

That's about my observations till now. But my questions are:
1. What are the uses/advantages of referencing a mel scripts?
2. What are the other differences between referencing objects from a file and getting the objects created by a referenced mel script?

One of the uses i see in referencing scripts is to append notes with files. If we script out a popup window containing notes and reference that script in a file, then everytime they open that file, they'll see that note. But i guess there will be much more use than that.

PosingMantis
07-07-2006, 05:18 PM
It cant be!! There must be someone who knows about referencing scripts, who has used this feature in production. Bonedaddy? James? Alias masters?;) anybody?

Bonedaddy
07-07-2006, 05:55 PM
File referencing in Maya is tetchy, and most around here don't recommend it. Off the top of my head, some of the problems include:

- Light linking
- Shader assignment
- Set assignment
- Render layers
- Visibility layers


That's for .mb or .ma files. I didn't know you could reference .mel files, but it does make sense, since a .mel file is basically the same as a .ma file, except a .ma is limited in the number of functions it can call, as noted above.

I am not entirely clear on how the file referencer works, but I am led to believe that it constructs a large list of everything in the scene that is from a certain file by examining the scene before and after the file import. Everything that is new in the scene after the import, it marks as from a certain file. Thus, when you "unreference" something, it runs through and deletes everything on that list.

That's my assumption. I'd think that with any serious MEL usage, however, it'd break. Like if you start setting attributes to random objects, and then unreference -- I doubt it'd reset the attributes to their original settings. I haven't tested this, though, so I could be wrong.

Most of this is inferred speculation on my part. If anyone (Robthebloke? John Homer? drGonzo?) wants to chime in with a bit more of an educated opinion, please feel free.

PosingMantis
07-07-2006, 06:52 PM
Thanks for the info Jason.

I agree that file referencing is quite techy, but not really buggy. As per my experience working on a pipeline that is built on referencing, its very helpful in a way that it makes it easy to re-work on anything in an earlier stage which can be propogated down the pipe in no time. Though it demands strictness to some degree with proper understanding, its easy to troubleshoot problems with referencing.

A brief on how maya referencer works:
Maya has a parent child hierarchy with respect to referencing also. The file that is being referenced is called child and the file that is referencing another file is called Parent.
For every referenced child file, maya creates a "Reference node" in the parent scene. Maya uses this node to store any changes/edits to the child scene in the parent scene, which means that any edit, right from a simple translation of objects to texturing objects to animating them, every little change in state of objects in child file, there is an entry made in the respective reference node. When reference is unloaded, the changes still remain in this node. When reference is reloaded, these changes are again applied to the child reference objects.

Maya also provides ways by which we can query, edit and even clean up these changes (or " reference edits") manually or by script. So troubleshooting reference file problems is all about understanding which of those edits has caused the problem and editing/removing those edits to set it right.

Consistency only matters when there is a link in some way between nodes in parent file to those in child file. In general, it matters for every object in the child scene that is transformed in some way in the parent scene. Everytime the reference is unloaded and reloaded, maya attempts to apply the stored changes back to the child scene's nodes, and of it fails to apply changes due to unavailability of some nodes, it throws problems.

I found referencing very interesting. Refer to the detailed info on mechanism of referencing in maya help.

But as of now I still have one question unanswered-
1. Why reference a .mel file into the scene? How can that feature be used?

JamesPiechota
07-08-2006, 02:31 AM
Very nice overview PosingMantis.

Referencing .mel scripts has been around since the early days of Maya - but I don't believe it has gotten much use in production. That's a bad sign in terms of stability.

I have heard of a couple studios who used referenced MEL scripts in their pipeline - if my memory serves me they were using it as a way to segment a scene. For example you could put shading assignments in their own MEL file and then reference that in.

The list of caveats though, is huge - particularly because this has been a little-known and even less-used feature for so long.

For example, make a script that creates a sphere and reference that script in. The sphere is correctly created (without a prefix or namespace) but you'll note that edits made to that sphere are not correctly stored on and re-applied from the reference node - you may also notice that after you save and re-open the scene the MEL reference is not loaded by default (not only that but it has become unassociated with its original reference node, and new one will be created).

However, make a scene that has a cube and sphere, and write a script that connects an attr on the cube to one on the sphere. Referencing that script connects the cube and sphere correctly - if you remember to reload the reference each time the scene is opened then you'll get that connection recreated each time.

Since Maya's referencing system does not allow references to modify the files they are referenced into (e.g. children can't modify parents), referencing scripts that modify existing nodes without creating new ones of their own, can be a potentially useful/powerful extension.

The potential for trouble, though, is such that I'd avoid using MEL references for this unless you have a pretty large pipeline, can enforce some pretty strict rules (either through plugins/callbacks or shear force of will), and sufficient engineer resources to troubleshoot and maintain.

PosingMantis
07-10-2006, 07:10 PM
Thanks for the info and hints James. I tried all things u mentioned and most of them worked except this one.
make a script that creates a sphere and reference that script in. The sphere is correctly created (without a prefix or namespace) but you'll note that edits made to that sphere are not correctly stored on and re-applied from the reference node This worked properly applied all the edits correctly on the sphere after loading again.

I'm right now exploring how the reference edits are stored in the .ma file. I've already interpretted many lines and codes it puts in to indicate the edits. I'll keep posting as and when i come to interesting conclusions.

I think learning how to edit .ma mainly from the "file referencing" pov help a lot in building a successful and efficient prod pipeline. So guys please keep posting anything u know about it.

Cheers.

Bonedaddy
07-10-2006, 09:29 PM
Yes, please do. I'm trying to figure out some pipeline stuff here and we've been looking warily at file referencing. We have a specific list of problems, and if I could overcome them with a bit of scripting, it'd be a huge relief.

JamesPiechota
07-11-2006, 01:17 AM
This worked properly applied all the edits correctly on the sphere after loading again.

My mistake - on Maya 7.0 the problem seems to crop up when saving and re-opening the scene. Because it's a MEL reference on file open the referenced file isn't correctly re-associated with its reference node.

We have a specific list of problems, and if I could overcome them with a bit of scripting, it'd be a huge relief.

Shoot. If I can help, I'd be happy to.

Are you all working with Maya 6.5 or 7? The lay of the land changed significantly between 6 and 6.5.

PosingMantis
07-18-2006, 07:08 PM
Hi Frens,

I return with a list of interesting observations after doing a good study on .ma file reference edits. For the study I setup reference (parent) scenes and modified the referenced objects in different ways. This ranged from a simple translation edit to complex edits like constraints, hierarchies and also modifying skinning information on the referenced nodes. This is not a complete study though and I'd very much appreciate if others add on to this or correct me if i've understood anything wrongly.

My conclusions are as follows:
Any reference edit to a file that's being referenced with a namespace "ref" can be found in the .ma file starting with the code: "createNode reference -n "refRN";". The list ends the line before the next "createNode" command. This block contains lot of setAttr commands going by the rules of .ma file that after every "createNode", a set of "setAttr" commands follow. The code looks something like this.
createNode reference -n "refRN";
setAttr ".ed" -type "dataReferenceEdits"
"refRN"
"refRN" 0
"refRN" 1
2 "|ref:nurbsSphere1" "translate" " -type \"double3\" 0.984522 0.648003 -1.95567";
setAttr ".ptag" -type "string" "";
The above code is an extract from .ma file of a parent scene that references a child scene containing a "nurbsSphere1" object which has been translated by "0.984522 0.648003 -1.95567". The three lines that start with "refRN" word appeared same in all cases i studied.
The digit "2" that is seen in the second last line indicates "setAttr" edit. It is 5 for "connectAttr", 0 for "parent" edit and 1 for "addAttr" command.
The number 1 after the last "refRN" line stands for the number of edit entries that follow it. In this case, since the only edit is a "setAttr" edit of the "nurbsSphere1" it shows 1.

The "connectAttr" edit has a bit more detail. "connectAttr" edit is created when there is a connection between a node in the parent scene to a node in the child scene. The code is:
createNode reference -n "refRN";
setAttr ".phl[0]" 0;
setAttr ".ed" -type "dataReferenceEdits"
"refRN"
"refRN" 0
"refRN" 1
5 4 "refRN" "|ref:nurbsSphere1.translateX" "refRN.placeHolderList[0]"
"";
setAttr ".ptag" -type "string" "";
Above code is a from a file in which only translateX of "ref:nurbSphere1" has been keyed.
Notice that there is a "setAttr ".phl[0]" 0;" line that is added. phl stands for "placeHolderList" and a phl plug is created in the "refRN" node for every connection from a node in the parent scene to another node in the child scene. Also after the first number which is 5, there comes another number. 4 stands for connection from node in parent scene to a node in child scene. 3 stands for the other way connection.

That's about my study so far. I have a few questions unanswered yet. They are:
1. What does the line - "refRN" 0 signify? Are there cases where some entreis come between that line and the next "refRN" line?
2. I've mentioned that 0 stands for "parent" edit, 1 for "addAttr", 2 for "setAttr" and 5 for connectAttr. What does 3 and 4 signify? Or are there any other command that is represented by some other code.
3. In case of connectAttr command, i said, 4 stands for incoming connection and 3 for outgoing. Is there any other kind of connection specified by the second code?

If anyone knows anything more about reference eidts, please post. Also BoneDaddy, please shoot your problems regarding file referencing coz even i'm looking forward to load myself with solutions to all kinds of file referencing problems.

Thanks.

JamesPiechota
07-19-2006, 08:31 AM
I've been meeting to write up a little doc on Referencing - unfortunately time's tight so hopefully this will help for now:

1. What does the line - "refRN" 0 signify? Are there cases where some entreis come between that line and the next "refRN" line?

One of the "refRN #" lines stores loaded edits and the other unloaded edits. As of 6.5 all references are stored on disk in the unloaded state. This means that if the reference is loaded when you save the scene Maya has to implicitly unload it - basically it goes through the steps of finding and building the reference edits without actually removing the referenced nodes from the scene. To keep these edits separate from edits which were created due to unloading a reference, Maya sticks them in a special place and writes them out in their own "refRN #" block. After the save operation completes these edits are deleted. To give this a whirl try saving the same scene twice, once with the reference loaded and once with it unloaded.

2. I've mentioned that 0 stands for "parent" edit, 1 for "addAttr", 2 for "setAttr" and 5 for connectAttr. What does 3 and 4 signify? Or are there any other command that is represented by some other code.

3 is disconnectAttr
4 is deleteAttr

A deleteAttr edit will be created if you delete a dynamic attribute from a referenced node.

A disconnectAttr edit will be created if you disconnect a connection which was made in a referenced file. If you connect two referenced nodes and then break that connection, neither the connection nor the disconnection will be stored as edits (the end result is that there has been no change to the referenced nodes). However if you disconnect a connection that was made in the referenced file a disconnectAttr edit will be created. Things get a little hairier when dealing with multiple levels of referencing - but the same ideas should hold true.

3. In case of connectAttr command, i said, 4 stands for incoming connection and 3 for outgoing. Is there any other kind of connection specified by the second code?

Code 0 is for a connection between two nodes in the same reference.

The other codes are only used when multiple references are involved. For example:



Create a reference to a sphere
Create a reference to a cube
Connect the sphere's translate to the cube's translate
Save
If you open up the parent scene, you'll see that there are two connectAttr edits, one stored on the sphere's reference node (e.g. sphereRN) and one on the cube's reference node (e.g. cubeRN). The edit stored on sphereRN has code 1, and the edit stored on cubeRN has code 2 - two edits are necessary in order to handle cases where only one of the two references is loaded. A similar situation can be created with multiple levels of references.

PosingMantis
07-23-2006, 06:32 PM
Hey James! Thanks again for the answers. I'll be back after more experiments.

Cheers.

llikethat
08-31-2006, 02:41 PM
Guys,

This is a wonderful thread. Lets add more information on to this. This topic is so very interesting too.

I have a query about .ma files thought this would be a great thread to post. We all know that .ma files are big in size(takes too long to load in maya) but good for editing and stuff. Is there a way to compress or zip a .ma file and then make the operating system to Unzip it on the fly when Maya actually loads the fle.

Coz' when u reference to many things on a scene. Then save it as a .ma file. The load time shoots up. That's the reason i wanted to know about it.

Any suggestions....

JamesPiechota
09-01-2006, 01:06 AM
I haven't used it much but there is an option under:

Settings And Preferences > Preferences > Files/Projects

To turn on compression when saving Ascii files ("Ascii File Compression Mode").

In order for this to work you have to have a "zip utility in your path" - in other words this works no problem on Linux, but you may have to do some fiddling on Windows (don't know about Mac). Unfortunately I haven't done that fiddling so I'm not sure what command name and flag arguments Maya is expecting.

On Linux, though, enabling this option will automatically gzip MA files when saving, and gunzip them when opening.

That being said, I'm not sure this will help with your problem. Zipping/unzipping a file takes time, so using compression will actually *increase* your load time. The only way I might see it helping is if you are loading files over a very slow network - it's possible that that transfer time on large files outweights the time it takes to zip/unzip the file. Even in that case, though, you'd want to do some testing to make sure Maya does the unzipping after transferring the file over the network (which I'm not so sure about).

JamesPiechota
09-01-2006, 01:14 AM
As for your original load time problem -

If you are referencing many small .ma files you might try turning off saving of the UI Configuration Node. This node saves the state of UI layout in the scene file so that it can be restored when the file is next opened - however the UI Configuration Node is ignored in referenced files and since it is fairly long it can add significantly to the size/length of a small file.

To disable saving of the UI Configuration node just type:

file -uc false;

Before saving. You can also make the change in your initialStartup.mel file.

JamesPiechota
09-01-2006, 01:19 AM
A couple more ideas:

Depending on your needs you may get some mileage out of using the Preload Reference Editor or simply opening the files with All References Unloaded. Both of these options will allow you to defer the loading of referenced files - if you don't care about some of the referenced parts of the scene at the moment, simply uncheck them in the Preload Reference Editor and Maya won't load them. If you need that data later you can always manually load them using the Reference Editor.

Finally you could always try saving your referenced files as MB. I don't like to suggest this because I've hit a lot of problems that required hacking an MA file to fix and would have been unfixable had the file been saved as MB.... but you can get a significant performance and disk space improvement by using MB, and if you limit it to files which do not contain references, you shouldn't hit *that* many problems.

llikethat
09-01-2006, 09:27 AM
That was an excellent piece of information James. I tried compressing the files with the option you'd specified, the load time didn't have any significant difference. As far as the preload reference editor, it works well only if we are working on a scene file. But when we submit a job on a renderfarm, this will not serve the purpose. That was an excellent piece of information James thanks again.

CGTalk Moderation
09-01-2006, 09:27 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.


1