View Full Version : XML for pipeline tasks

11 November 2006, 02:33 PM
How much is XML used in VFX pipelines? I keep hearing about Perl and Python, but not much about XML..

11 November 2006, 04:10 PM
well, afaik the collada format, used for digital asset exchange, is based on XML:

11 November 2006, 06:21 PM
XML is really just a text document format. This is very different from Perl/Python which are scripting panguages.

I am not entirely sure how XML would help. VFX is mainly dealing with images (exr, cineon etc) or scene files (Maya, XSI, Shake scripts etc) rather than text documents.

Most places have a database (and very tight naming conventions) to try and make sense of it all, however that is usually called with SQL from Perl, Python or inside an application eg as a Maya plugin call.

I guess you could have an XML metafile, but it would be really difficult making sure it is moved with the asset. Plus if you are going to do that, there are much simpler formats than XML. And you'd most likely embed it in a header/node within the file.

If you are always using the same few applications Collada isn't very useful - it is easier to use the native format.

Perl/Python on the other hand are used all the time for renaming files, converting files, generating RIB, connecting to databases etc.


11 November 2006, 07:44 PM
Thanks. XML is used as data carrier in a lot of other IT environments, that's why I asked.

11 November 2006, 04:00 PM
I guess you could have an XML metafile, but it would be really difficult making sure it is moved with the asset.

how so? i'm not sure what issues you'd run into with xml that you wouldn't with other meta-data types...?


11 November 2006, 05:43 PM
how so? i'm not sure what issues you'd run into with xml that you wouldn't with other meta-data types...?


Yeah you are right - this isn't really anything to do with XML in particular - just a problem of keeping 2 seperate files in sync.


11 November 2006, 11:08 PM
XML is very useful in pipelines that use a web service architecture. Class objects can be serialized easily into XML, then transferred or stored, and deserialized at the other end. Even with a central database, storing serialized objects as XML within SQL is a great way to persist pipeline data.


11 November 2006, 09:20 PM
XML is just a way to template text documents by hierarchical tags with attributes.
It is convenient as it's widespread and there are a lot of parsers/tools out there that you can use if you choose to use it for some file format.
It has advantages (clean, relatively human readable, responds well to compression algorithms etc.), and disadvantages (bloated when uncompressed, fairly rigid and prone to redundancy, can quickly become painful to preprocess if it gets big etc.).

AFAIK there is a modicum of interest into it from several directions.
In RSP we use XML templating for several of our own data formats, normally for the smallest and most agile granules, and so do other shops I've worked for or heard of, and some open formats like collada also go for it.

All in all it's not bad if you use it well and use it when its strenghts outweight its weaknesses, and it is getting some attention in our industry too, not just in webdesign, office or server solutions.

It doesn't require that much in-depth studying or extensive experience to be appreciated or used though, so it's of little concern.
It's definitely one of those things you can pick up as you go, and most parsers of different natures out there (minidom, celementstree, fasta and co.) you can normally figure out in a couple days.

12 December 2006, 12:03 AM
It seems to me XML is used quite a bit in VFX, if not XML, something conceptually similar to XML. I've worked at a couple of places that store asset descriptions in XML. The bigger studios do not store assets as scene files or even as data that can be read directly by software. They have an XML file that stores a description of a camera or a light rig or an asset and it has attributes and possibly refrences to scripts which are used to reconstruct the asset in the application when it is loaded. So, in a way, the asset doesn't really exist on disc at all, you just have this file that describes what the asset is. They do this so that the assets and shots are modular, can be easily moved between applications, and information about the assets, shots, whatever can be easily read and displayed in production tracking tools.

CGTalk Moderation
12 December 2006, 12:03 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.