CGTalk > Techniques > Texturing and Surfacing
Login register
Thread Closed share thread « Previous Thread | Next Thread »  
 
Thread Tools Search this Thread Display Modes
Old 02-12-2013, 12:41 PM   #1
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
Texture formats for VFX

Hey all,

What is your prefered texture format, and why?

Dealing with a great amount of textures I feel like I have to account for several different thing to optimize for rendering. Loading thousands of textures from disk quickly becomes a bottleneck and I'm curious to know how you solve it.

Do you convert to any mipmap format such as .tdl or tiled exr (note, not uv-tiles)
 
Old 02-12-2013, 04:49 PM   #2
Dillster
Always Learning
 
Dillster's Avatar
portfolio
Dylan Saunders
Dublin, Ireland
 
Join Date: Jan 2011
Posts: 2,903
Personally I just use jpegs, and occasionally png.
__________________
I like to learn.
 
Old 02-12-2013, 09:18 PM   #3
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
I don't you suffer a great loss of detail using 8-bit formats? I would assume that 16-bit tiffs/exr might be beneficial. Especially if you can use multiple channels in a exr. Diffuse, specular and reflection maps are usually always loaded at the same time anyways. I'd guess there could be some benefit of using such a format in a mipmap fashion.

I'm curious to know what pipelines people are using and jpegs and pngs have the benefit of low file sizes, and therefore low file I/O
 
Old 02-13-2013, 03:07 AM   #4
leigh
blahblah
 
leigh's Avatar
CGSociety Staff
portfolio
Leigh van der Byl
A cog in the wheel
Hertfordshire, United Kingdom
 
Join Date: Feb 2002
Posts: 29,791
The format we've used in most studios I've worked at is TIFF. The only exception was one studio I worked at that used .exr for some reason I could never really understand.

Regarding bit depth, depending on the show we'd sometimes use 16 bit for colour maps (in linear colourspace, and using a show-specific LUT) but mostly we'd just use 8 bit, and always use 8 bit for bump, spec, etc. Of course displacement maps are always 16 bit though.
__________________
leighvanderbyl.com
 
Old 02-13-2013, 04:08 AM   #5
Panupat
Expert
 
Panupat's Avatar
portfolio
Panupat Chong
Thailand
 
Join Date: Feb 2004
Posts: 1,942
After the look dev is done I run scripts to convert all textures to tdl/tiled exr and reapply them automatically.

I rarely use jpg. It's usually tiff and tga.
 
Old 02-13-2013, 07:42 AM   #6
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
Interesting, I've been thinking a lot lately about this since I've found myself in a project with 200+ small textures. I can really see the benefit of using mipmaps as to not have more texture resolution than needed. But I fear that converting a great amount of lightweight jpegs to heavier but tiled exr's might increase file loading times during rendering. I do like the .tdl approach, getting a directory of The different levels. I'm thinking tiled exr's work in a similar way, except they are contained in one file. My question is if it's possible to load a single tile into memory, i know that the opposite happens during rendering of buckets.

This might be more of concern to light and render artists but it might be something texture artists can have in mind?

It strikes me that Mari might be using exr internally, so a workflow based on multiple channels rather than multiple maps might already be in place in some texturing/rendering pipelines?

Last edited by molgamus : 02-13-2013 at 07:45 AM.
 
Old 02-13-2013, 09:02 AM   #7
scrimski
antieverything
portfolio
J. Schreyer
head->wall
ARRI Berlin
Berlin, Germany
 
Join Date: Aug 2003
Posts: 7,983
Quote:
But I fear that converting a great amount of lightweight jpegs to heavier but tiled exr's might increase file loading times during rendering.

Jpegs aren't lightweight, they need to be decompressed before anything can be processed.
 
Old 02-13-2013, 10:08 AM   #8
Panupat
Expert
 
Panupat's Avatar
portfolio
Panupat Chong
Thailand
 
Join Date: Feb 2004
Posts: 1,942
Quote:
Originally Posted by molgamus
But I fear that converting a great amount of lightweight jpegs to heavier but tiled exr's might increase file loading times during rendering.

3Delight converts all textures to tdl during scene translation (not sure about Vray but I guess it does as well). When your artists send files over to render farm, there are chances that every render node will have to spend some time converting the same textures over and over. So it's better to get it done before that imo.

If you use jpg (and anything with compression) then the converters will need to decompress before it can process the image. The extra time spent decompressing is really noticeable. Even more so with large file size.

Last edited by Panupat : 02-13-2013 at 10:27 AM.
 
Old 02-13-2013, 12:38 PM   #9
forelle
Know-it-All
portfolio
Jack Greasley
London , United Kingdom
 
Join Date: Mar 2010
Posts: 340
Quote:
Originally Posted by molgamus
It strikes me that Mari might be using exr internally, so a workflow based on multiple channels rather than multiple maps might already be in place in some texturing/rendering pipelines?


Mari uses it's own, slightly unique, method for storing textures, but the closest thing out there would be zip compressed, tiled, mip-mapped exrs.

For anything other than trivial scenes you REALLY, REALLY want to use a tiled mip-mapped texture format. As mentioned else where a JPG needs to be loaded and decompressed in full and then mip-mapped by the renderer. A 20kb on-disk JPG could take up 30Mb of memory and always will, no matter where the camera is or how much of the texture is visible.

A tiled texture, no matter how large on disk, can be very accurately monitored. If you only need 128x128 pixels out of a certain mip-level of a 32kx32k texture, that is all you need to load.

This is basically how Mari and all production renderers deal with it. Work out exactly which tile needs loading and load that, once done with it, get rid of it. The mip-map is pre-calculated so the lookup is trivial.

USE TILED, MIPPED TEXTURES.

Oh and JPG is in general a truly horrible format for textures. Use PNG, EXR, TIF or anything other than jpg.

Last edited by forelle : 02-13-2013 at 12:44 PM.
 
Old 02-13-2013, 03:40 PM   #10
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
I personally try to stay away from jpegs as much as possible. But this time I had no say in the matter. I could of course convert and re-assign them but the word around the office is that mipmaps could change the look sligthly. If I get time I'll test and compare. But I would still argue that a compressed image would put less strain on the network. Or can you actually read a specific tile or do you have to load the whole file?

Last edited by molgamus : 02-13-2013 at 06:39 PM.
 
Old 02-13-2013, 03:48 PM   #11
forelle
Know-it-All
portfolio
Jack Greasley
London , United Kingdom
 
Join Date: Mar 2010
Posts: 340
Quote:
Originally Posted by molgamus
I personally try to stay away from jpegs as much as possible. But this time I had no say in the matter. I could of course convert and re-assign them but the word around the office is that mipmaps could change the look sligthly. If I get time I'll test and compare. But I would still argument that a compressed image would put less strain on the network. Or can you actually read a specific tile or do you have to load the whole file?


The renderer will be mip-mapping them anyway, so you either pay the cost once on creation or every frame during rendering.

You can absolutely read a specific tile from the file without having the load the whole thing. I would be very surprised if any renderer specific texture format does not do this.

I know that ptx from renderman certainly does.

You can render many, many GBs of textures with a fixed memory footprint. Mari handles 100s of Gbs of textures in this way in real time. The correct tile, from the correct mip level is loaded. If you have an object with a huge texture applied that is very small on screen, only a very small amount of memory will be used, as only a single tile from the file needs to be loaded.
 
Old 02-13-2013, 03:57 PM   #12
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
This is reassuring, I'm using V-Ray and I guess it would make sense that the renderer does some sort of mipmapping that's hidden from the user. But in this case we don't filter our textures in maya or vray settings.

I'd prefer paying the cost once of course, however I'm not sure I wan't to convert the whole asset librarys textures. I'll have to give this some thought before I commit.
 
Old 02-14-2013, 01:02 PM   #13
molgamus
Frequenter
portfolio
Elias
Stockholm, Sweden
 
Join Date: Feb 2010
Posts: 165
Here's a little script I did for converting all textures in the scene to tiled EXR. It also connects the file nodes to the new tiled textures. Made for linux and V-Ray for Maya 2012.
Code:
def tex2exr(): import maya.cmds as cmds import os fileTex = cmds.ls(type = "file") print fileTex for each in fileTex: preName = cmds.getAttr(each+".fileTextureName") ext = preName.rsplit(".")[1] print "image is " + ext +":" if ext != "exr": os.system("/usr/ChaosGroup/V-Ray/Maya2012-x64/bin/img2tiledexr " + preName); postName = (preName.split(".")[0] + "_tiled.exr") print postName cmds.setAttr(each+".fileTextureName", postName, type="string") print "converting..." else: print "skipping..."
 
Old 02-14-2013, 01:02 PM   #14
CGTalk Moderation
Expert
CGTalk Forum Leader
 
Join Date: Sep 2003
Posts: 1,066,478
Thread automatically closed

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.
__________________
CGTalk Policy/Legalities
Note that as CGTalk Members, you agree to the terms and conditions of using this website.
 
Thread Closed share thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 11:00 AM.


Powered by vBulletin
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.