View Full Version : 8 hex floats in fwrite?

11 November 2003, 11:22 AM

I am a beginner Mel-scripter, and I am trying to export some mesh animations to another file format. When I write floats or vectors using fwrite, I get 16 digit hex numbers instead of 8 hex (which I was expecting).

Looking through the web, and using a little hex->float converter I have, it seems like the most common format for floats are 8 hex (like this: 40A66666). Is there a reason why Maya does 16 hex? Can I output 8 hex instead?


11 November 2003, 01:33 PM
I have never seen Maya output floats in Hex before. Ever. And I use fprint all the time.

Could you post the code you used to output Hex? Maybe you have an option or flag set somewhere that I have never seen or used.

11 November 2003, 02:22 PM
Yup, but I am not using fprint (that's for ASCII output), I am using the binary fwrite :)

The code is really simple, try this little script:

// get a float number
float $testNum = 1.5;

$fileName = ( `internalVar -userWorkspaceDir` + "floatTest.bin" );
$fileId=`fopen $fileName "w"`;

fwrite $fileId $testNum;
fclose $fileId;

Now when you open the resulting file in a hex-capable editor (like Ultraedit), you will see a sixteen-digit hex number and not an eight-digit one (00 00 00 00 00 00 F8 3F as opposed to 00 00 F8 3F (or 00 00 C0 3F as my hex calculator says)).


11 November 2003, 03:07 PM
Whoops! My fault, I read the message and saw fprint, not fwrite. Sorry I can't help you with that.:shrug:

11 November 2003, 09:42 PM
i don't know if this really helps:
maya uses certain numeric precision defaults; i know you can query and set them somehow, or maybe not, unfortunately it escapes me where i read about it, and i've been searching the new dumbf*%ck maya5 documentation but i can't find my way around as i used to anymore, sorry!

see the maya-mel-reference on the addAttr command for pointers - it gives you an idea of how maya handles types and numbers.

then i found this:
"Precision and maximum numerical sizes

For a string, matrix, or array, the maximum size is dependent only on the amount of memory available on your computer. Floats and ints, however, have limits on their precision and maximum size.

The maximum size of an int is the same as in the C language and is machine dependent. On most computers this range is from -2,147,483,648 to 2,147,483,647.

The maximum precision and range of a float is the same as a double in the C language and is machine dependent. Floats have limited precision, and round-off errors can accumulate in long calculations. However, since the precision for the float type is so high (about fifteen digits of accuracy), round-off errors usually do not become a problem."

(from:MEL language > Advanced programming topics >Limits#Precision and maximum numerical sizes)

I suppose maybe because of the 15digit precision you need the larger hex-format, but then again: i know shit about hex, i know a little mel- that's all!


11 November 2003, 10:15 PM
Hi all,

Yeah, I've been talking to some very helpful persons at Alias support about this. And it's all down to a "limitation" in fread and fwrite. They're both designed to tread float values as doubles, which makes it impossible to read or write binary files that contain single precision floats. This is all due to the developers decision to go for maximum precisios - which is fine. I just wish the program I'm translating to had done the same!

Well well... I have decided to go the ASCII way instead. It will multiply the development time by 50, but at least I'll learn lots of MEL :)


11 November 2003, 10:01 PM
This reminds me...

There are other nasty things lurking under the hood in Maya's evaluation of floating point numbers. One is that there's a certain number (something like 1E10) that is used when enumerating values for polygon surface normals to mean that that surface normal is to be calculated from the surface itself.

If you should err and try to bypass Maya's processing of that file in a way that results in that value being set to the (supposedly equivalent) 10,000,000,000, it will no longer match Maya's "flag" value and your normals will be screwed up.

-- Mark

CGTalk Moderation
01 January 2006, 03:00 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.