PDA

View Full Version : Shortening a float with "match" yields incorrect results

 DeadlyNightshade11-12-2012, 08:16 PMglobal float \$angle = -45; // The following will reduce the float to 4 decimals only. int \$negNumber = 0; if (\$angle < 0) { \$negNumber = 1; print ("negative number\n"); } \$angle = `match "[0-9]+[.]*[0-9][0-9][0-9][0-9]" (string(\$angle))`; if (\$negNumber == 1) \$angle = -\$angle; print ("Shortened angle value is: " + \$angle + "\n"); // Shortened angle value is: -0 This always happen when to \$angle variable gets fed a number that lack decimals. I also get this warning: // Warning: line 9: Converting string "" to a float value of 0. // So I understand what is going on - but I don't know what to do to avoid it. I need to reduce the float \$angle variable to -4- decimal and so far I've used "match" to do this - but appearently it's not flawless. Maya doesn't understand that I want to keep it as a float, so it auto-converts it to a faulty string.
djx
11-12-2012, 08:23 PM
Multipy by 10000, recast as int, recast as float, divide by 10000.0

David

edit: I remember seeing this somewhere...

float \$r = 1.0/3.0;
print (\$r + "\n");
float \$s = python("'%0.4f'%" + \$r);
print (\$s + "\n");

11-12-2012, 09:28 PM
Thanks.
I found out that redeclaring a variable needs to be in a separate procedure though - it took me a few minutes to realise that after getting tons of "Invalid redeclaration" -errors.

NaughtyNathan
11-13-2012, 08:13 AM
silly rabbit, match is for strings.
This is kinda like saying "sharpening a pencil with "hacksaw" yields incorrect results" ;)

David's already got it spot on, but here it is in function form anyway if you want pure MEL:
global proc float truncate(float \$value, int \$dp, int \$round)
{
float \$r = \$round*0.5*sign(\$value);
return (trunc(\$value*pow(10,\$dp)+\$r)/pow(10,\$dp));
}

the parameters are to make it flexible and versatile and are: number to operate on, number of decimal places to work to and the final parameter is whether you want it to round the value, or simply truncate it. example usage:
truncate -4.1956025 1 true;
// Result: -4.2
truncate 4.1956025 3 false;
// Result: 4.195

11-13-2012, 09:46 AM
silly rabbit, match is for strings.