PDA

View Full Version : Shortening a float with "match" yields incorrect results


DeadlyNightshade
11-12-2012, 08:16 PM
global float $angle = -45;

// The following will reduce the float to 4 decimals only.
int $negNumber = 0;
if ($angle < 0)
{
$negNumber = 1;
print ("negative number\n");
}
$angle = `match "[0-9]+[.]*[0-9][0-9][0-9][0-9]" (string($angle))`;
if ($negNumber == 1) $angle = -$angle;
print ("Shortened angle value is: " + $angle + "\n");
// Shortened angle value is: -0

This always happen when to $angle variable gets fed a number that lack decimals.
I also get this warning:
// Warning: line 9: Converting string "" to a float value of 0. //

So I understand what is going on - but I don't know what to do to avoid it.
I need to reduce the float $angle variable to -4- decimal and so far I've used "match" to do this - but appearently it's not flawless. Maya doesn't understand that I want to keep it as a float, so it auto-converts it to a faulty string.

djx
11-12-2012, 08:23 PM
Multipy by 10000, recast as int, recast as float, divide by 10000.0

David

edit: I remember seeing this somewhere...

float $r = 1.0/3.0;
print ($r + "\n");
float $s = python("'%0.4f'%" + $r);
print ($s + "\n");

DeadlyNightshade
11-12-2012, 09:28 PM
Thanks.
I found out that redeclaring a variable needs to be in a separate procedure though - it took me a few minutes to realise that after getting tons of "Invalid redeclaration" -errors.

NaughtyNathan
11-13-2012, 08:13 AM
silly rabbit, match is for strings.
This is kinda like saying "sharpening a pencil with "hacksaw" yields incorrect results" ;)

David's already got it spot on, but here it is in function form anyway if you want pure MEL:
global proc float truncate(float $value, int $dp, int $round)
{
float $r = $round*0.5*sign($value);
return (trunc($value*pow(10,$dp)+$r)/pow(10,$dp));
}

the parameters are to make it flexible and versatile and are: number to operate on, number of decimal places to work to and the final parameter is whether you want it to round the value, or simply truncate it. example usage:
truncate -4.1956025 1 true;
// Result: -4.2
truncate 4.1956025 3 false;
// Result: 4.195

DeadlyNightshade
11-13-2012, 09:46 AM
silly rabbit, match is for strings.
I blame it on bad advice:
http://forums.cgsociety.org/showthread.php?f=89&t=1066703
Picked that method back then because it looked easiest (didn't need a separate procedure for it) and it seemed to work - up until now.

djx
11-13-2012, 11:07 AM
"bad advice"? That's a bit unfair. I (obviously) remembered that thread too, and I think those guys gave you some good advice and several different methods. You just picked the wrong method for your needs. Live and learn I guess. Credit where credits due though.

David

CGTalk Moderation
11-13-2012, 11:07 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.