Problem with MAXScript math

[erilaz]

I'm assuming it's some kind of floating point error. I get the same thing. Perhaps something to do with the overflow?

[marktsang]

hi,
not sure its returning the wrong value but you can use ceil to force the correct value
ceil(pow 2 ((log10 (n-1))/(log10 2))) as integer

mark

[stuh505]

Well, the thing is that it's not consistent. When I evaluate it now I get different results.

[marktsang]

would you like to post the results?

mark

[Bobo]

What the heck?



Edit -- After restarting the math is evaluated correctly...this is not the first time it has behaved like this, does anyone else have erratic evaluations like this?[/color][/size]

Welcome to the single precision world!
It IS a floating point rounding error:


n = 33
32000.0 - (pow 2.0 ((log10 (n-1))/(log10 2))) * 1000
0.0117188


As you can see, the result of your expression is slightly less than 32.0, but very very close - the difference is just around 0.0000117188...

Since AS INTEGER always rounds down,

0.9999999 as integer
0


you are getting 31 from your calculation.

To automatically round up or down depending on whether the decimal part is > 0.5 or less than 0.5, you can add 0.5 to the result and then turn to integer.

( 0.5 + (pow 2.0 ((log10 (n-1))/(log10 2))) ) as integer

If the result of a calculation was, say, 12.4, 12.4+0.5 = 12.9, and the integer would be 12. If it was 12.6, 12.6+0.5 = 13.1, so the integer would be 13, correctly rounded up.

[stuh505]

Ok, it's a floating point round off error :P I have run into that problem before. I still don't know why it is not consistently wrong but I guess max must have some dynamic stuff in there for evaluation optimization that results in different results when it is near a round off point

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.