Neuro69
05-28-2008, 11:22 AM
I'm trying to write a script that converts UTM coordinates to geographic lat/long coordinates. The formula contains sections where small numbers (stored as double) are raised to the power of 5 or 6. These end up getting truncated to zero, even though Windows Calculater show results in the range ~ 2e-10. Just testing input directly in the Listener, it appears that anything smaller than 1d-6 is truncated to zero. I've tried formattedPrint as well, to see if it was only the display, but it makes no difference. Reading about double precision floats, I'm a bit confused about the different notations used, but if I understand correctly, double values should have a lower range limit around 2.225d-308? Am I missing something obvious, or are there issues with the maxscript number implementation?