PDA

View Full Version : Remove duplicates from INT/FLOAT array


sagunmanandhar
01-07-2011, 09:42 PM
hi,
im looking for something like stringArrayRemoveDuplicates() that works with int / float array. pls help

NaughtyNathan
01-08-2011, 01:19 AM
this is not really a hard funtion to write yourself, but you may have already noticed that stringArrayRemoveDuplicates is actually a MEL script, not a command. As this is the case you can just open it up and copy it. then just modify it slightly to work on ints or floats depending on your needs (it must be one or the other as MEL must return a specific type.)

:nathaN

norbertnacu
01-13-2011, 08:56 PM
global proc int[] intArrayRemoveDuplicates( int $intArr[] )
{
int $resuts[];

for( $intAr in $intArr )
{
int $isFound = false;
for( $resut in $resuts )
{
if( $intAr == $resut )
{
$isFound = true;
break;
}
}
if( $isFound == false )
$resuts[size($resuts)] = $intAr;
}

return $resuts;
}

intArrayRemoveDuplicates( {5, 5, 65, 65, 33, 22, 1} )


global proc float[] floatArrayRemoveDuplicates( float $floatArr[] )
{
float $resuts[];

for( $floatAr in $floatArr )
{
int $isFound = false;
for( $resut in $resuts )
{
if( $floatAr == $resut )
{
$isFound = true;
break;
}
}
if( $isFound == false )
$resuts[size($resuts)] = $floatAr;
}

return $resuts;
}
floatArrayRemoveDuplicates( {1.0, 1.0, 2.5, 3.2, 2.5, 1.0, 58.88} )

Thank You,
nn

ewerybody
01-14-2011, 03:58 PM
nice! could even be more elegant with also useful isInArray functions:

global proc float[] floatArrayRemoveDuplicates( float $array[] )
{
float $resuts[];
int $count = 0;

for( $item in $array )
if ( !isInFloatArray($item, $resuts) )
$resuts[$count++] = $item;

return $resuts;
}

global proc int isInFloatArray( float $value, float $array[] )
{
for ( $item in $array )
if ( $item == $value )
return true;

return false;
}

Keilun
01-14-2011, 07:56 PM
Given the limited precision of floats, the == operator on float isn't always what you want. Adding an optional tolerance value might be useful. I feel like there's something to that effect already in MEL somewhere, but I can't recall it.

Edit: Ah yes:

int equivalentTol(float $a, float $b, float $tol)

Examples:
equivalentTol(0.0, 0.0005, 0.001);
// Result : 1 //
equivalentTol(0.0, 0.005, 0.001);
// Result : 0 //

ewerybody
01-17-2011, 11:38 AM
hmm ... true but I'd rather implement that in an additional function to keep the real thing slim.

maybe like:
global proc float[] floatArrayRemoveDupTol( float $array[] , float $tol )
{
float $resuts[];
int $count = 0;

for( $item in $array )
if ( !isInFloatArrayTol($item, $resuts, $tol) )
$resuts[$count++] = $item;

return $resuts;
}

global proc int isInFloatArrayTol( float $value, float $array[], float $tol )
{
for ( $item in $array )
if ( equivalentTol($item, $value, $tol) )
return true;

return false;
}

floatArrayRemoveDupTol({0.1, 0.10001, 0.2, 0.23}, 0.03);
// Result: 0.1 0.2 //

denisT
01-18-2011, 04:21 AM
to keep the real thing slim...



floatArrayRemoveDupTol({0.07, 0.1, 0.04}, 0.03);
floatArrayRemoveDupTol({0.04, 0.1, 0.07}, 0.03);
// Result: ??? //

ewerybody
01-18-2011, 11:16 AM
ehmm... yes? :shrug:

denisT
01-18-2011, 05:49 PM
floatArrayRemoveDupTol({0.07, 0.1, 0.04}, 0.03);
floatArrayRemoveDupTol({0.04, 0.1, 0.07}, 0.03);
// Result: ??? //

i just want to say that the method might be not accurate but it has not to depend on order of array items.
the solution #1 is to sort a float array before the removing duplicates using tolerance.
but the method i use is to ROUND array values using some precision, and collect only unique after that:

global proc float roundFloat (float $f, float $precision)
{
$f /= $precision;
float $f1 = floor($f);
float $f2 = ceil($f);
float $v = (($f - $f1) > ($f2 - $f)) ? $f2 : $f1;
return ($v*$precision);
}
global proc int findFloatItem (float $item, float $array[])
{
for ($k = 0; $k < size($array); $k++) if ($item == $array[$k]) return $k;
return -1;
};
global proc float[] uniqueRoundFloatArray (float $array[], float $precision)
{
float $arr[], $i;
for ($item in $array)
{
$i = roundFloat($item, $precision);
if (findFloatItem($i,$arr) == -1) $arr[size($arr)] = $i;
}
return $arr;
// return (sort($arr)); // if you like
}
uniqueRoundFloatArray ({0.1, 0.10001, 0.2, 0.23}, 0.1);
// Result:0.1 0.2//
uniqueRoundFloatArray ({0.1, 0.10001, 0.2, 0.23}, 0.01);
// Result:0.1 0.2 0.23//

haggi
01-18-2011, 09:26 PM
Okay, this is maybe a little bit overdone but it works:

global float $ffArray[] = {1.0, 2.3, 4.4, 99, 2.3, 4, 2.3};
float $sorted[] = python("import pymel.core as pm; list(set((pm.getMelGlobal(\"float[]\", \"$ffArray\"))))");
print $sorted;

ewerybody
01-19-2011, 10:41 AM
@denis: true but I wasn't asking for a uniquifier with tolerance ;] this just was an example for the closest way to go.

@haggi: wow! But this doesn't fix the tolerance thing or does it? I'm too scared to try :D

denisT
01-19-2011, 01:45 PM
@denis: true but I wasn't asking for a uniquifier with tolerance ;] this just was an example for the closest way to go.

i wasn't answering to you. i just tried to warn against using a shortest way rather than the right one.

ewerybody
01-19-2011, 03:12 PM
i wasn't answering to you.
thats right. You qouted me and then said something seeming unrelated ??? Instead of saying what the problem is. Which I understand as well: A tolerance might not be what you want: You might want to truncate the floats decimal places and check against that. Which is a more understandable way indeed!

I'm just saying that It was not my idea but Keiluns. I never needed a function like that.

But as I think about that I remember a very cool function Nathan gave me and I think giving the number of decimals to keep is even more obvious.:

global proc int nRound(float $input)
{
return trunc($input + (0.5 * sign($input)) );
}

global proc float cutFloat(float $float, int $decimals)
{
int $factor = pow(10,$decimals);
return ((float)nRound($float * $factor) / $factor);
}

global proc int isInFloatArray( float $value, float $array[] )
{
for ( $item in $array )
if ( $item == $value )
return true;

return false;
}

global proc float[] floatArrayRemoveDuplicates2( float $array[] , float $decimals )
{
float $resuts[], $f;
int $count = 0;

for( $item in $array )
{
$f = cutFloat($item, $decimals);
if ( !isInFloatArray($f, $resuts) )
$resuts[$count++] = $f;
}
return $resuts;
}

floatArrayRemoveDuplicates2({0.1, 0.10001, 0.2, 0.23}, 1);
// Result:0.1 0.2//
floatArrayRemoveDuplicates2({0.1, 0.10001, 0.2, 0.23}, 2);
// Result:0.1 0.2 0.23//

The sorting thing I'd leave to the user. Putting a sort() around his stuff wouldn't be too much to ask I guess :D

$aFloatArray = {0.1, 0.10001, 0.23, 0.2, 0.79, 0.7, 0.71};
floatArrayRemoveDuplicates2($aFloatArray, 1);
// Result: 0.1 0.23 0.2 0.79 0.7 //
floatArrayRemoveDuplicates2(sort($aFloatArray), 1);
// Result: 0.1 0.2 0.7 0.79 //
sort(floatArrayRemoveDuplicates2($aFloatArray, 1));
// Result: 0.1 0.2 0.23 0.7 0.79 //

isoparmB
01-19-2011, 04:41 PM
If you wanted to cheat though:

float $floater[] = {12.0, 15.2, 15.21, 22.0, 86.062, 178.0, 12.01, 22.01};
int $precision = 1;
string $feeder = "";
int $i = 0;
for( $i = 0; $i <=(`size $floater` - 1); $i++) {
$feeder += $floater[$i] + ",";
}
float $result[] = python("list(set( [ round(float(x), " + $precision + ") for x in [" + $feeder + "] ] ) )");
print $result;


Or you could write it simply as straight python.
Cheers.

ewerybody
01-19-2011, 05:02 PM
Or you could write it simply as straight python.hehe :D yea so true.

denisT
01-19-2011, 08:23 PM
If you wanted to cheat though:
...

possibly. but this method is probably 5 times slower.

Joviex
01-23-2011, 05:16 AM
possibly. but this method is probably 5 times slower.

Possibly? Have you done any python or mixed MEL and Python?

Python is far faster than MEL. We are talking just straight Python, not nessicarily PyMEL (though there is no need for PyMEL here per se).

I'd write this in python in about two lines using generators and / or lamdas.

Yes, tighter code is not always "faster" code, but in this case, it is, and in most scenarios where you shave cycles pushing values to a stack and popping values off, it is.

Cheers.

denisT
01-23-2011, 07:46 AM
Possibly? Have you done any python or mixed MEL and Python?

Python is far faster than MEL. We are talking just straight Python, not nessicarily PyMEL (though there is no need for PyMEL here per se).


possibly you did read the thread carefully.
I didn't talk about straight Python, I talked just about that very case...

global proc float[] anotherMethod (float $floater[], int $precision) {
string $feeder = "";
int $i = 0;
for ($i = 0; $i <=(`size $floater` - 1); $i++) {
$feeder += $floater[$i] + ",";
}
float $result[] = python("list(set( [ round(float(x), " + $precision + ") for x in [" + $feeder + "] ] ) )");
return $result;
}



here is result of my method (see above):

$t = `timerX`;
for ($k=0; $k<10000; $k++) uniqueRoundFloatArray ({0.1, 0.10001, 0.2, 0.23}, 0.1);
timerX -st $t;
// Result: 0.39 //


here is isoparmB's method (MEL + Python):

$t = `timerX`;
for ($k=0; $k<10000; $k++) anotherMethod ({0.1, 0.10001, 0.2, 0.23}, 1);
timerX -st $t;
// Result: 2.05 //


so... I was wrong. The second method is more than 5 times slower. Do I have to explain why, or you can guess yourself? If you are so interested you can check the memory usage as well. Hopefully it will not surprise you too much.

If you look at original question you will see that it was the MEL question. And I gave the MEL answer.
I don't do a lot of Python coding. But I'll never do the MEL-Python mixing the way as in the sample above.

Best,

Joviex
01-24-2011, 02:55 AM
possibly you did read the thread carefully.


Again, possibly.


def pythonSort(inputArray, precision):
ret = []
for x in inputArray:
x = round(x, precision)
if not x in ret:
ret.append(x)
return ret


import time
start = time.time()

inputArray = [0.1, 0.10001, 0.2, 0.23]

for k in range(0,10000):
pythonSort(inputArray, 1)

print time.time()-start

and I took it to a rediculous level. This could be further optimized.

The results:

0.06 seconds.

So, yes, slightly faster.

I know this was a "MEL" question, but it doesn't have to be. This proc/def could be written in a seperate file and CALLED from MEL.

Just to see the time difference there....


python("import sort");
$t = `timerX`;
for ($k=0; $k<10000; $k++) python("sort.pythonSort([0.1, 0.10001, 0.2, 0.23], 1)");
timerX -st $t;


Times out at 0.61 seconds. So, yeah, from MEL -> python it takes 10x longer to push the stack, and pop the stack. We know this.

I was not attacking your solution, or complaining that it should be only ever in python! I have plenty of legacy to maintain in both. I simply wanted to add, for completeness, that MEL and python dont have to be bastard children of each other.

Python IS faster. MEL only solutions do not have to be MEL only solutions. It depends on what you want.

As for memory usage, if someone wanted to write the python as a generator (i.e. via yield) I can guarentee that would reduce overhead since only one element would ever be on the stack.

Cheers

CGTalk Moderation
01-24-2011, 02:55 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.