Proposal: array function to return a unique list of values

[shrickus]

I'm looking for something like a $distinct(array) function, that squashes an array into a unique list of values. Alternatively, I would think it could also be called $unique(...)

I've managed to write a couple expressions that build an object with keys, then gets the list of key names, but that seems too heavy handed to be very useful.

Ideally, this function would also take a second argument, a function that returns true/false (like $sift or $filter), to determine whether each element is unique. Arguments to this optional arg would probably follow the pattern of the $reduce(...) function, so the test logic can see what element is being checked, what its index is in the original array, and what the output array contains. But this is all just off the top of my head, so we have a starting point for any further discussions.

[michaelfranz]

Is anyone progressing this proposal?

[andrew-coleman]

I've been thinking for a while about how to best implement this. For starters, I need to upgrade the equality operator to work for non-atomic types. Currently [1,2] = [1,2] returns false which is clearly wrong. This needs fixing anyway. The bigger challenge is eliminating duplicate values in an efficient way, without comparing every value with every other (n-squared efficiency - terrible!). Hash sets normally come to the rescue here, but I would need to come up with a hash function that copes with all data types including hierarchies of maps and arrays.

[xogeny]

This seems pretty ambitious to me without external dependencies. There are whole libraries out there to handle things like equality testing. Getting into implementing uniqueness as well and you are opening to door to reimplementing something like lodash. I'm not saying it is necessarily a bad idea. But I would be cautious.

Also note that we've discussed in the past the idea of having a "core" language and "extensions". If structured that way, the core could be dependency free. But an extension might bring something like lodash in and be able to provide a lot of additional functionality with very little implementation cost.

[abrodeur]

Here is my solutions to the problem using current functions. One thing to note is since I do sort you do lose order.

(
    $unique := function($v, $i, $a){$i != 0 ? (( $v = $a[$i-1] ) ? "": $v) : $v };
    $notEmpty := function($v){$v  !=""};
	
    {
	"productIDs": **.ProductID,
        "uniqProductIDs" :$filter($map($sort( **.ProductID),$unique),$notEmpty)
        
    }
)

[fdavies93]

This probably only works for first-order objects as it relies on the in operator, but it's succinct and clear:

(
	$dupReduce := function($acc, $el) {
    	$el in $acc ? $acc : $append($acc,$el)
    };

	$removeDups := function ($a) {
    	$reduce($a,$dupReduce)
    };

	$removeDups(input)

)

The only part you'd need to change for more complex comparisons would be the test at the start of the $dupReduce higher-order function.

Alternately you wait for @andrew-coleman to extend in to do more complex comparisons.

[Zizaco]

$distinct(array) +1

that would be good

[darylwdavis]

I found @shrickus suggestion useful if the array is a selection of strings, for example a query:

devices.*.itemSelected

Then you can use the strings temporarily as object keys to eliminate duplicates, then return the keys:

$keys(devices.*{itemSelected:""})

[darylwdavis]

If the result contains only one object, $keys returns a string so you need to include all in brackets to always return an array.

[$keys(devices.*{itemSelected:""})]