De-duplicate an Array by value¶
In javascript, a common way of de-duplicating an array is by using something like the following:
const arr = [1, 1, 2, 3];
const dedupeArr = Array.from(new Set(arr));
This runs it through a Set, which ensures that there's only one element of each type, and then converts it back into an array. One footgun with this approach is that Set appears to test equality by reference. While this works for primitive values, if you're trying to de-duplicate an array of complex data this isn't going to work so well.
Instead, it's probably best if you create an equality function to test your data for equality pairwise - or define an Eq instance for your data if using something like fp-ts1.
Something like the following will do in the general case (given in typescript)
function deduplicate<T>(
eq: (left: T) => (right: T) => boolean,
arr: Array<T>
): Array<T> {
return arr.reduce((memo: Array<T>, x: T) => {
const contains = memo.some(eq(x));
return contains ? memo : memo.concat([x]);
}, []);
}