Add an order &dedupe option

[krader1961]

Related #1453

[SolitudeSF]

why is that an option instead of separate command?

[krader1961]

why is that an option instead of separate command?

For performance. It's more efficient to do the dedupe when ordering the values than it is to pipe them into a separate command. I'm also working on a separate command. See issue #1453.

[krader1961]

@SolitudeSF, I just created P.R. #1569 that implements a dedupe command. So now we can efficiently sort lists with duplicate elimination and dedupe arbitrary inputs.

[xiaq]

Sorry, but no.

Optimizing order | dedupe is indeed desirable, eventually. But it can and should be done transparently - the compiler can recognizing the pattern order | dedupe in a pipeline and use a more efficient implementation under the hood.

[krader1961]

Optimizing order | dedupe is indeed desirable, eventually. But it can and should be done transparently - the compiler can recognizing the pattern order | dedupe in a pipeline and use a more efficient implementation under the hood.

Yes, but I suspect I'll be dead before that happens. 😄

It's not clear to me why an obviously trivial optimization that works today, and the immediate future, isn't acceptable. Even if the optimization you envision is implemented in the distant future I don't see any reason to not implement the much simpler optimization today since it is trivial.