Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dedup functionality is wrong for numeric even with sorted #1665

Closed
13minutes-yt opened this issue Mar 12, 2024 · 2 comments · Fixed by #1666
Closed

dedup functionality is wrong for numeric even with sorted #1665

13minutes-yt opened this issue Mar 12, 2024 · 2 comments · Fixed by #1666
Labels
bug Something isn't working

Comments

@13minutes-yt
Copy link

A couple of problems I encountered on using dedup

echo -e "data\n3\n1\n3\n10\n5"|qsv dedup

data
1
10
3
5
1

it removed the duplicate yet it did not sort them numerically, and the 1 in the end is the duplicate count which is annoying (can be removed by -Q)

In the documentation, it is said to sort first

echo -e "data\n3\n1\n3\n10\n5"|qsv sort -N|qsv dedup --sorted

data
1
3
Aborting! Input not sorted! ByteRecord(["5"]) is greater than ByteRecord(["10"])

and of course

$ echo -e "data\n3\n1\n3\n10\n5"|qsv sort -N|qsv dedup
data
1
10
3
5
1
@jqnatividad jqnatividad added the bug Something isn't working label Mar 12, 2024
@jqnatividad
Copy link
Owner

jqnatividad commented Mar 12, 2024

Good find @13minutes-yt . This is indeed a bug with dedup, will have to give it a --numeric option as well.

As for the dupe count being returned to stderr by dedup, I needed that in a program (DP+) that uses qsv. The --quiet option is there to suppress that.

@jqnatividad
Copy link
Owner

#1666 fixes this. And you can now do:

echo -e "data\n3\n1\n3\n10\n5"| qsv dedup -N -Q 
data
1
3
5
10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants