Create Drop Duplicates Method

[exalate-issue-sync]

create a method similar to pandas drop_duplicates() for h2oframe

[exalate-issue-sync]

Michal Malohlava commented: There is workaround (CC: [~accountid:557058:89297402-cb5a-4710-9511-20f42b25451a])

[exalate-issue-sync]

Navdeep commented: [~accountid:557058:389d9607-5bd8-4611-8c6a-755fe9295223] Oh, im intrigued.

[exalate-issue-sync]

Michal Malohlava commented: Oki, so drop_duplicates() removes rows - at the end it is providing the same functionality as sort | uniq in shell.

[exalate-issue-sync]

Lauren DiPerna commented: 1) one workaround is to use sort and then it could be easy to check duplicated rows (there is no support for that right now)

2. another workaround is to create a new column with row hash and call drop duplicates using that column (however, it still will need sort above)

[exalate-issue-sync]

Neema Mashayekhi commented: Like in {{pandas.drop_duplicates()}} or data.table's {{unique()}}, customer would like the ability to:

keep either the first occurrence or the last one, {{fromLast = FALSE}} or {{fromLast = TRUE}} .

handle duplicate for multiple keys/columns. Using {{h2o.unique()}} only handles one key.

[https://rdrr.io/rforge/data.table/man/duplicated.html|https://rdrr.io/rforge/data.table/man/duplicated.html]

[exalate-issue-sync]

Neema Mashayekhi commented: Consider concatenating multiple columns into a single column: [1], ['A'] → ['1_A']

[h2o-ops]

JIRA Issue Migration Info

Jira Issue: PUBDEV-3292
Assignee: Pavel Pscheidl
Reporter: Lauren DiPerna
State: Resolved
Fix Version: 3.30.0.4
Attachments: N/A
Development PRs: Available

Linked PRs from JIRA

#4581