Is submission ID supposed to be unique?

[zross]

@abelvaldivia I see that you're removing duplicates based on submission ID and date. Does that mean submissionid can be repeated?

> dim(hhs)
[1] 17286   243
> length(unique(hhs$submissionid))
[1] 17276

[abelvaldivia]

@zross submission id supposed to be unique. If dim() and length(unique)() dont coincide in the number of rows is because there are records repeated. In this case the most recent record with the same submission ID should be kept and the others removed. This code that I wrote within the process-raw-data.R file supposed to do that before the data is loaded into the app

hhs <- ALL_hhs %>%
        select(-c(updatedat, endformtimestamp, startformtimestamp)) %>%
           unique() %>%
              left_join (ALL_hhs[,c("submissionid", 
                                    "updatedat", 
                                    "startformtimestamp", 
                                    "endformtimestamp")] %>%
                                   group_by (submissionid) %>%
                                      summarise (updatedat = max(as.Date(updatedat))),
                         by = "submissionid")

[zross]

That's the code I ran, you should double check. It may not be doing what you think it's doing. Do you want to look?

[abelvaldivia]

I will check

[abelvaldivia]

@zross I just run it and I got the same number of submission IDs. Maybe run the entire code in that file ?