missing data on a specific date

[HeatherGranger]

Looking at total selenium data for sites: E325354, E325355, E325356, E325791

'Get any available data between dates' says there is data from 2021-07-08 to 2021-09-01
Data is pulled into shinyrems for all sites except for E325791 which was taken on the 2021-09-01.

The 2021-09-01 date is pulled in with rems in RStudio, but isn't being pulled in with shinyrems.

[aylapear]

@HeatherGranger Are you able to provide the code you used to pull this using rems package? When I tried with rems I also had that data missing.

data <- rems::get_ems_data(
  which = "2yr",
  dont_update = TRUE, force = TRUE
)

data1 <- rems::filter_ems_data(
  x = data,
  emsid = c("E325354", "E325355", "E325356", "E325791"),
  parameter = NULL,
  from_date = "2020-07-08",
  to_date = "2021-09-01",
)

data1 %>% 
  group_by(EMS_ID) %>% 
  summarise(RowCount = n())

# output 
 EMS_ID  RowCount
  <chr>      <int>
1 E325354       72
2 E325355       72
3 E325356       72

It appears that rems is not pulling data including that date based on trying with a couple other examples. When I look at the code for rems::filter_ems_data() it says equals to or greater/less than for the date parameters so in theory it is written correctly but it doesn't seem to be working that way. I didn't test this extensively but it appears to be an issue within rems.

If I increase the date by one day then it does pull all four of those sites.

data2 <- rems::filter_ems_data(
  x = data,
  emsid = c("E325354", "E325355", "E325356", "E325791"),
  parameter = NULL,
  from_date = "2021-07-08",
  to_date = "2021-09-03",
)

data2 %>% 
  group_by(EMS_ID) %>% 
  summarise(RowCount = n())

# output 

  EMS_ID  RowCount
  <chr>      <int>
1 E325354       72
2 E325355       72
3 E325356       72
4 E325791       72

The app uses the rems::filter_ems_data() to filter the data in the first tab (1. Data). It seems to be more of an issue with the rems function than shinyrems. If you have an example using rems where this works that would be good for me to see. Otherwise I think the function in rems needs to be looked into.

[HeatherGranger]

fouryear <- get_ems_data(which = "4yr", ask = TRUE)

filtered_fouryear <- filter_ems_data(fouryear,
emsid = c("E325354", "E325355", "E325356", "E325791"))

I didn't use any date specifications @aylapear so that's probably why it worked for me. @KarHarker something to investigate in rems perhaps.

[KarHarker]

I will have a look, thanks for the heads up

[HeatherGranger]

@KarHarker do you remember if we reached a conclusion in this one?

[HeatherGranger]

@KarHarker I told @aylapear to investigate this one.