Why do integrate_to_ppi VIR values significantly change between bioRad v0.5.2.9410 -> v0.6.1?

[BerendWijers]

Hi all,

I've been trying to determine what the cause is for VIR values in differentiating between bioRad versions. I've looked with @bart1 and we've seen difference in various functions but can not attribute which function(s) actually cause the VIR values to go up with a significant factor. I hope someone can point me to in the right direction.

I've calculated the difference (factor) per grid cell of the resulting datasets and it turns out that all of them are different between bioRad versions with a rather steady factor. For the particular files I've enclosed, the mean of that factor is 2.2661314.

input files

Workflow

# B. Wijers, 30-11-2022
# env var
in_dir <- Sys.getenv('HOME')
fpath_pvol  <- paste0(c(in_dir,'NLHRW_pvol_20220403T0000_6356.h5'),
                      collapse = '/')
fpath_vp <- paste0(c(in_dir,'NLHRW_vp_20220403T0000_6356_v0-5-0.h5'),
                   collapse = '/')
# load libraries
library('bioRad')
# main
my_pvol <- bioRad::read_pvolfile(fpath_pvol)
my_vp <- bioRad::read_vpfiles(fpath_vp)
my_rbc <- bioRad::integrate_to_ppi(my_pvol,my_vp)
# display summary
rbc_summary <- summary(my_rbc$data)
rbc_summary$data

bioRad v0.6.1

Data summary:
      VIR               VID               R             overlap          eta_sum          eta_sum_expected  
 Min.   :      0   Min.   :     0   Min.   :     0   Min.   :0.4148   Min.   :        0   Min.   :   2.258  
 1st Qu.:      0   1st Qu.:     0   1st Qu.:     0   1st Qu.:0.4802   1st Qu.:        0   1st Qu.:   9.075  
 Median :      0   Median :     0   Median :     0   Median :0.5308   Median :        0   Median :  15.140  
 Mean   :   1299   Mean   :   118   Mean   :    65   Mean   :0.5603   Mean   :    27152   Mean   :  37.793  
 3rd Qu.:      0   3rd Qu.:     0   3rd Qu.:     0   3rd Qu.:0.6219   3rd Qu.:        0   3rd Qu.:  35.471  
 Max.   :5946351   Max.   :540577   Max.   :297645   Max.   :0.8397   Max.   :232347018   Max.   :1058.974 

bioRad v0.5.2.9410

Data summary:
      VIR                VID                  R                overlap      eta_sum          eta_sum_expected  
 Min.   :       0   Min.   :      0.0   Min.   :     0.00   Min.   :0.4147  Min.   :        0   Min.   :   2.252
 1st Qu.:       0   1st Qu.:      0.0   1st Qu.:     0.00   1st Qu.:0.4801  1st Qu.:        0   1st Qu.:   9.043
 Median :       0   Median :      0.0   Median :     0.00   Median :0.5306  Median :        0   Median :  15.097
 Mean   :    2883   Mean   :    262.1   Mean   :    63.67   Mean   :0.5601  Mean   :    27199   Mean   :  37.756
 3rd Qu.:       0   3rd Qu.:      0.0   3rd Qu.:     0.00   3rd Qu.:0.6217  3rd Qu.:        0   3rd Qu.:  35.471 
 Max.   :13470034   Max.   :1224548.6   Max.   :297530.46   Max.   :0.8397  Max.   :232347018   Max.   :1058.974

[bart1]

@BerendWijers as we already figured out quite a bit off the difference (relating to approx fun interpolating over NA's, and the shift in the cell centers iirc). I don't remember if those only resulted in differences in R and overlap or also partially VID? it would be good to already report that here. That saves a lot of double work.

[BerendWijers]

Hi @bart1 good call - I'll add that information too.

I notice this is a rather specific question. I'll dig into this myself and report what I find.

Can this be assigned to me?

[BerendWijers]

Hi all,

@bart1 and me dug deeper and we've found the cause. I'm closing this issue and I've made a report about integrate_profile

Small numerical differences arose from the following functions

• scan_convert.R::scan_to_spatial
  • uses cell centers now

Major differences:

• integrate_profile
  Closing issue as this is not directly related to the function integrate_to_ppi, see integrate_profile does not take lowest bin into account #543