investigate the logicle/biexp transformation consistency between flowJo and flowCore

[mikejiang]

When interchanging gating analysis between openCyto and other vendor software (e.g. cytobank, flowJo), CytoML needs to deal with transformation conversions between flowCore::logicle and flowJo biexp back and forth. Based on the current coercion formula

flowJo_to_logicle <- function (maxValue = 262144, widthBasis = -100, pos = 4.5, neg = 0) 
{ 
  myTrans <- logicleGml2_trans(M = pos
                               , T = maxValue
                               , A = neg
                               , W = log10(-widthBasis) / 2)
  myTrans$transform
}

The conversion has been indirectly validated by the gating results (flowJo-->openCyto-->cytobank) .
But it would actually helpful to see if we can compare the transformed data directly.
So here is the test

fj.biexp <- flowJoTrans(maxValue = 262144, widthBasis = -100, pos = 4.5, neg = 0, channelRange = 4096)
fc.biexp <- flowJo_to_logicle(maxValue = 262144, widthBasis = -100, pos = 4.5, neg = 0)  
plot(x = fj.biexp(1:1e5), y = fc.biexp(1:1e5))

Looks pretty linear which indicates the extra channelRange from flowJo is simply a linear-scaling factor. to verify

df <- data.frame(x = fj.biexp(1:1e5), y = fc.biexp(1:1e5))
res <- glm(x~y, data = df)
res <- summary(res)
res$coefficients
   Estimate   Std. Error     t value Pr(>|t|)
(Intercept)   -0.8156886 0.0007357149   -1108.702        0
y           4096.0871602 0.0009049816 4526155.072        0

all.equal(fj.biexp(1:1e5), fc.biexp(1:1e5) * 4096)
[1] "Mean relative difference: 0.0002257089"

However, flowJo seems to be more robust in terms of self-correcting the invalid input whereas flowCore is sensitive and easy to error out

> fj.biexp <- flowJoTrans(maxValue = 262144, widthBasis = -10, pos = 4.8, neg = 100, channelRange = 1024)
> fc.biexp <- flowJo_to_logicle(maxValue = 262144, widthBasis = -10, pos = 4.8, neg = 100)
> plot(x = fj.biexp(1:1e5), y = fc.biexp(1:1e5))
Error in flowCore:::logicle_transform(as.double(x), as.double(T), as.double(W),  : 
  Logicle Exception: IllegalParameter: A is too large

[mikejiang]

Also it is worth to mention that flowCore's logicle is more than 10x slower than flowJo. It means flowJo implementation (i.e. spline interpolation based on the precalculated calibration table ) is more efficient.

> microbenchmark(x = fj.biexp(1:1e6), y = fc.biexp(1:1e6), times = 10)
Unit: milliseconds
 expr        min         lq      mean     median         uq      max neval
    x   5.986118   6.198265  26.88554   6.751488   9.792922 198.7599    10
    y 329.089815 330.530181 359.04822 338.694623 376.122943 474.3350    10

@gfinak , So we may consider to switch the flowJo logicle if the transformation ever becomes a bottleneck.