GatingSet2cytobank unsupported transform - unitytransform?

[cjungR]

Hello,

I have some CyTOF files that I have gated using opencyto, and I'd like to export the GatingSet into cytobank with CytoML but I am encountering this error:

> outFile <- tempfile(fileext = ".xml")
> GatingSet2cytobank(gs, outFile)
Error in FUN(X[[i]], ...) : unsupported transform: unitytransform
In addition: Warning message:
In GatingSet2cytobank(gs, outFile) :
  With 'cytobank.default.scale' set to 'TRUE', data and gates will be re-transformed with cytobank's default scaling settings, which may affect how gates look like.

I did transform the .fcs files with a GatingML file exported from cytobank. Could this be the reason why? Is there a workaround for this issue? Thank you for your help.

[mikejiang]

How and why would you apply unitytransform (which does not do anything to the data as far as I know)? What is the output of getTransformation(gs[[1]])?
We currently only support asinht, logicle or flowJo biexponential , which are most commonly used transformations (used by flowJo or cytobank). If you only need linear scale for some channels, it is better to leave them untransformed.

[cjungR]

Hi Mike,

Here is how I transformed my data:

ncdf <- read.ncdfFlowSet("unstimulated.fcs")
gs <- GatingSet(ncdf)

gML <- read.gatingML.cytobank("CytExp_124156_Gates_v2.xml")

trans <- getTransformations(gML)
gs <- transform(gs, trans)

I transformed my data this way because my original goal was to see if I could replicate the gating by done by another person on cytobank through opencyto, and I initially had difficulty figuring out how I could transform fcs files properly. I attempted to use estimateLogicle() as shown in the vignettes, but I would get this error:

Error in logicle_transform(as.double(x), as.double(t), as.double(w), as.double(m), : Logicle Exception: DidNotConverge: scale() didn't converge

So my workaround was to use the GatingML file from cytobank and use cytoML to pull the transformations to apply to my fcs files. I was then able to successfully replicate the gating done on cytobank.

And here is the output for getTransformations(gs[[1]]):

$Sm147Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x194832a4>

$Sm152Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x18b743cc>

$Gd160Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x1d3f7bc0>

$Ho165Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x15a43794>

$Nd142Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x186e78f0>

$Nd143Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x19217ca4>

$Bi209Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x18d04d20>

$Ce140Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x1940caec>

$Dy162Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x1908c9b0>

$Er168Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x18decaa8>

$Er170Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x18d0d7e4>

$Eu151Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x193b6f50>

$Ir191Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x192a1490>

$Ir193Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x19203740>

$Y89Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x18c53ef8>

$Yb173Di
function (x) 
{
    length * ((asinh(x * sinh(m * log(10))/t) + a * log(10))/((m + 
        a) * log(10)))
}
<environment: 0x1912c6f4>

So it seems asinht was used to transform the fcs file? getTransformations(gML) also shows asinhtGml2 as the transformer. Now I realize I can manually transform my fcs files this way:

ncdf <- read.ncdfFlowSet("unstimulated.fcs")
gs <- GatingSet(ncdf)

trans <- asinhtGml2_trans()
chnls <- colnames(gs[[1]])
trans <- transformerList(chnls, trans)
gs <- transform(gs, trans)

which was successful, and after gating I can use GatingSet2cytobank to export as an .xml file for cytobank. However, I wasn't able to get the same gating results with my current gating template, but I think it should be ok with adjustments to my template.

But I wonder why GatingSet2cytobank was unable to produce an xml file with the way I originally did the transformations since asinht was used? If I manually transform with asinhtGml2_trans(), I can generate an .xml file, but with transformations pulled directly from a gatingML file, I cannot and get the "unsupported transform: unitytransform" error.

Thank you for your time and I appreciate all your help.

[mikejiang]

If what you posted was exactly what you did, and asinhtGml2 is the only transformer it gets from cytobank xml, and that is the only thing you applied to GatingSet, then there shouldn't be any unitytransform involved. A reproducible example would be necessary for me to trouble shoot further.

[cjungR]

Hi Mike,

Here is a .zip file containing the gatingML file used for transformations, gating template, and fcs file: https://drive.google.com/open?id=0B3ZDuiGAmNGQd01OaUowMVUyam8

Please note that I created the gating template file for an older version of opencyto, but it should work after turning off the validity test on the latest opencyto version. Thank you for your help!

[mikejiang]

The culprit is the coerce method that coerces the ellipsoid gate to polygon gate.
During that process, the original transform object associated with the gate parameters were lost. So you get unitytransform complains from CytoML simply because that is the default parameters attached to newly coerced polygonGate.

Anyway, I am glad you caught this. Pull the latest flowCore trunk, it should good to go. ( Let me how it turns out in Cytobank once you imported there)

[cjungR]

Hi Mike,

My apologies for the delay - but thank you so much for your help! An .xml file can now be generated to upload to Cytobank.

However, there are some populations that aren't being transferred over properly:

Here, it seems like only the +- and ++ populations were transferred correctly, and for some reason the -- and -+ populations were not (ie only the UR and LR gates were applied correctly). The gating above wasn't replicated correctly because the parent population specified for these two populations is the -- or LL population (Gd160Di-Ho165Di-). I can replicate the opencyto gating if I delete and regate the populations manually on Cytobank.

Here are the parent populations in question from my gating template:

9	*	CD14+/-CD16+/-	Sm147Di-	Gd160Di,Ho165Di	mindensity	gate_range=c(1,3)

Do you have any idea why this might be happening? And please let me know how I can further help in troubleshooting this issue. Thank you.

[mikejiang]

I don't there is any issue with openCyto or CytoML here
for line 9

9	*	CD14+/-CD16+/-	Sm147Di-	Gd160Di,Ho165Di	mindensity

I can see all 4 quadrants are present in both openCyto and cytobank (after imported)

With regards to

11 | mDCs | CD123+HLA+ | Gd160Di-Ho165Di- | Eu151Di,Yb173Di | mindensity

It is doing the exact right job you have instructed through the template, i.e. only get top right quadrant .
If you intend to get gate all 4 quadrants, then you need to tell it so by filling +/-+/- instead of ++(i.e.CD123+HLA+)

[cjungR]

Hi Mike,

I do agree that for line 9, all 4 quadrants are shown correctly, and that line 11 is correct and gets the top right quadrant. However, for line 11, the template specifies the parent population to be the lower left quadrant of the gating done in line 9 (Gd160Di-Ho165Di- population), which I believe is not correctly selected on Cytobank (it is selected correctly in opencyto, as shown in my images above).

For example, here is what happens when I change the active population on Cytobank for the line 9 gating:

This is what I was talking about in my previous post about only the +- and ++ populations here being correct on Cytobank. Here, it seems like there is an issue with the -- and -+ populations (ie -- population seems to have points from the UL and UR quadrants, and not the LL where the Gd160Di-Ho165Di- should be), causing line 11 to not show that 3.5% cluster that is seen on opencyto. Sorry for the confusion.

I hope this makes sense and that I am correctly understanding how the gating template should work. By selecting Gd160Di-Ho165Di- (ie CD14-CD16-) as the parent population in line 11, I am trying to gate what is in the upper right quadrant (CD123+HLA+) from the population gated by the lower left quadrant of line 9 (CD14-CD16-). Which is correctly showing up on opencyto but not on Cytobank.

[mikejiang]

I can't really understand what you are talking about unless you upload your updated template.

[cjungR]

This is all with the same gating template I posted earlier, along with the same .fcs file and transformations here: https://drive.google.com/open?id=0B3ZDuiGAmNGQd01OaUowMVUyam8

The plots I posted above were made by changing the "active population" on Cytobank to see if all the populations in the quadrants made by line 9 were gated correctly, and is not on the gating template.

[mikejiang]

Ok. I've fixed the bug related to #23 . Try the latest trunk branch and let me know if it works for you

[mikejiang]

BTW, to simply your gating tree, you can use toggle.helperGates(gt,gs) to hide those intermediate gates generated by autogating so that only the gates of interests will be imported to Cytobank

[cjungR]

The fix worked perfectly!

The populations in question now show up correctly, thanks so much for your help. And good to know that I can hide the intermediate gates, thank you for the tip.

Something else I've noticed is that the % population statistic on Cytobank plots do not show up for +- pop type mindensity gates. For example:

I can get the % statistic on the plot if I manually regate it. I wonder if this is a similar issue?

[mikejiang]

This is a separate issue regarding to the location of stats label. I will close this and open a new one for this.