Questions regarding to the gates in exported xml from GatingSet2cytobank

[mikejiang]

On 01/11/2017 01:23 PM, Florian Klemm wrote:

Hi Mike,

As an afterthought to my previous email, I realized that the data I extract with getData from my gating hierarchy results in a flowframe with a range of 206.2144 for most of my fluorescence channels. How does that relate to the coordinates for the gates in the exported xml file. Here all the values are between around -5 and 5? Is there a way that back to the original range while keeping flowJo’s transformations? Using “getTransformations” and then transform didn’t work because the object retrieved by the former was not accepted as a transformlist by the latter.

Best, Florian.

From: Florian Klemm florian.klemm@unil.ch
Date: Wednesday 11 January 2017 at 18:05
To: Mike wjiang2@fhcrc.org
Subject: Re: parsing FlowJo workspace and saving gating set

Hi Mike,

Thanks a lot for fixing the error. Saving the gs as a xml works now for me. Unfortunately, I encountered enother error when setting the scale option to FALSE. Please see the console output below. I want to avoid rescaling the data because then I can’t use the gates on my flowEnv that I need for flowSOM, as their position in relation to the original data will have changes (or am I wrong when assuming this?).

Best regards, Florian.

GatingSet2cytobank(gs, "./input/csv_flowjo/GBM_46/gbm_46_gs.xml", cytobank.default.scale = FALSE)

Error in FUN(X[[i]], ...) :

can't find the transformation function in GatingSet to inverse the range for :PE-A

[mikejiang]

@FloCuenca, Let me briefly explain the workflow here:
The data stored in your GatingSet parsed from flowJo xml is in transformed scale using flowJo's biexponential function. So does the gates.
When you run GatingSet2cytobank to export to the xml (i.e. the tailored version of gatingML for cytobank), all the gates are restored to the raw scale first and then re-transformed to the scale that could be understood by cytobank. So the gate coordinates will definitely be at different scale from the one you saw in GatingSet, regardless of the setting of cytobank.default.scale.
And it is intended behavior because when this xml goes into cytobank along with raw FCS files, cytobank will transform the raw data using its own scaling methods and thus expect the gates are stored in the same cytobank-compatible scale.

In your case, you want to simply use flowUtils::read.gatingML to parse the gates into flowEnv. what you can do is to extract the transformations from flowEnv and use them to transform the raw FCS so that the data and gates are in the consistent scale.
I agree it is a little cumbersome, but GatingSet2cytobank was designed for data interchanging with cytobank. To have an option of disabling the gate rescaling goes against its original purpose, I will have to think a little more about this before making any change.

[mikejiang]

@FloCuenca, Now you can turn off gate rescaling by 'rescale.gate = F'. However, I don't think it is going to help you too much. Because the names of the gates and the actual populations (represented as boolean gates named as 'GateSet_xx' ) are all encrypted. The unencrypted names are attached as the custom_info in xml which will be picked up by the parser from Cytobank system. Yet flowUtils is unaware of this information and leaves you clueless on finding your population of interests.

So, again, the reason is that it was designed for cytobank. But on the other hand, the genuine gatingML does not do a good job of capturing the gating hierarchy, which I mentioned here #14 (comment)

As I look through the demo code of FlowSOM below ( I assume this is what you want to do ),

> gatingFile <- system.file("extdata","manualGating.xml", package="FlowSOM")
> read.gatingML(gatingFile, flowEnv)
> filterList <- list( "B cells" = flowEnv$ID52300206, "ab T cells" = flowEnv$ID785879196, "yd T cells" = flowEnv$ID188379411, "NK cells" = flowEnv$ID1229333490, "NKT cells" = flowEnv$ID275096433)
> results <- list()
> for(cellType in names(filterList))results[[cellType]] <- filter(ff_c,filterList[[cellType]])@subSet

all it does is to get logical index from each population and merge/convert them into a factor (I assume these populations do not overlap ?).

And you actually can logical index directly from GatingSet without going through gatingML and re-applying the gates on the raw data.

result[["B cells"]] <- getIndices(gs[[1]], "B cells")
result[["ab T cells"]] <- getIndices(gs[[1]], "ab T cells")