Unknow reason error

[joe-jhou2]

Encountering error when loading fcs files,
fcs_raw = read.flowSet(as.character(meta$FileName), path = "TetramerExport/FCS", transformation = FALSE, truncate_max_range = FALSE)
Error in checkOffset(offsets, txt, ...) :
The HEADER and the TEXT segment define different starting point (4329:4328) to read the data.

The FCS file names are stored in the meta dataframe, which originally sourced from list.files(FCS file path).
The meta$FileName are matched with actual FCS file names, by checking
table(meta$FileName %in% ll) TRUE 1276

[mikejiang]

This is an indication that FCS files are either corrupted or not conforming to the FCS standards.
You can optionally ignore this by setting ignore.text.offset = TRUE so that it will force the parser to proceed.
See details in ?flowCore::read.FCS and search for ignore.text.offset argument

[joe-jhou2]

I use dumb approach to sort out which files cause this problem. Load into flowjo, it seems okay, nothing wired by eyeball check.
It's pretty strange that problematic files are exported as sub-populations from one file, all the other subpopulation files are fine.

[mikejiang]

If these offending files are exported from FlowJo, I suspect it is flowjo bug, which either didn't calculate and set the $BEGINDATA keyword properly, or the didn't do the FCS header right.
Anyway, I hope it is just the keyword error, otherwise you are at risk of getting not complete/correct/accurate data.

[joe-jhou2]

By setting ignore.text.offset, I gotta this error for those files,
000795_export_032918 ITN11560 tet V106 post_DR3_EBVJY134.fcs doesn't have the identical colnames as the other samples!
000796_export_032918 ITN11560 tet V106 post_DR3_FluBHAp44.fcs doesn't have the identical colnames as the other samples!
000797_export_032918 ITN11560 tet V106 post_DR3_Phlp1p22.fcs doesn't have the identical colnames as the other samples!
000798_export_032918 ITN11560 tet V106 post_DR3_Phlp5bp3.fcs doesn't have the identical colnames as the other samples!
000799_export_032918 ITN11560 tet V106 post_DR3_RVVP2p21.fcs doesn't have the identical colnames as the other samples!
000800_export_032918 ITN11560 tet V106 post_DR3_TTHp7.fcs doesn't have the identical colnames as the other samples!
000949_export_032918 ITN11642 tet V206 post DR3_EBVJY134.fcs doesn't have the identical colnames as the other samples!
000950_export_032918 ITN11642 tet V206 post DR3_FluBHAp44.fcs doesn't have the identical colnames as the other samples!
000951_export_032918 ITN11642 tet V206 post DR3_Phlp1p22.fcs doesn't have the identical colnames as the other samples!
000952_export_032918 ITN11642 tet V206 post DR3_Phlp5bp3.fcs doesn't have the identical colnames as the other samples!
000953_export_032918 ITN11642 tet V206 post DR3_RVVP2p21.fcs doesn't have the identical colnames as the other samples!
000954_export_032918 ITN11642 tet V206 post DR3_TTHp7.fcs doesn't have the identical colnames as the other samples!
Error in validObject(.Object) :
invalid class “flowSet” object: Some items identified in the data environment either have the wrong dimension or type.
In addition: There were 28 warnings (use warnings() to see them)

I go back to check those files, the file channels are identical as the others.
BUT, I look their filename,

fcs1[[1]]@description$FILENAME
[1] "TetramerExport/FCS/export_032918 ITN11560 tet V106 post_DR3_EBVJY134.fcs"
fcs1[[1]]@description$$FIL
[1] "export_11560 V106 post concat_1_DR3_EBVJY134.fcs"

They're not the same. Do you think that's the reason causing this error?

[mikejiang]

The error message definitely suggests there is channel discrepancy among these files,either some typo or spaces that are not easily noticeable by eyes or the order is not consistent.
Would you like to give cytoqc a try, which is the tool designed for this very purpose.

[joe-jhou2]

Get this:
wnloading GitHub repo RGLab/cytoqc@master
Skipping 1 packages not available: colortable
Installing 5 packages: kableExtra, colortable, selectr, rvest, webshot
Error: Failed to install 'cytoqc' from GitHub:
(converted from warning) package ‘colortable’ is not available (for R version 3.6.2)

But cannot find a package named colortable

[mikejiang]

Oh, yeah, forgot to add a note on this

remotes::install_github("thebioengineer/colortable")

[mikejiang]

BTW, you need R 4.0 in order to pull all the latest dependencies from bioconductor development branch(which will soon become the release 3.11)

[joe-jhou2]

I just notice that.

…

On Thu, Apr 2, 2020 at 4:36 PM Mike Jiang ***@***.***> wrote: BTW, you need R 4.0 in order to pull all the latest dependencies from bioconductor development branch(which will soon become the release 3.11) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#180 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFHYAYXKCMEDIRTEDVSHJLRKUOPXANCNFSM4L24XYMQ> .

[mikejiang]

It's a little extra upgrading work, but you will need to do that anyway next month when bioc 3.10 becomes obsolete.

[joe-jhou2]

Due to some reasons, I cannot upgrade R now. Do you have any tip about this error:
fcs_raw2 = read.flowSet(as.character(meta$FileName)[c(949:954)], path = "TetramerExport/FCS", ignore.text.offset = TRUE, transformation = FALSE, truncate_max_range = FALSE, name.keyword= "FILENAME", phenoData=list(name="FILENAME",Filename="$FIL"))

Error in FUN(X[[i]], ...) :
'TetramerExport/FCS/export_032918 ITN11642 tet V206 post DR3_EBVJY134.fcs' is not a valid file

I trace down a few fcs files seem problematic, even using read.FCS, the error still occurs.

fcs_indi1 = read.FCS("TetramerExport/FCS/export_032918 ITN11642 tet V206 post DR3_EBVJY134.fcs")

Error in read.FCS("TetramerExport/FCS/export_032918 ITN11642 tet V206 post DR3_EBVJY134.fcs") :
'TetramerExport/FCS/export_032918 ITN11642 tet V206 post DR3_EBVJY134.fcs' is not a valid file

However, loading to Flowjo, I cannot see any problem. So bizarre.

[SamGG]

Send this file to http://bioinformin.cesnet.cz/flowIO/
You'll get a detailed report.

[joe-jhou2]

Thanks. I load two files, one is readable by read.flowSet, the other is not.
However, both of them get the same error info. More confused.
1 | Header format check | Failed (detailes below)
2 | Byte offsets check | Failed (detailes below)
3 | Allowed characters check | Passed
4 | Duplicity check | Failed (detailes below)
5 | Required keywords check | Passed
6 | Standard FCS keywords check | Passed
7 | Check of non-empty values | Passed
8 | Check of numeric format | Failed (detailes below)
9 | Check individual keywords compliance | Failed (detailes below)

[SamGG]

"not a valid file" error seems to refer to the fact the file is not found. Are you definitively sure that the file name is correct? Replace reads.FCS by dir and tell us.

flowCore/R/IO.R

Lines 267 to 268 in 1124d1f

	if(!file.exists(filename))
	stop(paste("'", filename, "' is not a valid file", sep=""))

[joe-jhou2]

There are over 1000+ fcs files (each 10kb ish, a few events (0-100) included, exported from parental fcs files). Only these 5 files have the "not a valid file" issue. I've tried to re-export from original fcs files, the problem reoccur.

…

On Fri, Apr 3, 2020 at 2:03 PM Samuel Granjeaud ***@***.***> wrote: "not a valid file" error seems to refer to the fact the file is not found. Are you definitively sure that the file name is correct? Replace reads.FCS by dir and tell us. https://github.com/RGLab/flowCore/blob/1124d1fe850a4678c5afc5c9d66623c9bbf70d41/R/IO.R#L267-L268 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#180 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFHYA4PLOS465LCR7ZNDADRKZFL3ANCNFSM4L24XYMQ> .