Hi, We have been using flowCore for many years reading files from BD FACSCalibur, CyAn ADP, Beckman CytoFlex, Miltenyi MACSQuant Analyzer using read.FCS(), no problem up to version 2.0. However, we updated R and the packages and now flowCore 2.14 is returning an error reading the MACSQuant files. This error is:
$SPILLOVER keyword value is of improper size for number of spillover channels!
The culprit is apparently an apparently malformed $SPILLOVER: $SPILLOVER/4,FL5-H,FL4-H,FL5-A,FL4-A (see linked file below); no problem with older files from that cytometer not containing $SPILLOVER or with files from the other cytometers.
Not quite sure what caused the cytometer software to include it in the FCS files. The funny thing is that we were working with flowCore 2.0, no problem. We updated R and packages in one of the computers and we started seeing this error. Replicated in my personal computer (Windows, OSX) which I updated recently but never tested with these files. We have another computer still using flowCore 2.0 and no problem reading the files.
Unfortunately, it seems that we cannot downgrade flowCore (maybe downgrading R, so BiocManager installs with a lower version and similarly for packages? not willing doing it). We have many older files with this problem (never noticed that), so just trying to solve it at the cytometer is not an option.
We tried with different parameters and even looking at the code, but no luck (it seems that it is in the C++ part, which is beyond our skills).
Therefore, I wonder if a subsequent version could incorporate a parameter for optionally converting this error into a warning (something like emptyValue = FALSE, which incidentally we need to read these files). Another option could be changing the header and writing back the file, since read.FCSheader() successes, but I believe we would need to read the full file, defeating the solution.
You can get the MACSQuant files (with and without the $SPILLOVER keyword) here: https://app.box.com/s/6bha57tzl0pnh71pq152457mlwsaidel
The additional warnings when trying to read the file are not problematic. I am pasting them just for providing the full information.
read.FCS("rbio2023-10-02.0052.fcs", emptyValue = FALSE)
$SPILLOVER keyword value is of improper size for number of spillover channels!
Error: $SPILLOVER size discrepancy could not be resolved!
Además: There were 12 warnings (use warnings() to see them)
Warning messages:
1: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'Time' channel exceed its $PnR value 30.4978 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
2: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FSC-A' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
3: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FSC-W' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
4: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'SSC-A' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
5: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'SSC-H' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
6: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'SSC-W' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
7: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL4-A' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
8: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL4-H' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
9: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL4-W' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
10: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL5-A' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
11: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL5-H' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
12: In readFCSdata(con, offsets, txt, transformation, which.lines, ... :
Some data values of 'FL5-W' channel exceed its $PnR value 1000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
sessionInfo( )
R version 4.3.2 (2023-10-31)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.6.1
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0
locale:
[1] es_ES.UTF-8/es_ES.UTF-8/es_ES.UTF-8/C/es_ES.UTF-8/es_ES.UTF-8
time zone: Europe/Madrid
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] flowCore_2.14.0
loaded via a namespace (and not attached):
[1] compiler_4.3.2 RProtoBufLib_2.14.0 cytolib_2.14.0 Biobase_2.62.0
[5] S4Vectors_0.40.2 BiocGenerics_0.48.1 matrixStats_1.2.0 stats4_4.3.2
I have found that adding truncate_max_range = FALSE removes the warnings (I should have tried before), and the error is maybe more comprenhensive:
Hi.
rbio2021 has no spillover, so no problem with current flowCore. As you identified it, the spillover matrix of rbio2023 has a header but no data.
With flowCore 2.0, read rbio2023, then write a copy, then read it with flowCore 2.14. This is the best option IMHO. BTW, you could have multiple versions of R in your computer.
Best.
Yes, we have thought of that. However, I do not see why not having the option of reading these files throwing a warning. Data is readable, is just the $SPILLOVER one which is malformed. Reading files with problems in the header might result in reading the data block and returning a number of warnings, optionally errors.
Feel free to raise an issue in the github repo of flowCore.
Thank you, moving this to the issues in Github.
Perfect. Link here.