Search
Question: oligo: crash/segfault when fitting probelevel model
0
gravatar for Guido Hooiveld
12 weeks ago by
Guido Hooiveld2.2k
Wageningen University, Wageningen, the Netherlands
Guido Hooiveld2.2k wrote:

A fresh Linux install of R crashes/segfaults when fitting a probelevel model using oligo on a dataset. Is this a bug? If not, any idea how to fix this? I don't know whether it is important, but please note that R is configured to use multi threaded OpenBLAS (rather than the default single-threaded BLAS).

 

library(oligo)
library(pd.huex.1.0.st.v2)
library(oligoData)
data(affyExonFS)

res <- fitProbeLevelModel(affyExonFS)

Background correcting... OK
Normalizing... OK
Summarizing...
 *** caught segfault ***
address 0x7f93eacd3028, cause 'memory not mapped'

Traceback:
 1: .Call("R_sub_rcModelSummarize_plm", y, rowIndexList, PsiCode,     PsiK, input.scale, PACKAGE = "preprocessCore")
 2: theFun(y = transfo(mat), pnVec)
 3: runSummarize(tmpMat, probeInfo$man_fsetid, method = method, verbose = verbose)
 4: fitProbeLevelModel(affyExonFS)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

<< I selected '2'>>

> sessionInfo()
R version 3.4.1 Patched (2017-08-27 r73146)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux

Matrix products: default
BLAS/LAPACK: /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base     

other attached packages:
 [1] oligoData_1.8.0          pd.huex.1.0.st.v2_3.14.1 DBI_0.7                 
 [4] RSQLite_2.0              oligo_1.40.2             Biostrings_2.44.2       
 [7] XVector_0.16.0           IRanges_2.10.2           S4Vectors_0.14.3        
[10] Biobase_2.36.2           oligoClasses_1.38.0      BiocGenerics_0.22.0     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12               compiler_3.4.1            
 [3] BiocInstaller_1.26.0       GenomeInfoDb_1.12.2       
 [5] bitops_1.0-6               iterators_1.0.8           
 [7] tools_3.4.1                zlibbioc_1.22.0           
 [9] digest_0.6.12              bit_1.1-12                
[11] memoise_1.1.0              tibble_1.3.4              
[13] preprocessCore_1.38.1      lattice_0.20-35           
[15] ff_2.2-13                  rlang_0.1.2               
[17] Matrix_1.2-11              foreach_1.4.3             
[19] DelayedArray_0.2.7         GenomeInfoDbData_0.99.0   
[21] affxparser_1.48.0          bit64_0.9-7               
[23] grid_3.4.1                 blob_1.1.0                
[25] splines_3.4.1              codetools_0.2-15          
[27] matrixStats_0.52.2         GenomicRanges_1.28.4      
[29] SummarizedExperiment_1.6.3 RCurl_1.95-4.8            
[31] affyio_1.46.0             
> 
ADD COMMENTlink modified 12 weeks ago by James W. MacDonald45k • written 12 weeks ago by Guido Hooiveld2.2k
0
gravatar for James W. MacDonald
12 weeks ago by
United States
James W. MacDonald45k wrote:

Do you have a corrupted oligoData package?

> data(affyExonFS)
res <- fitProbeLevelModel(affyExonFS)
>
Loading required package: pd.huex.1.0.st.v2
Loading required package: RSQLite
Loading required package: DBI
Background correcting... OK
Normalizing... OK
Summarizing... OK
Extracting...
  Estimates... OK
  StdErrors... OK
  Weights..... OK
  Residuals... OK
  Scale....... OK
> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)

Matrix products: default
BLAS: /data/oldR/R-3.4.0/lib64/R/lib/libRblas.so
LAPACK: /data/oldR/R-3.4.0/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base     

other attached packages:
 [1] pd.huex.1.0.st.v2_3.14.1 DBI_0.7                  RSQLite_2.0             
 [4] oligoData_1.8.0          oligo_1.41.0             Biostrings_2.44.2       
 [7] XVector_0.16.0           IRanges_2.10.2           S4Vectors_0.14.3        
[10] Biobase_2.36.2           oligoClasses_1.38.0      BiocGenerics_0.22.0     

 

ADD COMMENTlink written 12 weeks ago by James W. MacDonald45k

Thanks James for your suggestion! But... no, reinstalling oligoData unfortunately did not solve it.

RMA normalization just works fine?!?. See below.

As said before, could it be that preprocessCore somehow is not working with a multi-threaded BLAS? However, I am not knowledgeable enough to show this is the case....

 

> data(affyExonFS)
> res <- rma(affyExonFS)
Background correcting
Normalizing
Calculating Expression
> res
ExpressionSet (storageMode: lockedEnvironment)
assayData: 22011 features, 20 samples
  element names: exprs
protocolData: none
phenoData
  rowNames: 10_5N.CEL 11_6T.CEL ... 9_5T.CEL (20 total)
  varLabels: exprs
  varMetadata: labelDescription channel
featureData: none
experimentData: use 'experimentData(object)'
Annotation: pd.huex.1.0.st.v2
>
ADD REPLYlink written 12 weeks ago by Guido Hooiveld2.2k

Ah, I missed that part. There is C code intended to work with parallel threads, but that's all beyond my ken. It's all Ben Bolstad's code, and he is long since gone, so I am afraid you are probably on your own.

The easiest fix would be to compile a version of R that uses the stock BLAS libraries.

ADD REPLYlink written 12 weeks ago by James W. MacDonald45k
1

It is debatable as to how far gone over the edge I am :) But it is true that my activity level has been minimal for some years now.

In any case preprocessCore does have and has long had it's own pthreads based multithreading implementation. You can control the number of threads it is using with a shell set R_THREADS variable  (eg export R_THREADS=4) before starting R. I suspect that there is some sort of interaction between this and multi-threaded BLAS that might be solvable by putting a mutex or two in the right place. But that is just a wild guess.

 

ADD REPLYlink written 12 weeks ago by Ben Bolstad1.2k

Also it might be possible to turn threading off in OpenBLAS rather than just recompiling R. See

https://github.com/xianyi/OpenBLAS/wiki/faq#multi-threaded

 

 

ADD REPLYlink written 12 weeks ago by Ben Bolstad1.2k

Thanks Ben for replying so promptly! As per your suggestion (and for completeness I also post this here): I have opened an issue on GitHub: https://github.com/bmbolstad/preprocessCore/issues/1 to which I also added some more information on my attempts to get it working.

ADD REPLYlink written 11 weeks ago by Guido Hooiveld2.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 238 users visited in the last hour