EBimage fftw issue
1
0
Entering edit mode
@mathieu-gendarme-15885
Last seen 2.5 years ago

Dear all,

I am trying to run a bash file calling an R script on a cluster running CentOS 7. My R sessionInfo looks like this:

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRblas.so
LAPACK: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.3.0   stringr_1.3.1   dplyr_0.7.8     purrr_0.3.0    
 [5] readr_1.3.1     tidyr_0.8.2     tibble_2.0.1    ggplot2_3.1.0  
 [9] tidyverse_1.2.1 EBImage_4.20.1 

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.5    locfit_1.5-9.1      haven_2.0.0        
 [4] lattice_0.20-35     colorspace_1.4-0    generics_0.0.2     
 [7] htmltools_0.3.6     rlang_0.3.1         pillar_1.3.1       
[10] withr_2.1.2         glue_1.3.0          BiocGenerics_0.24.0
[13] modelr_0.1.2        readxl_1.2.0        bindrcpp_0.2.2     
[16] jpeg_0.1-8          bindr_0.1.1         plyr_1.8.4         
[19] munsell_0.5.0       gtable_0.2.0        cellranger_1.1.0   
[22] rvest_0.3.2         htmlwidgets_1.3     parallel_3.4.1     
[25] broom_0.5.1         Rcpp_1.0.0          scales_1.0.0       
[28] backports_1.1.3     jsonlite_1.6        abind_1.4-5        
[31] hms_0.4.2           png_0.1-7           digest_0.6.18      
[34] stringi_1.2.4       tiff_0.1-5          grid_3.4.1         
[37] cli_1.0.1           bitops_1.0-6        tools_3.4.1        
[40] magrittr_1.5        RCurl_1.95-4.10     lazyeval_0.2.1     
[43] crayon_1.3.4        pkgconfig_2.0.2     xml2_1.2.0         
[46] lubridate_1.7.4     assertthat_0.2.0    httr_1.4.0         
[49] rstudioapi_0.9.0    R6_2.3.0            fftwtools_0.9-8    
[52] nlme_3.1-131        compiler_3.4.1

My problem is that if I try to run my script on the main node (this has no purpose, it was just for testing after getting into troubles) everything runs fine. However when I am trying to run the same script on a computation node than I am facing this issue:

Error: package or namespace load failed for ‘fftwtools’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/gendarme/apps/R/fftwtools/libs/fftwtools.so':
libfftw3.so.3: cannot open shared object file: No such file or directory
Execution halted

This is how my bash file looks like.

module load R
module load slurm
#!/bin/sh
#SBATCH --nodes=1
#SBATCH --cpus-per-task=20
#SBATCH --job-name=sample1
#BATCH --mem=64000
export LD_LIBRARY_PATH=/cm/shared/apps/fftw:$LD_LIBRARY_PATH
export R_LIBS_USER=$HOME/apps/R:$R_LIBS_USER
Rscript ~/data/IfTestCluster/IFscript_CustomComputeFeatures_Cluster.R

I specified on purpose with LD_LIBRARY where the fftw is located on the main ode. However it doesn't seem to be enough.

I also checked the fftw installation with

yum install fftw3

and it returns that the latest version is already installed.

Has anyone an idea why this happening and how fix it?

Thanks a lot for you help in advance.

Cheers,

Mathieu

EBImage CentOS • 302 views
ADD COMMENT
0
Entering edit mode
@wolfgang-huber-3550
Last seen 4 days ago
EMBL European Molecular Biology Laborat…

I think your problem can be reduced to properly installing the fftwtools package for R from CRAN (https://cran.r-project.org/web/packages/fftwtools/index.html) on your cluster. EBImage imports this package, but your problem is at the level of fftwtools, not of EBImage.

Installation of the system library fftw3 is probably necessary, but not sufficient. You may have to involve a unix system administrator to make sure that on your cluster, all nodes all coherently set up, and able to run fftwtools.

Suggestion: do you maybe run .bashrc or similar scripts when you interactively login to what you call "main node", but these are not run when one of your batch jobs starts on a compute node? This could lead to all sorts of weird difference between runtime environments.

ADD COMMENT
1
Entering edit mode

Thanks a lot Wolfgang. Indeed I knew that it is not an EBImage issue but more how the fftw library was retrieved. I installed on my local environment fftw3 again using anaconda and specified where the library was supposed to be with LDLIBRARYPATH and it worked. Thanks again. Cheers

ADD REPLY

Login before adding your answer.

Traffic: 571 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6