universalmotif, shuffle_sequences quits unexpectedly when running on HPC
1
0
Entering edit mode
@alexandreblais-23753
Last seen 4 months ago
France

Hello I am using universalmotif and enrich_motifs. Things run smoothly on my PC but a larger dataset has forced me to move to a computer cluster with more available memory. I am having problems running shuffle_sequences on that HPC, it will crash even with a single, relatively short sequence.

Some very simple code (from the vignette) and the error message that appears instantly:

data(ArabidopsisPromoters)
euler <- shuffle_sequences(ArabidopsisPromoters, k = 2, method = "euler")
terminate called without an active exception
Aborted (core dumped)

I am not getting any useful message that would guide me in troubleshooting this... I should specify I am running this on a single cpu with 50GB of RAM assigned. This runs absolutely fine on my PC installation (with R 4.1.1, Bioconductor 3.14, but universalmotif_1.12.2). I rolled back the version on the HPC to 1.12.2 (as on my PC) but it still quits abruptly. Any help would be greatly appreciated. Thanks. Alex


> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/imkl/2020.1.217/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so

locale:
 [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_CA.UTF-8        LC_COLLATE=en_CA.UTF-8
 [5] LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8
 [7] LC_PAPER=en_CA.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
 [1] universalmotif_1.12.3             BSgenome.Mmusculus.UCSC.mm9_1.4.0
 [3] BSgenome_1.62.0                   rtracklayer_1.54.0
 [5] Biostrings_2.62.0                 XVector_0.34.0
 [7] GenomicRanges_1.46.1              GenomeInfoDb_1.30.0
 [9] IRanges_2.28.0                    S4Vectors_0.32.3
[11] BiocGenerics_0.40.0

loaded via a namespace (and not attached):
 [1] SummarizedExperiment_1.24.0 tidyselect_1.1.1
 [3] purrr_0.3.4                 lattice_0.20-45
 [5] colorspace_2.0-2            vctrs_0.3.8
 [7] generics_0.1.1              yaml_2.2.2
 [9] utf8_1.2.2                  XML_3.99-0.8
[11] rlang_1.0.0                 pillar_1.6.5
[13] glue_1.6.1                  DBI_1.1.2
[15] BiocParallel_1.28.3         matrixStats_0.61.0
[17] GenomeInfoDbData_1.2.7      lifecycle_1.0.1
[19] zlibbioc_1.40.0             MatrixGenerics_1.6.0
[21] munsell_0.5.0               gtable_0.3.0
[23] restfulr_0.0.13             Biobase_2.54.0
[25] parallel_4.1.0              fansi_1.0.2
[27] Rcpp_1.0.8                  scales_1.1.1
[29] DelayedArray_0.20.0         Rsamtools_2.10.0
[31] rjson_0.2.21                ggplot2_3.3.5
[33] dplyr_1.0.7                 BiocIO_1.4.0
[35] grid_4.1.0                  cli_3.1.1
[37] tools_4.1.0                 bitops_1.0-7
[39] magrittr_2.0.2              RCurl_1.98-1.5
[41] tibble_3.1.6                crayon_1.4.2
[43] pkgconfig_2.0.3             MASS_7.3-55
[45] ellipsis_0.3.2              Matrix_1.4-0
[47] assertthat_0.2.1            R6_2.5.1
[49] GenomicAlignments_1.30.0    compiler_4.1.0
universalmotif • 211 views
ADD COMMENT
2
Entering edit mode
btremblay ▴ 60
@btremblay-15941
Last seen 5 months ago
Spain

Hi,

Indeed, that is a rather strange and cryptic error. Especially considering that the code that is causing the crash requires very little memory and completes within milliseconds.

However I've been able to recreate your error using my own institution's cluster (CentOS 7). Some quick debugging shows the last function which is called before the crash is from somewhere in the RcppThread package. I don't really understand what or why. I will dig a bit deeper and see if this is an issue on RcppThread's end. In the meanwhile, it appears that installing an older version of RcppThread fixes the issue (at least for me, let me know if this doesn't work):

remotes::install_github("tnagler/RcppThread@1.0.0")

Once done, reinstall universalmotif (from source, not as a pre-compiled binary if that is what you did) to re-compile the C++ code using the older RcppThread.

ADD COMMENT
1
Entering edit mode

Thanks for the quick response! The solution you suggested worked for me as well (I use Compute Canada Cedar, which also runs CentOS 7).

ADD REPLY

Login before adding your answer.

Traffic: 390 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6