modelGeneVar in MulticoreParam: wrong args for environment subassignment
0
0
Entering edit mode
@lluis-revilla-sancho
Last seen 1 day ago
European Union

I am having problems running scran::modelGeneVar, the error seems to come from BiocParallel not collecting the data correctly from the workers: wrong args for environment subassignment.

But I am not sure as I think I ran this code without problems previously. I am sorry I couldn't reduce the example to something reproducible, but I hope that the information provided is enough to redirect me . In previous process on the same R session I saw a spike on computer usage to more than 50 cores when multi_core only asks for 10 (maybe it is related).

Code and related information:

> gene_var_filt2 <- scran::modelGeneVar(gf_rescales, 
+                                       block = gf_rescales$type,
+                                       BPPARAM = multi_core,
+                                       assay.type = "corrected")
  |                                                               |   0%
Error in reducer$value.cache[[as.character(idx)]] <- values : 
  wrong args for environment subassignment
> traceback()
20: .reducer_add(reducer, njob, value)
19: .reducer_add(reducer, njob, value)
18: .collect_result(manager, reducer, progress, BPPARAM)
17: .bploop_impl(ITER = ITER, FUN = FUN, ARGS = ARGS, BPPARAM = BPPARAM, 
        BPOPTIONS = BPOPTIONS, BPREDO = BPREDO, reducer = reducer, 
        progress.length = length(redo_index))
16: bploop.lapply(manager, BPPARAM = BPPARAM, BPOPTIONS = BPOPTIONS, 
        ...)
15: bploop(manager, BPPARAM = BPPARAM, BPOPTIONS = BPOPTIONS, ...)
14: .bpinit(manager = manager, X = X, FUN = FUN, ARGS = ARGS, BPPARAM = BPPARAM, 
        BPOPTIONS = BPOPTIONS, BPREDO = BPREDO)
13: BiocParallel::bplapply(X, FUN, ..., BPPARAM = BPPARAM)
12: BiocParallel::bplapply(X, FUN, ..., BPPARAM = BPPARAM)
11: S4Arrays:::bplapply2(seq_along(grid), FUN_WRAPPER, grid, verbose, 
        FUN, ..., BPPARAM = BPPARAM)
10: gridApply(grid, FUN_WRAPPER, FUN, x, as.sparse, verbose, verbose_read_block, 
        ..., BPPARAM = BPPARAM, verbose = FALSE)
9: blockApply(x, FUN = .sparse_helper, beachmat_internal_FUN = FUN, 
       ..., grid = grid, as.sparse = NA, BPPARAM = BPPARAM)
8: .blockApply2(x, FUN = FUN, ..., grid = grid, coerce.sparse = coerce.sparse, 
       BPPARAM = BPPARAM, beachmat_by_row = TRUE)
7: rowBlockApply(x, FUN = block.FUN, block = as.integer(block) - 
       1L, nblocks = nlevels(block), ..., BPPARAM = BPPARAM) at utils_variance.R#31
6: .compute_mean_var(x, block = block, design = design, subset.row = s, 
       block.FUN = compute_blocked_stats_none, residual.FUN = compute_residual_stats_none, 
       BPPARAM = BPPARAM) at modelGeneVar.R#149
5: FUN(subset.row) at modelGeneVar.R#154
4: .model_gene_var(x = assay(x, i = assay.type), ...) at modelGeneVar.R#186
3: .local(x, ...)
2: scran::modelGeneVar(gf_rescales, block = gf_rescales$type, BPPARAM = multi_core, 
       assay.type = "corrected") at modelGeneVar.R#176
1: scran::modelGeneVar(gf_rescales, block = gf_rescales$type, BPPARAM = multi_core, 
       assay.type = "corrected")

> sessionInfo()
R version 4.4.0 (2024-04-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 22.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=es_ES.UTF-8        LC_COLLATE=en_GB.UTF-8    
 [5] LC_MONETARY=es_ES.UTF-8    LC_MESSAGES=en_GB.UTF-8   
 [7] LC_PAPER=es_ES.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Madrid
tzcode source: system (glibc)

attached base packages:
[1] stats4    stats     graphics  grDevices datasets  utils     methods  
[8] base     

other attached packages:
 [1] dplyr_1.1.4                 forcats_1.0.0              
 [3] scran_1.32.0                scds_1.20.0                
 [5] scDblFinder_1.18.0          scater_1.32.0              
 [7] scuttle_1.14.0              robustbase_0.99-2          
 [9] PCAtools_2.16.0             ggrepel_0.9.5              
[11] patchwork_1.2.0             here_1.0.1                 
[13] batchelor_1.20.0            GO.db_3.19.1               
[15] ggtext_0.1.2                DropletUtils_1.24.0        
[17] SingleCellExperiment_1.26.0 SummarizedExperiment_1.34.0
[19] DelayedMatrixStats_1.26.0   DelayedArray_0.30.1        
[21] SparseArray_1.4.8           S4Arrays_1.4.1             
[23] abind_1.4-5                 Matrix_1.7-0               
[25] MatrixGenerics_1.16.0       matrixStats_1.3.0          
[27] clustree_0.5.1              ggraph_2.2.1               
[29] ggplot2_3.5.1               bluster_1.14.0             
[31] biomaRt_2.60.0              ensembldb_2.28.0           
[33] AnnotationFilter_1.28.0     GenomicFeatures_1.56.0     
[35] AnnotationDbi_1.66.0        Biobase_2.64.0             
[37] GenomicRanges_1.56.1        GenomeInfoDb_1.40.1        
[39] IRanges_2.38.1              S4Vectors_0.42.1           
[41] BiocSingular_1.20.0         BiocParallel_1.38.0        
[43] AnnotationHub_3.12.0        BiocFileCache_2.12.0       
[45] dbplyr_2.5.0                BiocGenerics_0.50.0        
[47] usethis_2.2.3               targets_1.7.1              

loaded via a namespace (and not attached):
  [1] BiocIO_1.14.0            bitops_1.0-7            
  [3] filelock_1.0.3           tibble_3.2.1            
  [5] R.oo_1.26.0              cellranger_1.1.0        
  [7] polyclip_1.10-6          pROC_1.18.5             
  [9] XML_3.99-0.17            lifecycle_1.0.4         
 [11] httr2_1.0.1              edgeR_4.2.0             
 [13] rprojroot_2.0.4          processx_3.8.4          
 [15] lattice_0.22-6           MASS_7.3-60.2           
 [17] backports_1.5.0          magrittr_2.0.3          
 [19] limma_3.60.3             yaml_2.3.10             
 [21] metapod_1.12.0           cowplot_1.1.3           
 [23] DBI_1.2.3                ResidualMatrix_1.14.1   
 [25] zlibbioc_1.50.0          purrr_1.0.2             
 [27] R.utils_2.12.3           RCurl_1.98-1.14         
 [29] tweenr_2.0.3             rappdirs_0.3.3          
 [31] GenomeInfoDbData_1.2.12  irlba_2.3.5.1           
 [33] dqrng_0.4.1              codetools_0.2-20        
 [35] xml2_1.3.6               ggforce_0.4.2           
 [37] tidyselect_1.2.1         UCSC.utils_1.0.0        
 [39] farver_2.1.2             ScaledMatrix_1.12.0     
 [41] viridis_0.6.5            GenomicAlignments_1.40.0
 [43] jsonlite_1.8.8           BiocNeighbors_1.22.0    
 [45] tidygraph_1.3.1          systemfonts_1.1.0       
 [47] tools_4.4.0              progress_1.2.3          
 [49] ragg_1.3.2               Rcpp_1.0.13             
 [51] glue_1.7.0               gridExtra_2.3           
 [53] xfun_0.46                HDF5Array_1.32.0        
 [55] withr_3.0.0              BiocManager_1.30.23     
 [57] fastmap_1.2.0            rhdf5filters_1.16.0     
 [59] fansi_1.0.6              callr_3.7.6             
 [61] digest_0.6.36            rsvd_1.0.5              
 [63] secretbase_1.0.0         R6_2.5.1                
 [65] textshaping_0.4.0        colorspace_2.1-1        
 [67] scattermore_1.2          RSQLite_2.3.7           
 [69] R.methodsS3_1.8.2        utf8_1.2.4              
 [71] tidyr_1.3.1              generics_0.1.3          
 [73] renv_1.0.7.9000          data.table_1.15.4       
 [75] rtracklayer_1.64.0       prettyunits_1.2.0       
 [77] graphlayouts_1.1.1       httr_1.4.7              
 [79] pkgconfig_2.0.3          gtable_0.3.5            
 [81] blob_1.2.4               XVector_0.44.0          
 [83] base64url_1.4            ProtGenerics_1.36.0     
 [85] scales_1.3.0             png_0.1-8               
 [87] knitr_1.48               rstudioapi_0.16.0       
 [89] reshape2_1.4.4           rjson_0.2.21            
 [91] curl_5.2.1               cachem_1.1.0            
 [93] rhdf5_2.48.0             stringr_1.5.1           
 [95] BiocVersion_3.19.1       parallel_4.4.0          
 [97] vipor_0.4.7              restfulr_0.0.15         
 [99] pillar_1.9.0             grid_4.4.0              
 [ reached getOption("max.print") -- omitted 34 entries ]

Input data:

> gf_rescales
class: SingleCellExperiment 
dim: 33047 306459 
metadata(0):
assays(1): corrected
rownames(33047): ENSG00000243485 ENSG00000238009 ...
  ENSG00000278817 ENSG00000277196
rowData names(0):
colnames: NULL
colData names(10): Sample Barcode ... date_infection
  post_omicron
reducedDimNames(1): PCA
mainExpName: NULL
altExpNames(0):
> multi_core
class: MulticoreParam
  bpisup: FALSE; bpnworkers: 10; bptasks: 2147483647; bpjobname: BPJOB
  bplog: FALSE; bpthreshold: INFO; bpstopOnError: TRUE
  bpRNGseed: ; bptimeout: NA; bpprogressbar: TRUE
  bpexportglobals: TRUE; bpexportvariables: FALSE; bpforceGC: FALSE
  bpfallback: TRUE
  bplogdir: NA
  bpresultdir: NA
  cluster type: FORK

The gf_rescales$type column is present.

BiocParallel scran BiocParall • 147 views
ADD COMMENT

Login before adding your answer.

Traffic: 364 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6