Hi, I'm just dipping my toes in the water with my first scRNA dataset & am working through the examples at the simpleSingleCell workflow for droplet-based protocols. We've loaded directly from the hdf5 object instead of the *tsv.gz files but can't seem to get the barcodeRanks() function to work. It looks to me like there's a glitch in handling the chunks somehow, or that our hdf5 object has problems. The actual error we get is:

bcrank <- barcodeRanks(counts(sce))
Error: BiocParallel errors
 element index: 46, 47, 48, 49, 50, 51, ...
 first error: HDF5. Dataset. Read failed.
In addition: Warning messages:
1: In h5checktypeOrOpenLoc(file, readonly = TRUE, native = native) :
 An open HDF5 file handle exists. If the file has changed on disk meanwhile, the function may not work properly. Run ‘h5closeAll()’ to close all open HDF5 object handles.
2: In h5checktypeOrOpenLoc(file, readonly = TRUE, native = native) :
 An open HDF5 file handle exists. If the file has changed on disk meanwhile, the function may not work properly. Run ‘h5closeAll()’ to close all open HDF5 object handles.
3: call dbDisconnect() when finished working with a connection
4: stop worker failed:
 attempt to select less than one element in OneIndex

When I run traceback() on that we get:

10: stop(.error_bplist(res))
9: bplapply(seq_len(nblock), function(b) {
       if (get_verbose_block_processing()) 
           message("Processing block ", b, "/", nblock, " ... ", 
               appendLF = FALSE)
       viewport <- grid[[b]]
       block <- read_block(x, viewport)
       attr(block, "from_grid") <- grid
       attr(block, "block_id") <- b
       block_ans <- FUN(block, ...)
       if (get_verbose_block_processing()) 
           message("OK")
       block_ans
   }, BPPARAM = BPPARAM)
8: bplapply(seq_len(nblock), function(b) {
       if (get_verbose_block_processing()) 
           message("Processing block ", b, "/", nblock, " ... ", 
               appendLF = FALSE)
       viewport <- grid[[b]]
       block <- read_block(x, viewport)
       attr(block, "from_grid") <- grid
       attr(block, "block_id") <- b
       block_ans <- FUN(block, ...)
       if (get_verbose_block_processing()) 
           message("OK")
       block_ans
   }, BPPARAM = BPPARAM)
7: blockApply(x, GENERIC, na.rm = na.rm, grid = grid)
6: .BLOCK_row_summary(rowSums, t(x), na.rm = na.rm)
5: .local(x, na.rm, dims, ...)
4: colSums(m)
3: colSums(m)
2: unname(colSums(m))
1: barcodeRanks(counts(sce))

And for completeness our current versions of everything are

sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.5

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ensembldb_2.8.0             AnnotationFilter_1.8.0      GenomicFeatures_1.36.1     
 [4] AnnotationDbi_1.46.0        magrittr_1.5                AnnotationHub_2.16.0       
 [7] BiocFileCache_1.8.0         dbplyr_1.4.0                scater_1.12.2              
[10] ggplot2_3.1.1               DropletUtils_1.4.0          SingleCellExperiment_1.6.0 
[13] SummarizedExperiment_1.14.0 DelayedArray_0.10.0         BiocParallel_1.18.0        
[16] matrixStats_0.54.0          Biobase_2.44.0              GenomicRanges_1.36.0       
[19] GenomeInfoDb_1.20.0         IRanges_2.18.0              S4Vectors_0.22.0           
[22] BiocGenerics_0.30.0        

loaded via a namespace (and not attached):
 [1] ProtGenerics_1.16.0           bitops_1.0-6                  bit64_0.9-7                  
 [4] progress_1.2.2                httr_1.4.0                    tools_3.6.0                  
 [7] R6_2.4.0                      irlba_2.3.3                   HDF5Array_1.12.1             
[10] vipor_0.4.5                   DBI_1.0.0                     lazyeval_0.2.2               
[13] colorspace_1.4-1              withr_2.1.2                   tidyselect_0.2.5             
[16] gridExtra_2.3                 prettyunits_1.0.2             bit_1.1-14                   
[19] curl_3.3                      compiler_3.6.0                BiocNeighbors_1.2.0          
[22] rtracklayer_1.44.0            scales_1.0.0                  rappdirs_0.3.1               
[25] Rsamtools_2.0.0               stringr_1.4.0                 digest_0.6.19                
[28] R.utils_2.8.0                 XVector_0.24.0                pkgconfig_2.0.2              
[31] htmltools_0.3.6               limma_3.40.2                  rlang_0.3.4                  
[34] RSQLite_2.1.1                 shiny_1.3.2                   DelayedMatrixStats_1.6.0     
[37] dplyr_0.8.1                   R.oo_1.22.0                   RCurl_1.95-4.12              
[40] BiocSingular_1.0.0            GenomeInfoDbData_1.2.1        Matrix_1.2-17                
[43] Rcpp_1.0.1                    ggbeeswarm_0.6.0              munsell_0.5.0                
[46] Rhdf5lib_1.6.0                viridis_0.5.1                 R.methodsS3_1.7.1            
[49] stringi_1.4.3                 yaml_2.2.0                    edgeR_3.26.4                 
[52] zlibbioc_1.30.0               rhdf5_2.28.0                  plyr_1.8.4                   
[55] grid_3.6.0                    blob_1.1.1                    promises_1.0.1               
[58] dqrng_0.2.1                   crayon_1.3.4                  lattice_0.20-38              
[61] Biostrings_2.52.0             hms_0.4.2                     locfit_1.5-9.1               
[64] knitr_1.23                    pillar_1.4.1                  biomaRt_2.40.0               
[67] XML_3.98-1.19                 glue_1.3.1                    BiocManager_1.30.4           
[70] httpuv_1.5.1                  gtable_0.3.0                  purrr_0.3.2                  
[73] assertthat_0.2.1              xfun_0.7                      rsvd_1.0.0                   
[76] mime_0.6                      xtable_1.8-4                  later_0.8.0                  
[79] viridisLite_0.3.0             tibble_2.1.1                  GenomicAlignments_1.20.0     
[82] beeswarm_0.2.3                memoise_1.1.0                 interactiveDisplayBase_1.22.0

Thanks in advance,

Steve

I'm going to mark this as "solved by prayers to the god of filesystems".