I have been using BiocParallel for a while (using sockets and snow; please see below) and have noted that at least using these settings there seems to be no load balancing (the distribution of the work seems to occur only once, right at the beginning. Therefore workers that are faster (or have less calculations to do) are sometimes inactive while there are jobs queued for other slower workers). Are there settings of BiocParallel that would allow load balancing (for instance by changing to SerialParam / MulticoreParam / BatchJobsParam / DoparParam instances or by just changing the settings for SnowParam).
Thank you very much,
library(BiocParallel) bp.param <- SnowParam(workers = 32, type = "SOCK") bplapply(x, FUN, BPPARAM = bp.param)