Use MulticoreParam under BatchtoolsParam
1
0
Entering edit mode
@a9c5eb20
Last seen 11 months ago
United States

Hello,

Is it a good practice to use MulticoreParam on each of the cores/cpus requested by BatchtoolsParam?

Regards

BatchtoolsParam MulticoreParam BiocParallel • 980 views
ADD COMMENT
1
Entering edit mode
@martin-morgan-1513
Last seen 4 months ago
United States

Generally it is not a good idea to have nested parallel calls, because it is too easy to over-subscribe the compute nodes (e.g., on a six-core machine running six jobs, each job starts six more parallel jobs, so there are 36 jobs contending for only 6 cores). In batchtools it would be better to use slurm or other facilities for distributing appropriate numbers of tasks to each compute node.

ADD COMMENT

Login before adding your answer.

Traffic: 1049 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6