Search
Tutorial: doParallel foreach already exporting variable(s)
0
gravatar for solo7773
8 months ago by
solo777310
solo777310 wrote:

According to my use of doParallel, I share my experience here.

refGenome = BSgenome.Hsapiens.UCSC.hg19::BSgenome.Hsapiens.UCSC.hg19
# start parallel
  library(doParallel)
  nCPUcores = detectCores()
  if (nCPUcores < 3) {
    registerDoSEQ()
  }else{
    cl = makeCluster(nCPUcores - 1)
    registerDoParallel(cl)
  }

 

# example foreach 1
resParallel = foreach(g = 1:1000) %dopar% {
  # code omitted
}

 

# example foreach 2
resParallel = foreach(g = 1:1000, .export = c('refGenome')) %dopar% {
​  # code omitted
}

# example foreach 3
resParallel = foreach(g = 1:1000, .packages = c('BSgenome.Hsapiens.UCSC.hg19')) %dopar% {
  # code omitted
}

 

Explanation:

1) example foreach 1 throws fatal error, something like 'task 1 failed', 'object *** not found'

 

2) example foreach 2 works but gives warning like 'already exporting variable(s): refGenome'

 

Case 1) and 2) usually happen on Windows, because foreach use its own environment. When use functions or packages from outside, you need to use .export or .packages to make them available inside the foreach loop.

 

3) example foreach 3 finally does my work

 

I have to acknowledge that lots of solutions for case 1) are available online and I also benefit and learned from them. They help me to find ways for case 2). I suppress the annoying warning message. Hope it'll also be useful for you guys.

ADD COMMENTlink modified 8 months ago • written 8 months ago by solo777310
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 299 users visited in the last hour