The support.bioconductor.org editor has been updated to markdown! Please see more info at: Tutorial: Updated Support Site Editor

Question: Disable messages for bplapply when in parallel
1
gravatar for ioannis.vardaxis
11 months ago by
Norway/Trondheim/NTNU
ioannis.vardaxis20 wrote:

Hey,

I have a simple question.

I am using  the BiocParallel::bplapply in a package I am building, and the function runs in parallel. However every time the function will start running I get the following messages:

Setting options('download.file.method.GEOquery'='auto')
Setting options('GEOquery.inmemory.gpl'=FALSE)

If I use 4 workers I will get those messages 4 times. The thing is that BiocParallel::bplapply is run in a for loop and every time it is called it prints those messages which is very annoying because I print some messages in my code and those extra messages break the flow.

Is there any way to disable those mesages somehow? I have tried  suppressMessages and suppressPackageStartupMessages with no success.

if I dont run the program in parallel then no message is printed. So I guess the problem is that GEOquery package is loaded in each worker each time a worker is called and therefore the messages are produced.

 

Thanks

biocparallel bplapply • 257 views
ADD COMMENTlink modified 11 months ago by Martin Morgan ♦♦ 22k • written 11 months ago by ioannis.vardaxis20
Answer: Disable messages for bplapply when in parallel
0
gravatar for Martin Morgan
11 months ago by
Martin Morgan ♦♦ 22k
United States
Martin Morgan ♦♦ 22k wrote:

I guess you're on Widows and / or using 'SnowParam()`, which starts new processes and loads packages. If you are using a for loop then one thing you might do start the cluster outside the loop, and stop it at the end

register(SnowParam())    # default workers
bpstart()                # start default workers
...                      # use workers
bpstop()                 # top workers

The message comes from GEOquery, and can indeed be supressed

> suppressPackageStartupMessages(library(GEOquery))
> 

So I guess the problem is that you are not suppressing the message at the right place in the code -- do it inside the function that you are using in bplapply().

ADD COMMENTlink written 11 months ago by Martin Morgan ♦♦ 22k

I am on MAC and I register the snow workers like the following:

snow <- BiocParallel::SnowParam(workers = 4, type = 'SOCK', progressbar=FALSE)
BiocParallel::register(snow, default=TRUE)

I tried bpstart/bpstop but the messages are still printed every time the loop is run.

I tested the following: library(GEOquery) just to see if I get any messages, but no messages are printed.

I then tested to load my package (say pkg) and I get the messages.... which is very strange.

So the problem is when my package is loaded, and it is also loaded every time the workers start..

ADD REPLYlink modified 11 months ago by Martin Morgan ♦♦ 22k • written 11 months ago by ioannis.vardaxis20

remember to bpstart() once, before all your bplapply(). You'll get a message, but only once.

It sounds like your package depends or imports GEOquery, directly or indirectly, and that your test library(GEOquery) was done in a session where the package was already attached to the search path. In the worker function, use

suppressPackageStartupMessages(library(YourPackage))

Provide further details, e.g., the code for a minimal reproducible example or a link to github repository where your package is being developed, if you need further help.

ADD REPLYlink written 11 months ago by Martin Morgan ♦♦ 22k

I tried

suppressPackageStartupMessages(library(YourPackage))

inside the function I call in parallel but I still get the messages.

Link to my package:

https://bioconductor.org/packages/devel/bioc/html/MACPET.html

Now this version also runs bplapply() but not in a loop (the loop is prepared for an update of the pakage)

If you run the function MACPETUlt with the example provided at the documentation, but initiate a parallel backhead, then you will get the messages.

 

 

ADD REPLYlink written 11 months ago by ioannis.vardaxis20

I looked into this more thoroughly, and have updated BiocParallel (version 1.13.3) to suppress package startup messages. This version is available now under R and Bioc 'devel' via biocLite("Bioconductor/BiocParallel") or after tomorrow's builds for R and bioc 'devel' via biocLite("BiocParallel").

Your own code can be made more efficient with a change such as

diff --git a/R/MACPETUlt.R b/R/MACPETUlt.R
index 00881d2..12247a5 100644
--- a/R/MACPETUlt.R
+++ b/R/MACPETUlt.R
@@ -377,6 +377,11 @@ MACPETUlt = function(SA_AnalysisDir = "", SA_stages = c(0:3), SA_prefix = "MACPE
         S1_RbowtieIndexPrefix = S1_RbowtieIndexPrefix, S1_RbowtieRefDir = S1_RbowtieRefDir,
         S2_PairedEndBAMpath = S2_PairedEndBAMpath, S2_image = S2_image, S2_BlackList = S2_BlackList,
         S3_fileSelfDir = S3_fileSelfDir, S3_image = S3_image, S3_method = S3_method)
+    if (!bpisup()) {
+        bpstart()
+        on.exit(bpstop())
+    }
+                        
     #--------------------------------------------
     #---------------Check input is correct:
     #--------------------------------------------

which starts the workers once in your overall workflow, rather than in each call to bplapply().

Questions about package development should be addressed to the Bioconductor devel mailing list.

ADD REPLYlink written 11 months ago by Martin Morgan ♦♦ 22k

Perfect!

Thanks for the help!

ADD REPLYlink written 10 months ago by ioannis.vardaxis20
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 167 users visited in the last hour