Question: Help using 'handler' function with bptry()
gravatar for Peter Hickey
13 months ago by
Peter Hickey450
Walter and Eliza Hall Institute of Medical Research, Melbourne, Australia
Peter Hickey450 wrote:

In a long-running function I am writing, I use bptry() around bplapply() so that I might re-do failed tasks (this is such a cool feature!). If any failures are detected, I want to return a custom error message to help the user re-run just the failed tasks, along with the partial results. Right now, I'm using warning() to return the error message but this feels wrong because it really is an error and not a warning.

I feel like this might be a job for the bplist_error or bperror arguments of bptry(), but I don't understand them well enough. Help doing this is much appreciated.

Here's a toy example.

f <- function(x, BPPARAM, BPREDO = list()) {
    val <- bptry(bplapply(x, sqrt, BPPARAM = BPPARAM, BPREDO = BPREDO))
    if (!all(bpok(val))) {
        # Feels like this should be an error rather than a warning, but I 
        # want to return the partial results so can't use stop() (I think)
        warning("f() encountered errors: ",
                sum(!bpok(val)), " of ", length(val), " tasks failed.\n",
                "f() has returned partial results, including errors, ",
                "for debugging purposes.\n",
                "It may be possible to re-run just these failed tasks.\n", 
                "See help(\"f\")",
                call. = FALSE)
    data.frame(x = unlist(x), val = unlist(val))

BPPARAM <- MulticoreParam(workers = 2, stop.on.error = FALSE)
# Good input
a <- f(list(1, 2, 3), BPPARAM = BPPARAM)
#>   x      val
#> 1 1 1.000000
#> 2 2 1.414214
#> 3 3 1.732051
# Bad input
b <- f(list(1, "2", 3), BPPARAM = BPPARAM)
#> Warning: f() encountered errors: 1 of 3 tasks failed.
#> f() has returned partial results, including errors, for debugging purposes.
#> It may be possible to re-run just these failed tasks.
#> See help("f")
#> [[1]]
#> [1] 1
#> [[2]]
#> <remote_error in FUN(...): non-numeric argument to mathematical function>
#> traceback() available as 'attr(x, "traceback")'
#> [[3]]
#> [1] 1.732051
# Re-do failed task
c <- f(list(1, 2, 3), BPPARAM = BPPARAM, BPREDO = b)
#> resuming previous calculation ...
#>   x      val
#> 1 1 1.000000
#> 2 2 1.414214
#> 3 3 1.732051
#> * sessionInfo()
#> R version 3.5.0 (2018-04-23)
#> Platform: x86_64-apple-darwin15.6.0 (64-bit)
#> Running under: macOS Sierra 10.12.6
#> Matrix products: default
#> BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
#> locale:
#> [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base
#> other attached packages:
#> [1] BiocParallel_1.15.3 repete_0.0.0.9011   devtools_1.13.5
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_0.12.17           lobstr_0.0.0.9000      XVector_0.21.1
#>  [4] magrittr_1.5           GenomicRanges_1.33.5   BiocGenerics_0.27.0
#>  [7] zlibbioc_1.27.0        IRanges_2.15.13        munsell_0.4.3
#> [10] colorspace_1.3-2       rlang_0.2.0            stringr_1.3.1
#> [13] GenomeInfoDb_1.17.1    plyr_1.8.4             tools_3.5.0
#> [16] parallel_3.5.0         withr_2.1.2            fortunes_1.5-4
#> [19] digest_0.6.15          GenomeInfoDbData_1.1.0 BiocManager_1.29.45
#> [22] pryr_0.1.4             S4Vectors_0.19.5       bitops_1.0-6
#> [25] codetools_0.2-15       RCurl_1.95-4.10        memoise_1.1.0
#> [28] stringi_1.2.2          compiler_3.5.0         scales_0.5.0
#> [31] stats4_3.5.0
#> Bioconductor version '3.8'
#>   * 0 packages out-of-date
#>   * 1 packages too new
#> create a valid installation with
#>   BiocManager::install("reprex", update = TRUE, ask = FALSE)

#> Warning message:
#> 0 packages out-of-date; 1 packages too new

Created on 2018-05-25 by the reprex package (v0.2.0).

ADD COMMENTlink modified 13 months ago by Martin Morgan ♦♦ 23k • written 13 months ago by Peter Hickey450
Answer: Help using 'handler' function with bptry()
gravatar for Martin Morgan
13 months ago by
Martin Morgan ♦♦ 23k
United States
Martin Morgan ♦♦ 23k wrote:

The basic paradigm in R is that an error terminates execution, whereas a warning allows continuation. Within this paradigm, it's not possible to both signal an error and return a value. You could use bplist_error= to do something special and then signal the condition, e.g., store the value for later retrieval

handler <- function() {
    value <- NULL
    list(error = function(err) {
        value <<- attr(err, "result")
    }, get = function() {

used as

h <- handler()
x <- bptry(
    bplapply(list(1, "2", 3), sqrt),
    bplist_error = h$error, bperror = stop

This could be achieved probably more directly by testing the return value of bptry() with bpok(), and using that to store-then-signal.

I think this would be quite confusing for your average user; it's confusing enough for me!

ADD COMMENTlink written 13 months ago by Martin Morgan ♦♦ 23k

Perhaps I need to think about this more. The design I had in mind is as follows.

  • f(): A user-level function that takes a SummarizedExperiment. It breaks up the assay-data into chunks and uses BiocParallel to process these chunks in parallel. Finally, it constructs a new assay from these outputs that is added to the input SummarizedExperiment and returned.
  • .f(): An internal function that operates on the chunks of data. The user should never need to call this directly.

f() is a long-running function (on the order of hours) because there are hundreds to thousands of data chunks to process. In my experience, .f() will occasionally fail (e.g., due to a dropped node on the cluster). When any of the calls to .f() have failed, then the call to f() has also failed because the SummarizedExperiment cannot be constructed. So I feel like an error (rather than a warning or just a message) should be issued. When these types of failures occur, I'd like to offer the user an easy way to resume the computation without having to re-do all the work. And I'd like to offer this without the user having to know too much about BiocParallel.

This is a mock-up of the desired user experience:

# Instance where everything works out as expected
x <- f(se, arg1, arg2, BPPARAM) 
# User can go ahead and use 'x' 

# Instance where errors occur due to cluster issues 
x <- f(se, arg1, arg2, BPPARAM) 
# f() raises an error/warning/message and returns the chunk-level data for debugging. 
#> Hey, the job encountered some issues. You can try re-doing the failed calculation by running 'f(se, arg1, arg2, BPPARAM, BPREDO = x)' 
x <- f(se, arg1, arg2, BPPARAM, BPREDO = x) 
# Everything has now worked, user can go ahead and use 'x'
ADD REPLYlink modified 13 months ago • written 13 months ago by Peter Hickey450

In the case of a warning, you could return an object that contains the info to re-do the calculation; probably the original SummarizedExperiment with metadata() (so that a user doing se <- f(se, ...) doesn't lose data), or maybe better 'remember' at the end of f() the info to redo as well as a signature (e.g., digest() of the original SummarizedExperiment), with the instruction to invoked f(..., BPREDO = se). Then throw a error.

(probably moved to bioc-devel mailing list territory...

ADD REPLYlink written 13 months ago by Martin Morgan ♦♦ 23k

Very good point re not clobbering result if invoked with se <- f(se, ...). Hmm, there are more dragons here than I'd initially considered. I'll keep thinking on this (it may be more trouble than it's worth). Thanks for your advice, Martin.

ADD REPLYlink modified 13 months ago • written 13 months ago by Peter Hickey450
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 199 users visited in the last hour