"BiocGenerics:::testPackage('PackageName')" works from R console but fails using "R CMD check PackageName" at terminal
Entering edit mode
Last seen 6.8 years ago
United States

Hi Everyone,

I ran into an error that I am trying to solve where my unit tests work from the R console via BiocGenerics:::testPackage("PackageName") but don't work when using R CMD check PackageName from the console.  Below is a description of a small R package that can duplicate the error.  Since I am not very creative I just call the package ErrorReproducer :(.

R -> ErrorReproducer.R

ErrorReproducer <- setClass("ErrorReproducer",
                                   slots = c(expressionData = "matrix",
                                             clinicalData = "data.frame"),
                                   validity  = function(object) {
                                     rowExpr <- rownames(object@expressionData)
                                     rowClinical <- rownames(object@clinicalData)
                                     if(length(intersect(rowExpr, rowClinical)) == 0) {
                                       return("There are no shared samples between the expression data and clinical data.")

inst -> unitTests -> test_ErrorReproducer.R


runTests <- function() {

test_ErrorReproducer = function () {
  #Setup some data to test the class generator function
  testMatrix <- as.matrix(mtcars[, 1:4])
  testDF <- mtcars[, 5:8]
  myObject <- ErrorReproducer(expressionData = testMatrix, clinicalData = testDF)

  rownames(testDF) <- 1:dim(testDF)[1]
  #No shared observations in the gene expression data (we do not need this check for the clinical data since it is a data frame)
  test1 <- try(ErrorReproducer(expressionData = testMatrix, clinicalData = testDF), silent = TRUE)
  checkEquals(test1[1], "Error in validObject(.Object) : \n  invalid class “ErrorReproducer” object: There are no shared samples between the expression data and clinical data.\n")

tests -> runTests.R

require("ErrorReproducer") || stop("unable to load ErrorReproducer package")
require("BiocGenerics") || stop("unable to load BiocGenerics package")





Package: ErrorReproducer
Type: Package
Title: Error Reproducer
Date: 2015-5-11
Version: 0.99.1
Author: Human
Maintainer: Human <human@species.org>
Depends: R (>= 3.1.0), methods
Suggests: BiocGenerics, RUnit
Description: Error Test
License: GPL (>= 2)

Using the code above, when I install and test the package within R using BiocGenerics:::testPackage("ErrorReproducer") the test returns no errors.

RUNIT TEST PROTOCOL -- Mon May 11 15:19:01 2015
Number of test functions: 1
Number of errors: 0
Number of failures: 0

1 Test Suite :
ErrorReproducer RUnit Tests - 1 test function, 0 errors, 0 failures
Number of test functions: 1
Number of errors: 0
Number of failures: 0

If however I do the same from the command line using R CMD check ErrorReproducer I get the following:

* checking tests ...
  Running ‘runTests.R’
Running the tests in ‘tests/runTests.R’ failed.
Last 13 lines of output:
  ErrorReproducer RUnit Tests - 1 test function, 0 errors, 1 failure
  FAILURE in test_ErrorReproducer: Error in checkEquals(test1[1], "Error in validObject(.Object) : \n  invalid class “ErrorReproducer” object: There are no shared samples between the expression data and clinical data.\n") :
    1 string mismatch
  Test files with failing tests
  Error in BiocGenerics:::testPackage("ErrorReproducer") :
    unit tests failed for package ErrorReproducer
  Execution halted


Does anyone know what I might be missing here or misunderstanding?  I just want to be able to use unit tests with my package, so having them fail from the terminal but not from the console is somewhat confusing.

Thanks in advance to anyone who is willing to help me with this.


> sessionInfo()
R version 3.1.3 (2015-03-09)
Platform: x86_64-unknown-linux-gnu (64-bit)
Running under: Ubuntu 14.04.2 LTS

 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RUnit_0.4.28           ErrorReproducer_0.99.1

loaded via a namespace (and not attached):
[1] BiocGenerics_0.12.1 parallel_3.1.3      tools_3.1.3
software error cmd check bioconductor • 1.5k views
Entering edit mode

The unit test fails for me in both scenarios on R-3.2.0 (which you should be using if the package you are developing is intended for submission to Bioconductor). 

A tip to make the BiocGenerics run act more like the R CMD check run is to look at the 'check' script in $R_HOME/bin. On my system it ends with this line:

echo 'tools:::.check_packages()' | R_DEFAULT_PACKAGES= LC_COLLATE=C "${R_HOME}/bin/R" --no-restore --slave --args ${args}

That tells us that R runs with different settings under R CMD check. Note that the locale is different which can impact tests involving sort order. Also R_DEFAULT_PACKAGES is unset meaning that no packages are attached by default when R starts up. So now start R like this:


And run


Does it still succeed?


Entering edit mode
Last seen 6.8 years ago
United States

Hi Dan,

I am using R on a server maintained by the organization I work at so I don't think I can control the version.  For the same reason I either cannot access or don't know where the R_HOME directory is.  I ran R as specified and BiocGenerics:::testPackage("ErrorReproducer") doesn't return any failures.

In any case, I figured out the problem and a work around.  The issue is that R CMD check adds the R escape character to what it views as invalid characters.  In this case, it appears the try function is returning the italic version of the double quotes around the class name in the character string I want to test.

To remove any character that might be automatically escaped, I use the make.names function.  Therefore the updated code that works is as follows:

test1 <- try(ErrorReproducer(expressionData = testMatrix, clinicalData = testDF), silent = TRUE) 
test1 <- make.names(test1[1]) 


Thanks again for your help.




Entering edit mode

I have options(useFancyQuotes=FALSE) in my otherwise minimal ~/.Rprofile file. maybe that helps?

A better paradigm for testing error strings might use tryCatch() with an error handler (that in this case simply returns the error string)

setClass("A", representation(x="numeric"))
test1 <- tryCatch(new("A", x="a"), error=conditionMessage)

For trapping your own errors an even more interesting approach is to use R's condition system to throw custom errors, and then validate that you receive the custom error. The following throws a familiar error if the argument is FALSE, but an error with a (S3) derived class 'my' (not very imaginative) if the argument is TRUE

f <- function(use.my) {
    cond <- simpleError("oops")
    if (use.my)
        class(cond) <- c("my", class(cond))

So the expected error is thrown and tested with

res1 <- tryCatch(f(TRUE), my=function(err) TRUE)
checkIdentical(TRUE, res1)
Entering edit mode

Dear Dr. Morgan,


I will most certainly implement your solution(s).  And obviously, your solution should be added as the correct answer to this problem :). 



Wade Copeland


Login before adding your answer.

Traffic: 506 users visited in the last hour
Help About
Access RSS

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6