initialization of AnnotationHub() in version 2.12.0 is problematically verbose
1
0
Entering edit mode
@robert-k-bradley-5997
Last seen 2.4 years ago
United States

After updating to R 3.5.0 and Bioconductor 3.7, I discovered that initializing an AnnotationHub object (version 2.12.0) is problematically verbose. I tried wiping the cached annotationhub.sqlite3 and index.rds, but that didn't fix the problem. The verbosity cannot be suppressed by suppressMessages() or suppressWarnings().

Please see an example and my sessionInfo below.

Thank you in advance for your help!

Thank you,

Rob

R version 3.5.0 (2018-04-23) -- "Joy in Playing"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE)
> library (AnnotationHub)
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, parApply, parCapply, parLapply,
    parLapplyLB, parRapply, parSapply, parSapplyLB

The following objects are masked from ‘package:stats’:

    IQR, mad, sd, var, xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, basename, cbind, colMeans,
    colnames, colSums, dirname, do.call, duplicated, eval, evalq,
    Filter, Find, get, grep, grepl, intersect, is.unsorted, lapply,
    lengths, Map, mapply, match, mget, order, paste, pmax, pmax.int,
    pmin, pmin.int, Position, rank, rbind, Reduce, rowMeans, rownames,
    rowSums, sapply, setdiff, sort, table, tapply, union, unique,
    unsplit, which, which.max, which.min

> ah = AnnotationHub()
updating metadata: retrieving 1 resource
  |======================================================================| 100%

MAPPING START: ((null)) [(null)]
SCALAR: output_dir ((null)) [(null)]
Value: (output_dir), Tag: (str)
SCALAR: output ((null)) [(null)]
Value: (output), Tag: (str)
SCALAR: staging_deploy_root ((null)) [(null)]
Value: (staging_deploy_root), Tag: (str)
SCALAR: biocadmin@staging.bioconductor.org:/loc/www/bioconductor-test.fhcrc.org ((null)) [(null)]
Value: (biocadmin@staging.bioconductor.org:/loc/www/bioconductor-test.fhcrc.org), Tag: (str)
SCALAR: production_deploy_root ((null)) [(null)]
Value: (production_deploy_root), Tag: (str)
SCALAR: webadmin@master.bioconductor.org:/extra/www/bioc ((null)) [(null)]
Value: (webadmin@master.bioconductor.org:/extra/www/bioc), Tag: (str)
SCALAR: release_version ((null)) [(null)]
Value: (release_version), Tag: (str)
SCALAR: 3.7 ((null)) [(null)]
Value: (3.7), Tag: (str)
SCALAR: r_version_associated_with_release ((null)) [(null)]
Value: (r_version_associated_with_release), Tag: (str)
SCALAR: 3.5.0 ((null)) [(null)]
Value: (3.5.0), Tag: (str)
SCALAR: devel_version ((null)) [(null)]
Value: (devel_version), Tag: (str)
SCALAR: 3.8 ((null)) [(null)]
Value: (3.8), Tag: (str)
SCALAR: versions ((null)) [(null)]
Value: (versions), Tag: (str)
SEQUENCE START: ((null)) [(null)]
SCALAR: 3.7 ((null)) [(null)]
Value: (3.7), Tag: (str)
SCALAR: 3.8 ((null)) [(null)]

...output omitted due to limit of field length...

SCALAR: mirroradmin@tuna.tsinghua.edu.cn ((null)) [(null)]
Value: (mirroradmin@tuna.tsinghua.edu.cn), Tag: (str)
SCALAR: mirror_url ((null)) [(null)]
Value: (mirror_url), Tag: (str)
SCALAR: http://mirrors.tuna.tsinghua.edu.cn/bioconductor/ ((null)) [(null)]
Value: (http://mirrors.tuna.tsinghua.edu.cn/bioconductor/), Tag: (str)
SCALAR: https_mirror_url ((null)) [(null)]
Value: (https_mirror_url), Tag: (str)
SCALAR: https://mirrors.tuna.tsinghua.edu.cn/bioconductor/ ((null)) [(null)]
Value: (https://mirrors.tuna.tsinghua.edu.cn/bioconductor/), Tag: (str)
SCALAR: rsync ((null)) [(null)]
Value: (rsync), Tag: (str)
SCALAR: secure_mirror_from_master ((null)) [(null)]
Value: (secure_mirror_from_master), Tag: (str)
SCALAR: country_code ((null)) [(null)]
Value: (country_code), Tag: (str)
SCALAR: cn ((null)) [(null)]
Value: (cn), Tag: (str)
SCALAR: city ((null)) [(null)]
Value: (city), Tag: (str)
SCALAR: Peking ((null)) [(null)]
Value: (Peking), Tag: (str)
MAPPING END
SEQUENCE END
MAPPING END
SEQUENCE END
MAPPING END
snapshotDate(): 2018-04-23
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.4

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] AnnotationHub_2.12.0 BiocGenerics_0.26.0 

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.16                  AnnotationDbi_1.42.1         
 [3] magrittr_1.5                  IRanges_2.14.8               
 [5] bit_1.1-12                    xtable_1.8-2                 
 [7] R6_2.2.2                      blob_1.1.1                   
 [9] httr_1.3.1                    tools_3.5.0                  
[11] Biobase_2.40.0                DBI_1.0.0                    
[13] htmltools_0.3.6               yaml_2.1.19                  
[15] bit64_0.9-7                   digest_0.6.15                
[17] interactiveDisplayBase_1.18.0 shiny_1.0.5                  
[19] later_0.7.2                   S4Vectors_0.18.1             
[21] promises_1.0.1                curl_3.2                     
[23] memoise_1.1.0                 RSQLite_2.1.1                
[25] mime_0.5                      compiler_3.5.0               
[27] BiocInstaller_1.30.0          stats4_3.5.0                 
[29] httpuv_1.4.3                  pkgconfig_2.0.1              
AnnotationHub • 693 views
ADD COMMENT
0
Entering edit mode

My inquiry above is related to, but distinct from, my previous question regarding a "quiet" mode for AnnotationHub:

A: "quiet" mode for AnnotationHub?

As you can see from the response to my question above, suppressMessages() or invisible() was sufficient to prevent the messages. The messages are now much more verbose and cannot be suppressed by suppressMessages() or invisible().

ADD REPLY
0
Entering edit mode
@martin-morgan-1513
Last seen 3 months ago
United States

I don't see that output in my session. I'd guess that this comes from these lines of AnnotationHub code

    library(httr); library(yaml)
    yaml <- content(GET("http://bioconductor.org/config.yaml"), 
                    encoding="UTF-8", as="text")
    obj <- yaml.load(yaml)

It seems to come from the yaml library, installed with the 'DEBUG' flag set:

> download.package("yaml", ".", repos = BiocInstaller::biocinstallRepos(), type = "source")
> quit()
$ tar xzf yaml_*gz
$ cd yaml
yaml$ grep -C2 -r "SCALAR:" *
src/r_parse.c-        case YAML_SCALAR_EVENT:
src/r_parse.c-#if DEBUG
src/r_parse.c:          Rprintf("SCALAR: %s (%s) [%s]\n", event.data.scalar.value, event.data.scalar.tag, event.data.scalar.anchor);
src/r_parse.c-#endif
src/r_parse.c-          err = handle_scalar(&event, &s_stack_tail, s_handlers, eval_expr, eval_warning);
yaml$

If you installed from source on your end, then disable the DEBUG flag, otherwise it's an upstream problem for the yaml maintainer; let me know and I'll follow up with them.

 

 

ADD COMMENT
0
Entering edit mode

Hi Martin,

Thank you for the quick reply! I didn't manually install yaml, although I presume that it was installed/updated when I recently upgraded to R 3.5.0. It would be great if you would be willing to please reach out to the yaml maintainer.

I tried updating yaml manually via biocLite() and discovered that after doing so, I can no longer even execute AnnotationHub():

> library (yaml)
> source ("https://bioconductor.org/biocLite.R")
Bioconductor version 3.7 (BiocInstaller 1.30.0), ?biocLite for help
> biocLite ("yaml")
BioC_mirror: https://bioconductor.org
Using Bioconductor 3.7 (BiocInstaller 1.30.0), R 3.5.0 (2018-04-23).
Installing package(s) ‘yaml’
trying URL 'https://cran.rstudio.com/bin/macosx/el-capitan/contrib/3.5/yaml_2.1.19.tgz'
Content type 'application/x-gzip' length 185161 bytes (180 KB)
==================================================
downloaded 180 KB

The downloaded binary packages are in
    /var/folders/vv/zt0ykhrj5zb2cjnfddb6_szr0000gn/T//RtmpozrT0b/downloaded_packages
> library (AnnotationHub)
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ, clusterExport, clusterMap, parApply, parCapply, parLapply, parLapplyLB, parRapply, parSapply,
    parSapplyLB

The following objects are masked from ‘package:dplyr’:

    combine, intersect, setdiff, union

The following objects are masked from ‘package:stats’:

    IQR, mad, sd, var, xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, basename, cbind, colMeans, colnames, colSums, dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
    grepl, intersect, is.unsorted, lapply, lengths, Map, mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank, rbind, Reduce,
    rowMeans, rownames, rowSums, sapply, setdiff, sort, table, tapply, union, unique, unsplit, which, which.max, which.min

> ah = AnnotationHub()
Error in value[[3L]](cond) : failed to connect
  reason: lazy-load database '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/yaml/R/yaml.rdb' is corrupt
  Consider rerunning with 'localHub=TRUE'
In addition: Warning message:
In .biocVersionDate(biocVersion()) : internal error -3 in R_decompress1
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.4

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  grDevices utils     datasets  stats     graphics  methods   base     

other attached packages:
[1] AnnotationHub_2.12.0 BiocGenerics_0.26.0  BiocInstaller_1.30.0 yaml_2.1.19          dplyr_0.7.4          tidyr_0.8.0          readr_1.1.1          tibble_1.4.2        
[9] magrittr_1.5        

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.16                  pillar_1.2.2                  compiler_3.5.0                later_0.7.2                   bindr_0.1.1                  
 [6] tools_3.5.0                   digest_0.6.15                 bit_1.1-12                    RSQLite_2.1.1                 memoise_1.1.0                
[11] pkgconfig_2.0.1               rlang_0.2.0                   shiny_1.0.5                   DBI_1.0.0                     curl_3.2                     
[16] bindrcpp_0.2.2                httr_1.3.1                    IRanges_2.14.8                S4Vectors_0.18.1              hms_0.4.2                    
[21] stats4_3.5.0                  bit64_0.9-7                   Biobase_2.40.0                glue_1.2.0                    R6_2.2.2                     
[26] AnnotationDbi_1.42.1          purrr_0.2.4                   blob_1.1.1                    promises_1.0.1                htmltools_0.3.6              
[31] assertthat_0.2.0              mime_0.5                      interactiveDisplayBase_1.18.0 xtable_1.8-2                  httpuv_1.4.3                 
ADD REPLY
0
Entering edit mode

This seems to be something going wrong with your yaml installation, triggered by the same lines of AnnnotationHub code but really just tickling a deeper problem in your installation. You could try verifying this, e.g, with this modified version of the commands (without AnnotationHub in the picture)

    yaml <- httr::content(httr::GET("http://bioconductor.org/config.yaml"), 
                    encoding="UTF-8", as="text")
    obj <- yaml::yaml.load(yaml)

if there are still problems make sure there are no instances of R running, start a new R session, use remove.packages("yaml") (repeating until told there are no versions of yaml to remove), try installing yaml again, and once more test with the lines above.

ADD REPLY
0
Entering edit mode

Thank you for the suggestion. I tried the above yaml commands, but didn't encounter a problem:

> yaml <- httr::content(httr::GET("http://bioconductor.org/config.yaml"), encoding="UTF-8", as="text")
> obj <- yaml::yaml.load(yaml)

 

ADD REPLY
0
Entering edit mode

I tried uninstalling both yaml and AnnotationHub and then reinstalling AnnotationHub. I'm not encountering problems now! I don't know what the origin of the problem was, but it seems to be fixed.

Thank you again for your quick responses and help!

Best wishes,

Rob

> library (AnnotationHub)
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ, clusterExport, clusterMap, parApply, parCapply, parLapply, parLapplyLB, parRapply, parSapply,
    parSapplyLB

The following objects are masked from ‘package:dplyr’:

    combine, intersect, setdiff, union

The following objects are masked from ‘package:stats’:

    IQR, mad, sd, var, xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, basename, cbind, colMeans, colnames, colSums, dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
    grepl, intersect, is.unsorted, lapply, lengths, Map, mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank, rbind, Reduce,
    rowMeans, rownames, rowSums, sapply, setdiff, sort, table, tapply, union, unique, unsplit, which, which.max, which.min

> ah = AnnotationHub()
snapshotDate(): 2018-04-23
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.4

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  grDevices utils     datasets  stats     graphics  methods   base     

other attached packages:
[1] AnnotationHub_2.12.0 BiocGenerics_0.26.0  dplyr_0.7.4          tidyr_0.8.0          readr_1.1.1          tibble_1.4.2         magrittr_1.5        

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.16                  pillar_1.2.2                  compiler_3.5.0                BiocInstaller_1.30.0          later_0.7.2                  
 [6] bindr_0.1.1                   tools_3.5.0                   digest_0.6.15                 bit_1.1-12                    RSQLite_2.1.1                
[11] memoise_1.1.0                 pkgconfig_2.0.1               rlang_0.2.0                   shiny_1.0.5                   DBI_1.0.0                    
[16] curl_3.2                      yaml_2.1.19                   bindrcpp_0.2.2                httr_1.3.1                    IRanges_2.14.8               
[21] S4Vectors_0.18.1              hms_0.4.2                     stats4_3.5.0                  bit64_0.9-7                   glue_1.2.0                   
[26] Biobase_2.40.0                R6_2.2.2                      AnnotationDbi_1.42.1          purrr_0.2.4                   blob_1.1.1                   
[31] promises_1.0.1                htmltools_0.3.6               assertthat_0.2.0              mime_0.5                      interactiveDisplayBase_1.18.0
[36] xtable_1.8-2                  httpuv_1.4.3                 
ADD REPLY

Login before adding your answer.

Traffic: 341 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6