Download errors during Bioconductor 3.12 update
0
1
Entering edit mode
@mjsteinbaugh
Last seen 4 months ago
Cambridge, MA

I'm seeing download errors with Bioconductor 3.12 for some packages currently, primarily some large data packages.

Here's an example error with reactome.db:

install("reactome.db")
Bioconductor version 3.12 (BiocManager 1.30.10), R 4.0.3 (2020-10-10)
Installing package(s) 'reactome.db'
installing the source package ‘reactome.db’

trying URL 'https://bioconductor.org/packages/3.12/data/annotation/src/contrib/reactome.db_1.74.0.tar.gz'
Content type 'application/x-gzip' length 430092992 bytes (410.2 MB)
================
downloaded 137.5 MB

Error in download.file(url, destfile, method, mode = "wb", ...) :
  (converted from warning) downloaded length 144140511 != reported length 430092992
Calls: install ... tryCatchList -> tryCatchOne -> doTryCatch -> download.file

I'm also seeing this with some of the BSgenome packages.

sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] bb8_0.2.32

loaded via a namespace (and not attached):
[1] compiler_4.0.3      BiocManager_1.30.10 magrittr_1.5
[4] tools_4.0.3         crayon_1.3.4        remotes_2.2.0
[7] rlang_0.4.8
reactome.db Bioconductor BiocManager • 2.4k views
ADD COMMENT
1
Entering edit mode

This 'works for me'. Usually the error download length XXX != reported length YYY indicates that an internet connectivity issue, and the best first line of attack is to try again, perhaps from a better connection. A second approach might be to visit the URL in, e.g., a browser https://bioconductor.org/packages/3.12/data/annotation/src/contrib/reactome.db_1.74.0.tar.gz to download the file, and then to install it locally, install.packages("reactome.db_1.74.0.tar.gz", repos = NULL, type = "source").

ADD REPLY
0
Entering edit mode

Thanks Martin, I'm still seeing it this morning on my home connection. I'm guessing it's an ISP / CDN issue, but just wanted to let you guys know. I'll see if I can replicate it on an Amazon EC2 instance.

ADD REPLY
1
Entering edit mode

Yeah this seems to be an issue specifically with the internal download.packages call. I can download the package manually using curl or wget fine and install directly from the tarball. The download only fails inside of the R install command.

ADD REPLY
2
Entering edit mode

Good catch. The method used to download packages is influenced by the global option download.file.method as documented on ?download.file. So it would be interesting to know whether

## "curl" or "wget" or "libcurl" or "wininet" (Windows only)
options(download.file.method = "curl")
BiocManager::install("reactome.db")

succeeds.

ADD REPLY
0
Entering edit mode

Thanks this is helpful for debugging.

libcurl seems to be the culprit on my machine.

options(download.file.method = "libcurl")
BiocManager::install("geneLenDataBase")
> BiocManager::install("geneLenDataBase")
Bioconductor version 3.12 (BiocManager 1.30.10), R 4.0.3 (2020-10-10)
Installing package(s) 'geneLenDataBase'
installing the source package ‘geneLenDataBase’

trying URL 'https://bioconductor.org/packages/3.12/data/experiment/src/contrib/geneLenDataBase_1.26.0.tar.gz'
Content type 'application/x-gzip' length 100436635 bytes (95.8 MB)
================================================
downloaded 92.8 MB

Warning in download.file(url, destfile, method, mode = "wb", ...) :
  downloaded length 97321263 != reported length 100436635
Calls: <Anonymous> ... tryCatchList -> tryCatchOne -> doTryCatch -> download.file
Warning in download.file(url, destfile, method, mode = "wb", ...) :
  URL 'https://bioconductor.org/packages/3.12/data/experiment/src/contrib/geneLenDataBase_1.26.0.tar.gz': Timeout of 5 seconds was reached
Calls: <Anonymous> ... tryCatchList -> tryCatchOne -> doTryCatch -> download.file
Error in download.file(url, destfile, method, mode = "wb", ...) :
  download from 'https://bioconductor.org/packages/3.12/data/experiment/src/contrib/geneLenDataBase_1.26.0.tar.gz' failed
Warning in download.packages(pkgs, destdir = tmpd, available = available,  :
  download of package ‘geneLenDataBase’ failed
Calls: <Anonymous> ... doTryCatch -> install.packages -> download.packages

curl works, but with some issues.

options(download.file.method = "curl")
BiocManager::install("geneLenDataBase")
Installing package(s) 'geneLenDataBase'
Warning: unable to access index for repository https://bioconductor.org/packages/3.12/data/annotation/bin/macosx/contrib/4.0:
  Found continuation line starting '    <script type="te ...' at begin of record.
Warning: unable to access index for repository https://bioconductor.org/packages/3.12/data/experiment/bin/macosx/contrib/4.0:
  Found continuation line starting '    <script type="te ...' at begin of record.
Warning: unable to access index for repository https://bioconductor.org/packages/3.12/workflows/bin/macosx/contrib/4.0:
  Found continuation line starting '    <script type="te ...' at begin of record.
Warning: unable to access index for repository https://r.acidgenomics.com/bin/macosx/contrib/4.0:
  Line starting '<!DOCTYPE html> ...' is malformed!
installing the source package ‘geneLenDataBase’

wget also works but is a little overly verbose with the HTTP headers.

options(download.file.method = "wget")
BiocManager::install("geneLenDataBase")
ADD REPLY
0
Entering edit mode

A second work-around might be to change the option "timeout", e.g., options(timeout = 180L) as described on the help page of ?download.files. It would be good to know if this also works.

ADD REPLY

Login before adding your answer.

Traffic: 737 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6