Cannot connect to ExperimentHub server
1
0
Entering edit mode
enricoferrero ▴ 660
@enricoferrero-6037
Last seen 3.1 years ago
Switzerland

Hello,

I'm trying to get data from the ExperimentHub in a new R 4.0.2 environment, but I get the following error:

library(ExperimentHub)    
> eh <- ExperimentHub()
Cannot connect to ExperimentHub server, using 'localHub=TRUE' instead
/home/ferreen2/.cache/ExperimentHub
  does not exist, create directory? (yes/no): yes
Error in .updateHubDB(hub_bfc, .class, url, proxy, localHub) : 
  Invalid Cache: sqlite file
  Hub has not been added to cache
  Run again with 'localHub=FALSE'
> eh <- ExperimentHub(localHub = FALSE)
Cannot connect to ExperimentHub server, using 'localHub=TRUE' instead
Error in .updateHubDB(hub_bfc, .class, url, proxy, localHub) : 
  Invalid Cache: sqlite file
  Hub has not been added to cache
  Run again with 'localHub=FALSE'

It looks like the problem is the internet connection, but I am connected:

> pingr::is_online()
[1] TRUE
> curl::has_internet()
Testing for internet connectivity via https_proxy... success!
[1] TRUE

Has anyone come across this issue before? Is there anything I can do to get this to work?

Thank you! Enrico

P.S.: here's my session info for reference:

> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/prog/OpenBLAS/0.2.20-GCC-6.4.0-2.28/lib/libopenblas_haswellp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ExperimentHub_1.14.0 AnnotationHub_2.20.0 BiocFileCache_1.12.0 dbplyr_1.4.4        
[5] BiocGenerics_0.34.0 

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6                  later_1.1.0.1                 pillar_1.4.6                 
 [4] compiler_4.0.2                BiocManager_1.30.10           tools_4.0.2                  
 [7] digest_0.6.25                 packrat_0.5.0                 bit_4.0.4                    
[10] RSQLite_2.2.0                 memoise_1.1.0                 lifecycle_0.2.0              
[13] tibble_3.0.3                  pkgconfig_2.0.3               rlang_0.4.7                  
[16] shiny_1.5.0                   DBI_1.1.0                     rstudioapi_0.11              
[19] curl_4.3                      yaml_2.2.1                    fastmap_1.0.1                
[22] dplyr_1.0.1                   httr_1.4.2                    IRanges_2.22.2               
[25] generics_0.0.2                vctrs_0.3.2                   S4Vectors_0.26.1             
[28] rappdirs_0.3.1                stats4_4.0.2                  bit64_4.0.2                  
[31] tidyselect_1.1.0              Biobase_2.48.0                glue_1.4.1                   
[34] R6_2.4.1                      AnnotationDbi_1.50.3          purrr_0.3.4                  
[37] blob_1.2.1                    magrittr_1.5                  promises_1.1.1               
[40] ellipsis_0.3.1                htmltools_0.5.0               assertthat_0.2.1             
[43] xtable_1.8-4                  mime_0.9                      interactiveDisplayBase_1.26.3
[46] httpuv_1.5.4                  crayon_1.3.4                  BiocVersion_3.11.1
ExperimentHub AnnotationHub SingleR • 8.0k views
ADD COMMENT
0
Entering edit mode

Can you access https://experimenthub.bioconductor.org/ in the browser? Via readLines("https://experimenthub.bioconductor.org") (returning HTML that does not look like an error code)?

ADD REPLY
0
Entering edit mode

Thank you. Yes, I can access the website on the browser as well as through readLines().

For the sake of completeness, I am behind a company firewall, which is set up through the HTTP_PROXY environment variable. I have no problems with other connections from within R (e..g, downloading packages).

ADD REPLY
3
Entering edit mode
shepherl 4.1k
@lshep
Last seen 2 hours ago
United States

It looks like you are running behind a proxy. If you are running behind a proxy please set the proxy in ExperimentHub either by setting a system environment variable EXPERIMENT_HUB_PROXY or by running the following in R setExperimentHubOption(PROXY, <value>) before running the ExperimentHub() constructor.

ADD COMMENT
0
Entering edit mode

Thanks for getting back to me!

Yes, I'm behind a company firewall, which is set up through the HTTP_PROXY environment variable. I have no problems with other connections from within R (e..g, downloading packages).

I tried (variations of) the following:

setExperimentHubOption("PROXY", "http://my.proxy.com:8080")

and

Sys.setenv(EXPERIMENT_HUB_PROXY = Sys.getenv("HTTP_PROXY"))

But I always get this error now:

> eh <- ExperimentHub()
Assuming valid proxy connection through 'NULLNULLNULLNULLlist(proxy = "http://my.proxy.com", proxyport = 8080, proxyauth = 1)NULLNULL'
 If you experience connection issues consider using 'localHub=TRUE'
Error in bfcadd(bfc, rname = paste0(tolower(.class), ".sqlite3"), fpath = remote_db,  : 
  is.character(proxy) is not TRUE

Any idea what's wrong now?

ADD REPLY
0
Entering edit mode

This looks like a bug on our end. We will investigate more and get back to you shortly.

ADD REPLY
0
Entering edit mode

Sorry for the inconvenience, Its difficult for me to debug this since I'm not behind a proxy. Would you mind trying the following and pasting the result here.

library(ExperimentHub)
library(httr)
setExperimentHubOption("PROXY", "http://my.proxy.com:8080")
proxy = getExperimentHubOption("PROXY")
GET("https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3", proxy=proxy)
ADD REPLY
0
Entering edit mode

Yes, I get a positive result here:

> GET("https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3", proxy=proxy)
Response [https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3]
  Date: 2020-08-04 14:47
  Status: 200
  Content-Type: application/octet-stream
  Size: 3.59 MB
<BINARY BODY="">

Though it also works when I omit the proxy argument:

> GET("https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3")
Response [https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3]
  Date: 2020-08-04 15:08
  Status: 200
  Content-Type: application/octet-stream
  Size: 3.59 MB
<BINARY BODY="">
ADD REPLY
0
Entering edit mode

Just stating the obvious here, but it would look like ExperimentHub() is not parsing correctly the PROXY option or EXPERIMENT_HUB_PROXY environment variable correctly as a simple character.

ADD REPLY
1
Entering edit mode

Yes. I just want to make sure the proxy gets passed as the correct format to be utilized to allow the download. If it is suppose to be in the class format request instead of character then I need to adjust the formatting check for it rather than making sure its parsed correctly.

ADD REPLY
1
Entering edit mode

I pushed up a fix to BiocFileCache which backs the hubs. We think we have corrected the issue. The fix is in version 1.12.1 for release 3.11 and 1.13.1 for devel 3.12. These versions of the package should be available after tonight's rebuild (approx. ~1 PM EST tomorrow afternoon)

Cheers

ADD REPLY
0
Entering edit mode

OK, I could update BiocFileCache to 1.12.1 today. Then using: setExperimentHubOption("PROXY", Sys.getenv("HTTP_PROXY")), I can finally connect to the hubs!

Thanks Lori for your help and fixing this so swifty.

P.S.: ideally the hubs could automatically/directly take the PROXY option from the value of the HTTP_PROXY variable, which is pretty standard.

P.P.S: Everything works, but I still get this strange-looking message btw:

Assuming valid proxy connection through 'NULLNULLNULLNULLlist(proxy = "http://my.proxy.com", proxyport = XXXX, proxyauth = 1)NULLNULL'
ADD REPLY
1
Entering edit mode

Yes. Sorry. I was leaving for vacation the day it was reported. I'll fix the message to print more cleanly shortly.

ADD REPLY
1
Entering edit mode

I just pushed changes up to fix the proxy ugly message. They should get picked up by builder tonight and propagate to users tomorrow afternoon. Cheers and thanks for the bug report!

ADD REPLY
0
Entering edit mode

Hi, I am experiencing the same issue with a clean install of version 1.8.0 of ExperimentHub, in R version 3.5.2. I am also behind a company firewall, and followed along with your answers below but am still experiencing the same issue. I was able to run your experiment

library(ExperimentHub) library(httr) setExperimentHubOption("PROXY", "http://my.proxy.com:8080") proxy = getExperimentHubOption("PROXY") GET("https://experimenthub.bioconductor.org/metadata/experimenthub.sqlite3", proxy=proxy)

and got a positive response as the previous user. Would I need to update R to 4.0 and also update ExperimentHub to the most current version to fix the problem?

Thanks, Liyang

ADD REPLY
1
Entering edit mode

Unfortunately yes. We cannot update older versions of the package. The fix is in Bioc 3.11 which minimally uses R 4.0.

ADD REPLY
0
Entering edit mode

Thanks for your response!

ADD REPLY

Login before adding your answer.

Traffic: 583 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6