Hey,

Everytime I try to load a resource from annotationhub (following this guide), I get this error:

downloading from ‘https://annotationhub.bioconductor.org/fetch/52446’
    ‘https://annotationhub.bioconductor.org/fetch/52447’
retrieving 2 resources
Error: failed to load 'AnnotationHub' resource
  name: AH47004
  title: clinvar.vcf.gz from dbSNP, GRCh37 assembly
  reason: 2 resources failed to download
In addition: There were 50 or more warnings (use warnings() to see the first 50)
<font face="sans-serif, Arial, Verdana, Trebuchet MS">
</font>Warning messages:
1: In curl::curl_fetch_disk(url, x$path, handle = handle) :
  progress callback must return boolean

With the error message repeated for 50 times. Any idea what the problem could be?

Thanks

The first part of the answer is that the warnings are coming from the currently available version of the httr package, which has a bug. Update to the version available in github

biocLite("hadley/httr")

(this might require first installing 'devtools', biocLite("devtools')).

The attempt to access the resource will then return more informatively

> hub[['AH47004']]
require("VariantAnnotation")
downloading from 'https://annotationhub.bioconductor.org/fetch/52446'
    'https://annotationhub.bioconductor.org/fetch/52447'
retrieving 2 resources
Error: failed to load 'AnnotationHub' resource
  name: AH47004
  title: clinvar.vcf.gz from dbSNP, GRCh37 assembly
  reason: 2 resources failed to download
In addition: Warning messages:
1: download failed
  hub path: 'https://annotationhub.bioconductor.org/fetch/52446'
  cache path: '/home/mtmorgan/.AnnotationHub/52446'
  reason: Access denied to remote resource 
2: download failed
  hub path: 'https://annotationhub.bioconductor.org/fetch/52447'
  cache path: '/home/mtmorgan/.AnnotationHub/52447'
  reason: Access denied to remote resource 

AnnotationHub forwards the request to the original source

> hub['AH47004']$sourceurl
[1] "ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/clinical_vcf_set/clinvar.vcf.gz"

and manually trying to visit that url, e.g., in a web browser, tells us that the resource is no longer available. 

The workflow could be updated with other AnnotationHub resources. For instance

> mcols(query(hub, "clinvar.vcf"))[,"sourceurl", drop=FALSE]
DataFrame with 3 rows and 1 column
                                                                                                 sourceurl
                                                                                               <character>
AH47004                ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/clinical_vcf_set/clinvar.vcf.gz
AH47022 ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b142_GRCh37p13/VCF/clinical_vcf_set/clinvar.vcf.gz
AH47030    ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b142_GRCh38/VCF/clinical_vcf_set/clinvar.vcf.gz

The latter two resources are still available. We will investigate whether the first is permanently no longer available, and if so update the hub to mark these records as no longer available, and will update the workflow to use alternative resources. Thanks for the heads up; we would like this process of identifying old resources to be more automatic.

Hey Martin, thanks for the answer!

But now I have a different problem - when running:

AnnotationHub()[['AH47030']]

I get this error:

snapshotDate(): 2015-11-19
downloading from ‘https://annotationhub.bioconductor.org/fetch/52498’
    ‘https://annotationhub.bioconductor.org/fetch/52499’
retrieving 2 resources
Error: failed to load 'AnnotationHub' resource
  name: AH47030
  title: clinvar.vcf.gz from dbSNP, GRCh38_b142 assembly
  reason: 2 resources failed to download
In addition: There were 15 warnings (use warnings() to see them)

Followed by:

Warning messages:
1: 'AnnotationHub' database may not be current
  database: ‘C:/Users/ElhananBY/Documents/AppData/.AnnotationHub/annotationhub.sqlite3’
  reason: Problem with the SSL CA cert (path? access rights?)
2: In curl::curl_fetch_disk(url, x$path, handle = handle) :
  progress callback must return boolean
...
7: In curl::curl_fetch_disk(url, x$path, handle = handle) :
  progress callback must return boolean
8: download failed
  hub path: ‘https://annotationhub.bioconductor.org/fetch/52498’
  cache path: ‘C:/Users/ElhananBY/Documents/AppData/.AnnotationHub/52498’
  reason: Problem with the SSL CA cert (path? access rights?)
9: In curl::curl_fetch_disk(url, x$path, handle = handle) :
  progress callback must return boolean
...
14: In curl::curl_fetch_disk(url, x$path, handle = handle) :
  progress callback must return boolean
15: download failed
  hub path: ‘https://annotationhub.bioconductor.org/fetch/52499’
  cache path: ‘C:/Users/ElhananBY/Documents/AppData/.AnnotationHub/52499’
  reason: Problem with the SSL CA cert (path? access rights?)

Also, when trying to insert AnnotationHub() to a variable:

> hub = AnnotationHub()
Warning message:
'AnnotationHub' database may not be current
  database: ‘C:/Users/ElhananBY/Documents/AppData/.AnnotationHub/annotationhub.sqlite3’
  reason: Problem with the SSL CA cert (path? access rights?) 

Further, the last two sources don't work for me:

AH47022 ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b142_GRCh37p13/VCF/clinical_vcf_set/clinvar.vcf.gz
AH47030    ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b142_GRCh38/VCF/clinical_vcf_set/clinvar.vcf.gz

The address seems to be wrong - it shouldn't contain "clinical_vcf_set", but rather just be:

ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b142_GRCh38/VCF/clinvar-latest.vcf.gz

So now I just manually downloaded the vcf.gz file and used readvcf to input it, which seems to work.

Thanks