Using RCurl and XML to access an API
2
0
Entering edit mode
Aaron Wolen ▴ 20
@aaron-wolen-3682
Last seen 9.6 years ago
I'd like to access an API from within R but I'm not sure how to do so. The API uses HTTP and supports both POST and PUT methods. I'm able to access it successfully from the terminal using the following curl command: curl --data @genes.xml http://api-url Where genes.xml is a local file containing the genes I want to submit and http://api-url is the API's address. However, I'm not sure how to replicate this command in R. The request payload and response payload are both XML streams, so I'm fairly certain this will require a combination of the RCurl and XML packages, but I've had no luck getting RCurl to work. I'd really appreciate some help with this. Thanks, Aaron [[alternative HTML version deleted]]
• 1.5k views
ADD COMMENT
0
Entering edit mode
@martin-morgan-1513
Last seen 5 days ago
United States
Hi Aaron -- Aaron Wolen wrote: > I'd like to access an API from within R but I'm not sure how to do so. The > API uses HTTP and supports both POST and PUT methods. I'm able to access it > successfully from the terminal using the following curl command: > curl --data @genes.xml http://api-url > > Where genes.xml is a local file containing the genes I want to submit and > http://api-url is the API's address. However, I'm not sure how to replicate > this command in R. The request payload and response payload are both XML > streams, so I'm fairly certain this will require a combination of the RCurl > and XML packages, but I've had no luck getting RCurl to work. I'd really > appreciate some help with this. To get more help you'll probably have to provide more information about what specifically is at the other end of the curl request, especially a reproducible (as in, a real url and appropriate data argument, and an indication of what you'd like back, e.g., as from the command line) example. I think you want to end up using getURL("http://api-url", ...) where ... are arguments that capture the command line --data @genes.xml. The place to look for ... is in the 'man' page for curl_easy_setopt, where an option like CURLOPT_READFUNCTION becomes a named argument 'readfunction'. I'd guess perhaps reader <- function(file) { function() paste(readLines(file), collapse="\n") } res <- getURL("http://api-url", readfunction=reader("genes.xml")) If the return value is XML or HTML, you might then xml <- xmlTreeParse(res, useInternal=TRUE) and the query xml with the xpath language, described here http://www.w3.org/TR/xpath, especially section 2.5, abbreviated syntax. Hope that provides some help; if you provide something to explore against, you might get some more explicit help. Martin > Thanks, > Aaron > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD COMMENT
0
Entering edit mode
Thanks for the the responses, Martin and Hervé. I apologize for the sparse details. I didn't provide the actual URL because I didn't have the express permission of the API's authors to share it publicly and it's still in private beta. I have since communicated with the authors and was told it would be okay to share details with the BioConductor community. The API is for a suite of bioinformatic web applications available at http://toppgene.cchmc.org. Details about the API can be found at http://toppgene.cchmc.org/wiki. Anyway, Martin's suggestion was definitely a step in the right direction. I was able to successfully make a call to the API with the code he provided. However, this returned the following error message: The specified HTTP method is not allowed for the requested resource (HTTP method GET is not supported by this URL). How I could I make a similar call using the PUT or POST method instead of GET? As per Hervé Pagès suggestion I perused the source code for biomaRt and found it uses the postForm function. Perhaps that is what should to be used with the ToppGene API. On Sun, Sep 13, 2009 at 3:37 PM, Martin Morgan <mtmorgan@fhcrc.org> wrote: > Hi Aaron -- > > Aaron Wolen wrote: > > I'd like to access an API from within R but I'm not sure how to do so. > The > > API uses HTTP and supports both POST and PUT methods. I'm able to access > it > > successfully from the terminal using the following curl command: > > curl --data @genes.xml http://api-url > > > > > Where genes.xml is a local file containing the genes I want to submit and > > http://api-url is the API's address. However, I'm not sure how to > replicate > > this command in R. The request payload and response payload are both XML > > streams, so I'm fairly certain this will require a combination of the > RCurl > > and XML packages, but I've had no luck getting RCurl to work. I'd really > > appreciate some help with this. > > To get more help you'll probably have to provide more information about > what specifically is at the other end of the curl request, especially a > reproducible (as in, a real url and appropriate data argument, and an > indication of what you'd like back, e.g., as from the command line) > example. > > I think you want to end up using > > getURL("http://api-url", ...) > > where ... are arguments that capture the command line --data @genes.xml. > The place to look for ... is in the 'man' page for curl_easy_setopt, > where an option like CURLOPT_READFUNCTION becomes a named argument > 'readfunction'. I'd guess perhaps > > reader <- function(file) { > function() paste(readLines(file), collapse="\n") > } > > res <- getURL("http://api-url", readfunction=reader("genes.xml")) > > If the return value is XML or HTML, you might then > > xml <- xmlTreeParse(res, useInternal=TRUE) > > and the query xml with the xpath language, described here > http://www.w3.org/TR/xpath, especially section 2.5, abbreviated syntax. > > Hope that provides some help; if you provide something to explore > against, you might get some more explicit help. > > Martin > > > Thanks, > > Aaron > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > > [[alternative HTML version deleted]]
ADD REPLY
0
Entering edit mode
@herve-pages-1542
Last seen 1 day ago
Seattle, WA, United States
Hi Aaron, This kind of operations (retrieving XML docs from the web and parsing their content) is at the heart of some Bioconductor packages like biomaRt or rtracklayer: http://bioconductor.org/packages/release/bioc/html/biomaRt.html http://bioconductor.org/packages/release/bioc/html/rtracklayer.html Maybe you could have a look at how things are done in those packages so you get an idea of how that works. Cheers, H. Aaron Wolen wrote: > I'd like to access an API from within R but I'm not sure how to do so. The > API uses HTTP and supports both POST and PUT methods. I'm able to access it > successfully from the terminal using the following curl command: > curl --data @genes.xml http://api-url > > Where genes.xml is a local file containing the genes I want to submit and > http://api-url is the API's address. However, I'm not sure how to replicate > this command in R. The request payload and response payload are both XML > streams, so I'm fairly certain this will require a combination of the RCurl > and XML packages, but I've had no luck getting RCurl to work. I'd really > appreciate some help with this. > > Thanks, > Aaron > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M2-B876 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpages at fhcrc.org Phone: (206) 667-5791 Fax: (206) 667-1319
ADD COMMENT

Login before adding your answer.

Traffic: 979 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6