Illumina Expression analysis from iDAT files
2
2
Entering edit mode
@sean-davis-490
Last seen 3 months ago
United States

Hi, all.

I have a set of iDAT files from illumina expression arrays and have been looking around for a workflow that begins with the iDAT files. Illuminaio can read them and beadarray and lumi packages (to name a couple) can deal with illumina data normalization and processing, but I didn't find a bridge between the illuminaio output and further processing. I very well may have missed something, but I thought I would check.

Thanks, Sean
beadarray lumi illuminaio idat • 6.2k views
ADD COMMENT •
0
Entering edit mode

I'm feeling like a time traveler jumping into a discussion that started 6.9 years ago, but here I go: I'm experiencing this exact problem, and can't find any solution either. Did I miss something?

I hope you are all having a nice day!

Best, Julia

ADD REPLY •
0
Entering edit mode

See the read.idat() function in the limma package:

library(limma)
?read.idat

or read later threads on the same topic:

One way to find these IDAT posts is to click on TAGS and search for "idat".

ADD REPLY •
1
Entering edit mode
Mike Smith ★ 6.6k
@mike-smith
Last seen 9 hours ago
EMBL Heidelberg
Hi Sean, The short answer is that you are correct, there isn't currently a quick and easy way to read IDAT files into R and then feed the data to downstream processing packages. The inclusion of the expression IDAT reading routines in illuminaio is still fairly new compared to the support for other platforms. My hope is for downstream packages to use the reading routines from illuminaio directly, in a similar fashion to crlmm and minfi. I'm currently working on implementing this for beadarray. As a short fix you can use illuminaio to generate a text file similar to the output from GenomeStudio, but this isn't a very satisfactory solution. Let me know if you run into any difficulties, the expression reading code has not been that widely used. Cheers, Mike Smith On 26 November 2013 14:33, Sean Davis <sdavis2@mail.nih.gov> wrote: > Hi, all. > > I have a set of iDAT files from illumina expression arrays and have been > looking around for a workflow that begins with the iDAT files. Illuminaio > can read them and beadarray and lumi packages (to name a couple) can deal > with illumina data normalization and processing, but I didn't find a bridge > between the illuminaio output and further processing. I very well may have > missed something, but I thought I would check. > > Thanks, > Sean > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > -- Mike Smith Research Associate Statistics & Computational Biology Laboratory Cambridge University [[alternative HTML version deleted]]
ADD COMMENT •
0
Entering edit mode
Thanks, Mike, for the confirmation and for the continued work on this stuff. I wrote a quick little function that uses illuminaio readIDAT and generates a LumiBatch from the lumi package. The function doesn't have any bells-and-whistles, but it will suffice for me right now. Sean On Tue, Nov 26, 2013 at 10:07 AM, Mike Smith <grimbough@gmail.com> wrote: > Hi Sean, > > The short answer is that you are correct, there isn't currently a quick and > easy way to read IDAT files into R and then feed the data to downstream > processing packages. The inclusion of the expression IDAT reading routines > in illuminaio is still fairly new compared to the support for other > platforms. > > My hope is for downstream packages to use the reading routines from > illuminaio directly, in a similar fashion to crlmm and minfi. I'm > currently working on implementing this for beadarray. As a short fix you > can use illuminaio to generate a text file similar to the output from > GenomeStudio, but this isn't a very satisfactory solution. > > Let me know if you run into any difficulties, the expression reading code > has not been that widely used. > > Cheers, > Mike Smith > > > On 26 November 2013 14:33, Sean Davis <sdavis2@mail.nih.gov> wrote: > > > Hi, all. > > > > I have a set of iDAT files from illumina expression arrays and have been > > looking around for a workflow that begins with the iDAT files. > Illuminaio > > can read them and beadarray and lumi packages (to name a couple) can deal > > with illumina data normalization and processing, but I didn't find a > bridge > > between the illuminaio output and further processing. I very well may > have > > missed something, but I thought I would check. > > > > Thanks, > > Sean > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: > > http://news.gmane.org/gmane.science.biology.informatics.conductor > > > > > > -- > Mike Smith > Research Associate > Statistics & Computational Biology Laboratory > Cambridge University > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > [[alternative HTML version deleted]]
ADD REPLY •
0
Entering edit mode

Dear Sean and Mike, I have same question as Sean asked. As I know, the beadarray package in R allow us the do the analysis from bead-level data. This package require the bead ID, intensity and position of bead. The output of readIDAT include quant file ( bead ID, mean and sd of ntensity) but there is no position of bead. In addition it each sample seperated by green and red chanel. Mike Smith do you know to handle this gap to use intensity file by beadarray package in R? thank you very much

ADD REPLY •
0
Entering edit mode
@gordon-smyth
Last seen 2 hours ago
WEHI, Melbourne, Australia

Dear Sean and Mike,

We (actually Matt Ritchie) have written a nice little function that uses readIDAT() to read the IDAT files and assembles the pieces into a data object ready for limma analysis. The output is an EList object, but it could be an ExpressionSet instead.

It has been suggested that this function could be added to the limma package, but I wondered whether the illuminaio package would be the best home.

What would you like us to do?

Best wishes
Gordon

Later

Following the discussions here, we added the function read.idat() to the limma package. It builds upon illuminaio::readIDAT and provides a quick and easy way to progress from IDAT files to downstream analysis. It is our recommended pipeline for Illumina arrays, see
ADD COMMENT •
0
Entering edit mode
Hi Gordon, Matt has kindly already sent me a copy of his script for creating the EList object. Combined with Sean's code for generating a LumiBatch and the script I'm putting together for beadarray I think we have most of the downstream expression analysis packages covered. My initial hope was that individual packages would import illuminaio (I like the elegance of having a single function in the library) and modify their data input routines to take both the text and IDAT formats. I'll discuss it with Kasper as illuminaio is ultimately his package. Cheers, Mike On 28 November 2013 02:20, Gordon K Smyth <smyth@wehi.edu.au> wrote: > Dear Sean and Mike, > > We (actually Matt Ritchie) have written a nice little function that uses > readIDAT() to read the IDAT files and assembles the pieces into a data > object ready for limma analysis. The output is an EList object, but it > could be an ExpressionSet instead. > > It has been suggested that this function could be added to the limma > package, but I wondered whether the illuminaio package would be the best > home. > > What would you like us to do? > > Best wishes > Gordon > > Date: Tue, 26 Nov 2013 09:33:17 -0500 >> From: Sean Davis <sdavis2@mail.nih.gov> >> To: "bioconductor@r-project.org" <bioconductor@r-project.org> >> Subject: [BioC] Illumina Expression analysis from iDAT files >> >> Hi, all. >> >> I have a set of iDAT files from illumina expression arrays and have been >> looking around for a workflow that begins with the iDAT files. Illuminaio >> can read them and beadarray and lumi packages (to name a couple) can deal >> with illumina data normalization and processing, but I didn't find a >> bridge >> between the illuminaio output and further processing. I very well may >> have >> missed something, but I thought I would check. >> >> Thanks, >> Sean >> > > ______________________________________________________________________ > The information in this email is confidential and inte...{{dropped:18}}
ADD REPLY •
0
Entering edit mode
I agree here that having packages maintain their own routines probably makes sense. What I am not clear on yet is if we have all the annotation to account for the various control probes. Does anyone know? Sean Hi Gordon, Matt has kindly already sent me a copy of his script for creating the EList object. Combined with Sean's code for generating a LumiBatch and the script I'm putting together for beadarray I think we have most of the downstream expression analysis packages covered. My initial hope was that individual packages would import illuminaio (I like the elegance of having a single function in the library) and modify their data input routines to take both the text and IDAT formats. I'll discuss it with Kasper as illuminaio is ultimately his package. Cheers, Mike On 28 November 2013 02:20, Gordon K Smyth <smyth@wehi.edu.au> wrote: > Dear Sean and Mike, > > We (actually Matt Ritchie) have written a nice little function that uses > readIDAT() to read the IDAT files and assembles the pieces into a data > object ready for limma analysis. The output is an EList object, but it > could be an ExpressionSet instead. > > It has been suggested that this function could be added to the limma > package, but I wondered whether the illuminaio package would be the best > home. > > What would you like us to do? > > Best wishes > Gordon > > Date: Tue, 26 Nov 2013 09:33:17 -0500 >> From: Sean Davis <sdavis2@mail.nih.gov> >> To: "bioconductor@r-project.org" <bioconductor@r-project.org> >> Subject: [BioC] Illumina Expression analysis from iDAT files >> >> Hi, all. >> >> I have a set of iDAT files from illumina expression arrays and have been >> looking around for a workflow that begins with the iDAT files. Illuminaio >> can read them and beadarray and lumi packages (to name a couple) can deal >> with illumina data normalization and processing, but I didn't find a >> bridge >> between the illuminaio output and further processing. I very well may >> have >> missed something, but I thought I would check. >> >> Thanks, >> Sean >> > > ______________________________________________________________________ > The information in this email is confidential and inte...{{dropped:12}}
ADD REPLY •
0
Entering edit mode
If you're using Mark Dunning's package, you should... library(illuminaHumanv4.db) conn <- illuminaHumanv4_dbconn() sql <- 'SELECT ReporterGroupName, COUNT(ArrayAddress) AS probes FROM ExtraInfo GROUP BY ReporterGroupName' dbGetQuery(conn, sql) ## data frame with 100 rows and 2 columns ## ReporterGroupName probes ## <character> <integer> ## 1 NA 47224 ## 2 ERCC-00002 1 ## 3 ERCC-00003 1 ## 4 ERCC-00004 1 ## 5 ERCC-00009 1 ## ... ... ... ## 96 cy3_hyb,low_stringency_hyb 4 ## 97 housekeeping 7 ## 98 labeling 2 ## 99 low_stringency_hyb 4 ## 100 negative 770 Whether the resulting mapped probes get put into the right place in a given data structure is another matter... *He that would live in peace and at ease, * *Must not speak all he knows, nor judge all he sees.* Benjamin Franklin, Poor Richard's Almanack<http: archive.org="" details="" poorrichardsalma00franrich=""> On Thu, Nov 28, 2013 at 1:53 PM, Sean Davis <sdavis2@mail.nih.gov> wrote: > I agree here that having packages maintain their own routines probably > makes sense. What I am not clear on yet is if we have all the annotation > to account for the various control probes. Does anyone know? > > Sean > Hi Gordon, > > Matt has kindly already sent me a copy of his script for creating the EList > object. Combined with Sean's code for generating a LumiBatch and the > script I'm putting together for beadarray I think we have most of the > downstream expression analysis packages covered. My initial hope was that > individual packages would import illuminaio (I like the elegance of having > a single function in the library) and modify their data input routines to > take both the text and IDAT formats. I'll discuss it with Kasper as > illuminaio is ultimately his package. > > Cheers, > > Mike > > > On 28 November 2013 02:20, Gordon K Smyth <smyth@wehi.edu.au> wrote: > > > Dear Sean and Mike, > > > > We (actually Matt Ritchie) have written a nice little function that uses > > readIDAT() to read the IDAT files and assembles the pieces into a data > > object ready for limma analysis. The output is an EList object, but it > > could be an ExpressionSet instead. > > > > It has been suggested that this function could be added to the limma > > package, but I wondered whether the illuminaio package would be the best > > home. > > > > What would you like us to do? > > > > Best wishes > > Gordon > > > > Date: Tue, 26 Nov 2013 09:33:17 -0500 > >> From: Sean Davis <sdavis2@mail.nih.gov> > >> To: "bioconductor@r-project.org" <bioconductor@r-project.org> > >> Subject: [BioC] Illumina Expression analysis from iDAT files > >> > >> Hi, all. > >> > >> I have a set of iDAT files from illumina expression arrays and have been > >> looking around for a workflow that begins with the iDAT files. > Illuminaio > >> can read them and beadarray and lumi packages (to name a couple) can > deal > >> with illumina data normalization and processing, but I didn't find a > >> bridge > >> between the illuminaio output and further processing. I very well may > >> have > >> missed something, but I thought I would check. > >> > >> Thanks, > >> Sean > >> > > > > ______________________________________________________________________ > > The information in this email is confidential and inte...{{dropped:12}} > > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > [[alternative HTML version deleted]]
ADD REPLY •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 651 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6