problems with readIllumina() for many BeadChips
2
0
Entering edit mode
Ina Hoeschele ▴ 620
@ina-hoeschele-2992
Last seen 3.3 years ago
United States
Hi Mark et al., I still have issues reading data from multiple BeadChips using readIllumina (same as below). My BeadChip is HumanHT-12 v4 Expression BeadChip My data are in 8 directories, corresponding to 8 chips, containing only Beadscan output (no other txt files). In each directory, there is one "SQL server compact edition database file", and for each of the sections A through L, there is a Microsoft Office Excel comma separated values file, a Text document, an IDAT file, a LOCS file, a TIFF image and an XML document. There are NO other files except for 3 files whose names end in _fiducial #1_Red_center, _fiducial #2_Red_center, and _fiducial #3_Red_center. I have been able to read the data into R with readIllumina using this code: setwd("C:/PHS_MESA_Epigenomics/5655508006") BLData = readIllumina(useImages=FALSE, illuminaAnnotation="Humanv4", sectionNames=c("5655508006_A","5655508006_B","5655508006_C","565550800 6_D","5655508006_E","5655508006_F","5655508006_G","5655508006_H","5655 508006_I","5655508006_J","5655508006_K","5655508006_L")) Question 1: if I use the same code but omit sectionNames, then it does not read the data. I will have to deal with hundred(s) of BeadChips and so specifying sectionNames for each chip is awkward. Why does nit not automatically find all 12 txt files? Moreover, currently I read the data from each directory (= BeadChip) separately - that would be hundred(s) read statements - is there a way to get readIllumina to read all txt files in ALL subdirectories with a single call to readIllumina? Question 2: this is about the arguments to readIllumina. The arguments as described in the beadarray manual are: dir, useImages, illuminaAnnotation, sectionNames and metricsFile. Then it states "other argumdents to pass when reading bead-level text files". What are these other arguments, where are they described, are they important (does one have to look at the source code to find out??) In the old version of the beadarray package, I used additional arguments that seem to no longer work and are no longer described, for example textType, singleChannel, normalizeMethod, backgroundMethod. When I include some of these old arguments, readIllumina fails. Here is the error message that I get when I try to read the data in one directory without specifying sectionNames: > setwd("C:/PHS_MESA_Epigenomics/5655508006") > BLData = readIllumina(useImages=FALSE, illuminaAnnotation="Humanv4") Error in file(file, "rt") : cannot open the connection In addition: Warning message: In file(file, "rt") : cannot open file 'NA/NA': No such file or directory > sessionInfo() R version 2.12.1 (2010-12-16) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] grid stats graphics grDevices utils datasets methods [8] base other attached packages: [1] preprocessCore_1.12.0 geneplotter_1.28.0 lattice_0.19-13 [4] annotate_1.28.0 AnnotationDbi_1.12.0 RColorBrewer_1.0-2 [7] gplots_2.8.0 caTools_1.11 bitops_1.0-4.1 [10] gdata_2.8.1 gtools_2.6.2 beadarray_2.0.3 [13] Biobase_2.10.0 loaded via a namespace (and not attached): [1] DBI_0.2-5 limma_3.6.9 RSQLite_0.9-4 xtable_1.5-6 Many thanks for your time and help. Ina ----- Original Message ----- From: "Mark Dunning" <mark.dunning@gmail.com> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> Cc: bioconductor at stat.math.ethz.ch Sent: Monday, December 20, 2010 6:02:12 AM Subject: Re: [BioC] problems with readIllumina() Hi Ina, beadarray has changed a bit in the past two years. The readIllumina function should be roughly the same. However it expects the contents of the directory to be the output of Illumina's scanning software. For instance, if there are files with the extension .txt other than the bead-level files it could get confused easily. You can use the sectionNames argument to readIllumina to specify which files to read in. Regards, Mark On Wed, Dec 15, 2010 at 11:25 PM, Ina Hoeschele <inah at="" vbi.vt.edu=""> wrote: > Hi Karl and others, > ? I am getting the same error message with the readIllumina function and the new beadarray package: > >> setwd("C:/PHS_MESA_Epigenomics/5655508006") >> BLData.normexp = readIllumina(singleChannel=TRUE, useImages=FALSE, illuminaAnnotation="Humanv4", normalizeMethod="none", backgroundMethod="normexp") > Error in file(file, "rt") : cannot open the connection > In addition: Warning message: > In file(file, "rt") : cannot open file 'NA/NA': No such file or directory > >> sessionInfo() > R version 2.12.0 (2010-10-15) > Platform: i386-pc-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=English_United States.1252 > [2] LC_CTYPE=English_United States.1252 > [3] LC_MONETARY=English_United States.1252 > [4] LC_NUMERIC=C > [5] LC_TIME=English_United States.1252 > > attached base packages: > [1] stats ? ? graphics ?grDevices utils ? ? datasets ?methods ? base > > other attached packages: > [1] beadarray_2.0.2 Biobase_2.10.0 > > loaded via a namespace (and not attached): > [1] limma_3.6.4 ?tools_2.12.0 > > > When I was working with similar data 2 years ago I had to specify the array names like this: > readIllumina(textType=".txt", > ? ? ? ?arrayNames = c("4207121082_A_1","4207121082_A_2","4207121082_B_1", > ? ? ? ?"4207121082_B_2","4207121082_C_1","4207121082_C_2","4207121082_D_1", > ? ? ? ?"4207121082_D_2","4207121082_E_1","4207121082_E_2","4207121082_F_1", > ? ? ? ?"4207121082_F_2"), singleChannel=TRUE, useImages=FALSE, > ? ? ? ?normalizeMethod="none", backgroundMethod="normexp") > > Is this because I do not have a targets file? Do I get the error message because I did not specify the array names for 5655508006? > > Above, I am trying to read the data on a single chip (5655508006) - how can I read data on multiple chips (that are in different sub- directories (5655508006, 5655508014, etc.)) with one call to readIllumina? > > Many thanks for any help. > > Ina > > ----- Original Message ----- > From: "karl koechert" <karl.koechert at="" mdc-berlin.de=""> > To: bioconductor at stat.math.ethz.ch > Sent: Tuesday, November 2, 2010 10:13:15 AM > Subject: [BioC] problems with readIllumina() > > Hey, > > I've been using beedarray quite often, but now with the new version I'm running in some error when trying to reading in the bead level data with readIllumina(). Does anyone know why this is happening or had a similar experience? What is meant with file 'NA/NA'? > > Thanks for any help! > Karl > > see below: > >> bld=readIllumina(arrayNames=targets$Array, textType=".txt", backgroundMethod="none", annoPkg="Humanv4", useImages=FALSE) > Fehler in file(file, "rt") : kann Verbindung nicht ?ffnen //error: can't open connection > Zus?tzlich: Warnmeldung: //additional: error > In file(file, "rt") : > ?kann Datei 'NA/NA' nicht ?ffnen: No such file or directory // can't open file 'NA/NA' >> bld=readIllumina(useImages = FALSE, illuminaAnnotation = "Humanv4") > Fehler in file(file, "rt") : kann Verbindung nicht ?ffnen //error: can't open connection > Zus?tzlich: Warnmeldung: > In file(file, "rt") ://additional: error > ?kann Datei 'NA/NA' nicht ?ffnen: No such file or directory // can't open file 'NA/NA' >> sessionInfo() > R version 2.12.0 (2010-10-15) > Platform: i386-pc-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=German_Germany.1252 ?LC_CTYPE=German_Germany.1252 ? ?LC_MONETARY=German_Germany.1252 > [4] LC_NUMERIC=C ? ? ? ? ? ? ? ? ? ?LC_TIME=German_Germany.1252 > > attached base packages: > [1] stats ? ? graphics ?grDevices utils ? ? datasets ?methods ? base > > other attached packages: > [1] illuminaHumanv4BeadID.db_1.8.0 org.Hs.eg.db_2.4.6 ? ? ? ? ? ? RSQLite_0.9-2 > [4] DBI_0.2-5 ? ? ? ? ? ? ? ? ? ? ?AnnotationDbi_1.12.0 ? ? ? ? ? limma_3.6.6 > [7] beadarray_2.0.1 ? ? ? ? ? ? ? ?Biobase_2.10.0 > > loaded via a namespace (and not attached): > [1] tools_2.12.0 > > > ? ? ? ?[[alternative HTML version deleted]] > > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
beadarray beadarray • 2.1k views
ADD COMMENT •
0
Entering edit mode
Mike Smith ★ 6.6k
@mike-smith
Last seen 1 hour ago
EMBL Heidelberg
Hi Ina, I've removed the offending line of code from the createTargetsFile function, which was causing the errors you've been seeing. Version 2.0.4 should work without requiring the section names to be specified. The directory scanning functionality is something I've been working on a lot in the past week and the development version of the package should be more resilient to having a variety of files in a directory. Out of interest, what data is stored in the .csv files you have? I wasn't aware that BeadScan produced anything with that extension, so if there's useful data in there I may need to make some more allowances in the directory scanning function. As far as reading multiple chips goes, we tend to avoid it as you start to run out of memory very quickly once you call some of the processing functions. Beadarray also only performs outlier removal, intensity normalization etc. within a chip, rather than between them, so there is little advantage in reading everything in in one go. Our approach is generally to list all the directories in a vector and loop through it, or to farm each directory out on a cluster. We then combine them once they've been summarized. The additional arguments to readIllumina mentioned in the help file are passed to functions such as readBeadLevelTextFile allowing you to specify seperators etc. I agree the documentation is lacking (non existant), and we'll try to make it clearer shortly. I hope this is of some help, On 15 Feb 2011 22:06, "Ina Hoeschele" <inah@vbi.vt.edu> wrote: [[alternative HTML version deleted]]
ADD COMMENT •
0
Entering edit mode
thank you very much, Mike. The . csv files contain Illumina code, N (number of beads/bead type I assume), mean GRN, and dev GRN. Ina ----- Original Message ----- From: "Mike Smith" <grimbough@gmail.com> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> Cc: bioconductor at stat.math.ethz.ch Sent: Thursday, February 17, 2011 9:26:07 AM Subject: Re: [BioC] problems with readIllumina() for many BeadChips Hi Ina, I've removed the offending line of code from the createTargetsFile function, which was causing the errors you've been seeing. Version 2.0.4 should work without requiring the section names to be specified. The directory scanning functionality is something I've been working on a lot in the past week and the development version of the package should be more resilient to having a variety of files in a directory. Out of interest, what data is stored in the .csv files you have? I wasn't aware that BeadScan produced anything with that extension, so if there's useful data in there I may need to make some more allowances in the directory scanning function. As far as reading multiple chips goes, we tend to avoid it as you start to run out of memory very quickly once you call some of the processing functions. Beadarray also only performs outlier removal, intensity normalization etc. within a chip, rather than between them, so there is little advantage in reading everything in in one go. Our approach is generally to list all the directories in a vector and loop through it, or to farm each directory out on a cluster. We then combine them once they've been summarized. The additional arguments to readIllumina mentioned in the help file are passed to functions such as readBeadLevelTextFile allowing you to specify seperators etc. I agree the documentation is lacking (non existant), and we'll try to make it clearer shortly. I hope this is of some help, On 15 Feb 2011 22:06, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote:
ADD REPLY •
0
Entering edit mode
Mike et al., I am sorry but how do I install the development verision 2.0.4 of beadarray? Thanks, Ina ----- Original Message ----- From: "Ina Hoeschele" <inah@vbi.vt.edu> To: "Mike Smith" <grimbough at="" gmail.com=""> Cc: bioconductor at stat.math.ethz.ch Sent: Thursday, February 17, 2011 10:11:32 AM Subject: Re: [BioC] problems with readIllumina() for many BeadChips thank you very much, Mike. The . csv files contain Illumina code, N (number of beads/bead type I assume), mean GRN, and dev GRN. Ina ----- Original Message ----- From: "Mike Smith" <grimbough@gmail.com> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> Cc: bioconductor at stat.math.ethz.ch Sent: Thursday, February 17, 2011 9:26:07 AM Subject: Re: [BioC] problems with readIllumina() for many BeadChips Hi Ina, I've removed the offending line of code from the createTargetsFile function, which was causing the errors you've been seeing. Version 2.0.4 should work without requiring the section names to be specified. The directory scanning functionality is something I've been working on a lot in the past week and the development version of the package should be more resilient to having a variety of files in a directory. Out of interest, what data is stored in the .csv files you have? I wasn't aware that BeadScan produced anything with that extension, so if there's useful data in there I may need to make some more allowances in the directory scanning function. As far as reading multiple chips goes, we tend to avoid it as you start to run out of memory very quickly once you call some of the processing functions. Beadarray also only performs outlier removal, intensity normalization etc. within a chip, rather than between them, so there is little advantage in reading everything in in one go. Our approach is generally to list all the directories in a vector and loop through it, or to farm each directory out on a cluster. We then combine them once they've been summarized. The additional arguments to readIllumina mentioned in the help file are passed to functions such as readBeadLevelTextFile allowing you to specify seperators etc. I agree the documentation is lacking (non existant), and we'll try to make it clearer shortly. I hope this is of some help, On 15 Feb 2011 22:06, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote: _______________________________________________ Bioconductor mailing list Bioconductor at r-project.org https://stat.ethz.ch/mailman/listinfo/bioconductor Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLY •
0
Entering edit mode
Mike Smith ★ 6.6k
@mike-smith
Last seen 1 hour ago
EMBL Heidelberg
Hi Ina, Sorry, I think my message has probably created some confusion. Version 2.0.4 is not the developmental version, its a patched version of the release. You can install it using biocLite("beadarray"). I uploaded the patch yesterday, so it may take a day or so to filter through. If you want to install the developmental version (currently 2.1.11) you normally need to be running a developmental version of R, although there was some discussion about this a few weeks back on this list. This version should be considered experimental and potentially buggy, but as you've seen that can probably be applied to parts of the release package too! Mike Smith PhD Student Computational Biology Group Department of Oncology Cambridge University On 17 Feb 2011 16:52, "Ina Hoeschele" <inah@vbi.vt.edu> wrote: [[alternative HTML version deleted]]
ADD COMMENT •
0
Entering edit mode
Hi Mike et al., first thanks for the help, I can now read and process the bead- level data from multiple (HumanHT-12 v4) Expression BeadChips just fine. Unfortunately I've run into the next hurdle, where I have a hard time finding information in the beadarray vignette or manual. The background corrected intensities in the txt files (expectedly) contain some negative values, but I can only find the log2 transformation. When I worked with the old version of beadarray a couple of years ago I used the background correction method of rma (backgroundMethod="normexp" in readIllumina) to obtain positive intensities prior to transformation, but this option seems no longer available? So how does beadarray deal with the negative intensities? Also, is there any opinion on the order of steps - transform bead- level intensities, then summarize, or summarize and then log2 transform, (either followed by normalization)? Many thanks ... Ina ----- Original Message ----- From: "Mike Smith" <grimbough@gmail.com> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> Cc: bioconductor at stat.math.ethz.ch Sent: Thursday, February 17, 2011 1:59:11 PM Subject: Re: [BioC] problems with readIllumina() for many BeadChips Hi Ina, Sorry, I think my message has probably created some confusion. Version 2.0.4 is not the developmental version, its a patched version of the release. You can install it using biocLite("beadarray"). I uploaded the patch yesterday, so it may take a day or so to filter through. If you want to install the developmental version (currently 2.1.11) you normally need to be running a developmental version of R, although there was some discussion about this a few weeks back on this list. This version should be considered experimental and potentially buggy, but as you've seen that can probably be applied to parts of the release package too! Mike Smith PhD Student Computational Biology Group Department of Oncology Cambridge University On 17 Feb 2011 16:52, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote:
ADD REPLY •
0
Entering edit mode
Hi Mike et al., I still have not had any response to my question about transformations in beadarray that yield non-negative values - does the current version of beadarray still have this capability? Or are the neagtive values just ignored in the bead summary? To make things easier, I have been trying to use the expressionQCPipeline function of bead array, but it keeps giving me the message that plots already exist and are skipped! I've tried different chips and keep getting the same message! Please see below. Any suggestion why this is happening? I cannot find any plots that 'already exist'. Many thanks, Ina > expressionQCPipeline(BLData,transFun=logGreenChannelTransform,qcDir= "QC.Dir[iChip]",plotType=".jpeg",horizontal=TRUE,controlProfile=NULL, + overWrite=FALSE,nSegments=9,outlierFun=illuminaOutlierMethod,tagsToD etect=list(housekeeping="housekeeping",Biotin="phage_lambda_genome", + Hybridisation="phage_lambda_genome:high"),zlim=c(5,7),positiveContro lTags=c("housekeeping","phage_lambda_genome"), + hybridisationTags=c("phage_lambda_genome:low","phage_lambda_genome:m ed","phage_lambda_genome:high"),negativeTag="permuted_negative", + boxplotFun=logGreenChannelTransform,imageplotFun=logGreenChannelTran sform) Making per-array plots for section 1 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Loading required package: hwriter Making per-array plots for section 2 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 3 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 4 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 5 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 6 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 7 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 8 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 9 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 10 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 11 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Making per-array plots for section 12 Positive controls Positive control plot exists. Skipping to next plot Hyb controls Hybridisation control plot exists. Skipping to next plot Outliers Outlier plot exists. Skipping to next plot imageplot Positive control plot exists. Skipping to next plot Creating probe metrics Calculating outlier Metrics 35901 outliers found on the section 33408 outliers found on the section 30521 outliers found on the section 32013 outliers found on the section 32032 outliers found on the section 30939 outliers found on the section 32048 outliers found on the section 30359 outliers found on the section 32026 outliers found on the section 28328 outliers found on the section 28367 outliers found on the section 27180 outliers found on the section > sessionInfo() R version 2.12.1 (2010-12-16) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] grid stats graphics grDevices utils datasets methods [8] base other attached packages: [1] hwriter_1.3 geneplotter_1.28.0 lattice_0.19-13 [4] annotate_1.28.0 AnnotationDbi_1.12.0 RColorBrewer_1.0-2 [7] gplots_2.8.0 caTools_1.11 bitops_1.0-4.1 [10] gdata_2.8.1 gtools_2.6.2 beadarray_2.0.4 [13] Biobase_2.10.0 loaded via a namespace (and not attached): [1] DBI_0.2-5 limma_3.6.9 RSQLite_0.9-4 tools_2.12.1 xtable_1.5-6 > ----- Original Message ----- From: "Ina Hoeschele" <inah@vbi.vt.edu> To: "Mike Smith" <grimbough at="" gmail.com=""> Cc: bioconductor at stat.math.ethz.ch Sent: Sunday, February 20, 2011 9:20:07 PM Subject: Re: [BioC] problems with readIllumina() for many BeadChips Hi Mike et al., first thanks for the help, I can now read and process the bead- level data from multiple (HumanHT-12 v4) Expression BeadChips just fine. Unfortunately I've run into the next hurdle, where I have a hard time finding information in the beadarray vignette or manual. The background corrected intensities in the txt files (expectedly) contain some negative values, but I can only find the log2 transformation. When I worked with the old version of beadarray a couple of years ago I used the background correction method of rma (backgroundMethod="normexp" in readIllumina) to obtain positive intensities prior to transformation, but this option seems no longer available? So how does beadarray deal with the negative intensities? Also, is there any opinion on the order of steps - transform bead- level intensities, then summarize, or summarize and then log2 transform, (either followed by normalization)? Many thanks ... Ina ----- Original Message ----- From: "Mike Smith" <grimbough@gmail.com> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> Cc: bioconductor at stat.math.ethz.ch Sent: Thursday, February 17, 2011 1:59:11 PM Subject: Re: [BioC] problems with readIllumina() for many BeadChips Hi Ina, Sorry, I think my message has probably created some confusion. Version 2.0.4 is not the developmental version, its a patched version of the release. You can install it using biocLite("beadarray"). I uploaded the patch yesterday, so it may take a day or so to filter through. If you want to install the developmental version (currently 2.1.11) you normally need to be running a developmental version of R, although there was some discussion about this a few weeks back on this list. This version should be considered experimental and potentially buggy, but as you've seen that can probably be applied to parts of the release package too! Mike Smith PhD Student Computational Biology Group Department of Oncology Cambridge University On 17 Feb 2011 16:52, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote: _______________________________________________ Bioconductor mailing list Bioconductor at r-project.org https://stat.ethz.ch/mailman/listinfo/bioconductor Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLY •
0
Entering edit mode
Hi, Sorry for not replying sooner. The new version of beadarray was written with flexibility in mind. Many of the previous functions were completely re-written to allow users to specify their own means of correcting, summarising and transforming data. For your particular example, I'm afraid the normexp correction was removed. This was a mistake on our part as we believed it to not be in use. However, the normexp correction function from limma can operate on a vector of intensities and indices of negative controls and It would not be difficult to construct a transformaton function that did the correction and put the results back in the bead-level object using 'insertBeadData'. I will reply with this function later today. Regarding your expressionQCPipeline problem; it should be trying to write plots to a directory called "QC.Dir[iChip]". If you run >dir("QC.Dir[iChip]") what does it return? As you've already seen, the default transformation in beadarray is log2. Any beads with negative intensity will be transformed to 'NA' and not contribute to the summary information for their particular bead-type. However, if you specify the transformation function greenChannelTrransform as the 'transFun' argument in 'summarize'; greenChannelTransform function (BLData, array) { x = getBeadData(BLData, array = array, what = "Grn") } then they will could still be included, although I would guess they would be removed as outliers. By producing summary data on the un-logged scale you would be able to experiment with other transformation schemes that operate on the summary data such as vst [lumi] or neqc [limma]. As far as I know, the benefits of using normexp on the bead-level vs bead summary data have not been explored. Hope this helps. Mark On Thu, Feb 24, 2011 at 9:26 PM, Ina Hoeschele <inah at="" vbi.vt.edu=""> wrote: > Hi Mike et al., > ?I still have not had any response to my question about transformations in beadarray that yield non-negative values - does the current version of beadarray still have this capability? Or are the neagtive values just ignored in the bead summary? > > To make things easier, I have been trying to use the expressionQCPipeline function of bead array, but it keeps giving me the message that plots already exist and are skipped! I've tried different chips and keep getting the same message! Please see below. Any suggestion why this is happening? I cannot find any plots that 'already exist'. > > Many thanks, Ina > >> expressionQCPipeline(BLData,transFun=logGreenChannelTransform,qcDir ="QC.Dir[iChip]",plotType=".jpeg",horizontal=TRUE,controlProfile=NULL, > + overWrite=FALSE,nSegments=9,outlierFun=illuminaOutlierMethod,tagsT oDetect=list(housekeeping="housekeeping",Biotin="phage_lambda_genome", > + Hybridisation="phage_lambda_genome:high"),zlim=c(5,7),positiveCont rolTags=c("housekeeping","phage_lambda_genome"), > + hybridisationTags=c("phage_lambda_genome:low","phage_lambda_genome :med","phage_lambda_genome:high"),negativeTag="permuted_negative", > + boxplotFun=logGreenChannelTransform,imageplotFun=logGreenChannelTr ansform) > Making per-array plots for section 1 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Loading required package: hwriter > Making per-array plots for section 2 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 3 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 4 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 5 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 6 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 7 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 8 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 9 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 10 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 11 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Making per-array plots for section 12 > Positive controls > Positive control plot exists. Skipping to next plot > Hyb controls > Hybridisation control plot exists. Skipping to next plot > Outliers > Outlier plot exists. Skipping to next plot > imageplot > Positive control plot exists. Skipping to next plot > Creating probe metrics > Calculating outlier Metrics > 35901 ?outliers found on the section > 33408 ?outliers found on the section > 30521 ?outliers found on the section > 32013 ?outliers found on the section > 32032 ?outliers found on the section > 30939 ?outliers found on the section > 32048 ?outliers found on the section > 30359 ?outliers found on the section > 32026 ?outliers found on the section > 28328 ?outliers found on the section > 28367 ?outliers found on the section > 27180 ?outliers found on the section >> sessionInfo() > R version 2.12.1 (2010-12-16) > Platform: i386-pc-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=English_United States.1252 > [2] LC_CTYPE=English_United States.1252 > [3] LC_MONETARY=English_United States.1252 > [4] LC_NUMERIC=C > [5] LC_TIME=English_United States.1252 > > attached base packages: > [1] grid ? ? ?stats ? ? graphics ?grDevices utils ? ? datasets ?methods > [8] base > > other attached packages: > ?[1] hwriter_1.3 ? ? ? ? ?geneplotter_1.28.0 ? lattice_0.19-13 > ?[4] annotate_1.28.0 ? ? ?AnnotationDbi_1.12.0 RColorBrewer_1.0-2 > ?[7] gplots_2.8.0 ? ? ? ? caTools_1.11 ? ? ? ? bitops_1.0-4.1 > [10] gdata_2.8.1 ? ? ? ? ?gtools_2.6.2 ? ? ? ? beadarray_2.0.4 > [13] Biobase_2.10.0 > > loaded via a namespace (and not attached): > [1] DBI_0.2-5 ? ? limma_3.6.9 ? RSQLite_0.9-4 tools_2.12.1 ?xtable_1.5-6 >> > > > ----- Original Message ----- > From: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> > To: "Mike Smith" <grimbough at="" gmail.com=""> > Cc: bioconductor at stat.math.ethz.ch > Sent: Sunday, February 20, 2011 9:20:07 PM > Subject: Re: [BioC] problems with readIllumina() for many BeadChips > > Hi Mike et al., > ? first thanks for the help, I can now read and process the bead- level data from multiple (HumanHT-12 v4) Expression BeadChips just fine. Unfortunately I've run into the next hurdle, where I have a hard time finding information in the beadarray vignette or manual. The background corrected intensities in the txt files (expectedly) contain some negative values, but I can only find the log2 transformation. When I worked with the old version of beadarray a couple of years ago I used the background correction method of rma (backgroundMethod="normexp" in readIllumina) to obtain positive intensities prior to transformation, but this option seems no longer available? So how does beadarray deal with the negative intensities? Also, is there any opinion on the order of steps - transform bead- level intensities, then summarize, or summarize and then log2 transform, (either followed by normalization)? > Many thanks ... Ina > > > ----- Original Message ----- > From: "Mike Smith" <grimbough at="" gmail.com=""> > To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> > Cc: bioconductor at stat.math.ethz.ch > Sent: Thursday, February 17, 2011 1:59:11 PM > Subject: Re: [BioC] problems with readIllumina() for many BeadChips > > Hi Ina, > > Sorry, I think my message has probably created some confusion. > > Version 2.0.4 is not the developmental version, its a patched version of the > release. ?You can install it using biocLite("beadarray"). ?I uploaded the > patch yesterday, so it may take a day or so to filter through. > > If you want to install the developmental version (currently 2.1.11) you > normally need to be running a developmental version of R, although there was > some discussion about this a few weeks back on this list. ?This version > should be considered experimental and potentially buggy, but as you've seen > that can probably be applied to parts of the release package too! > > Mike Smith > PhD Student > Computational Biology Group > Department of Oncology > Cambridge University > On 17 Feb 2011 16:52, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote: > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor >
ADD REPLY •
0
Entering edit mode
Hi Ina, You may have noticed the typo in greenChannelTransform. It should of course be: ?greenChannelTransform function (BLData, array) { ? ?x = getBeadData(BLData, array = array, what = "Grn") return(x) } Mark On Fri, Feb 25, 2011 at 8:48 AM, Mark Dunning <mark.dunning at="" gmail.com=""> wrote: > Hi, > > Sorry for not replying sooner. The new version of beadarray was > written with flexibility in mind. Many of the previous functions were > completely re-written to allow users to specify their own means of > correcting, summarising and transforming data. For your particular > example, I'm afraid the normexp correction was removed. This was a > mistake on our part as we believed it to not be in use. However, the > normexp correction function from limma can operate on a vector of > intensities and indices of negative controls and It would not be > difficult to construct a transformaton function that did the > correction and put the results back in the bead-level object using > 'insertBeadData'. I will reply with this function later today. > > Regarding your expressionQCPipeline problem; it should be trying to > write plots to a directory called "QC.Dir[iChip]". If you run > >>dir("QC.Dir[iChip]") > > what does it return? > > As you've already seen, the default transformation in beadarray is > log2. Any beads with negative intensity will be transformed to 'NA' > and not contribute to the summary information for their particular > bead-type. However, if you specify the transformation function > greenChannelTrransform as the 'transFun' argument in 'summarize'; > > ?greenChannelTransform > function (BLData, array) > { > ? ?x = getBeadData(BLData, array = array, what = "Grn") > } > > then they will could still be included, although I would guess they > would be removed as outliers. By producing summary data on the > un-logged scale you would be able to experiment with other > transformation schemes that operate on the summary data such as vst > [lumi] or neqc [limma]. As far as I know, the benefits of using > normexp on the bead-level vs bead summary data have not been explored. > > Hope this helps. > > Mark > > On Thu, Feb 24, 2011 at 9:26 PM, Ina Hoeschele <inah at="" vbi.vt.edu=""> wrote: >> Hi Mike et al., >> ?I still have not had any response to my question about transformations in beadarray that yield non-negative values - does the current version of beadarray still have this capability? Or are the neagtive values just ignored in the bead summary? >> >> To make things easier, I have been trying to use the expressionQCPipeline function of bead array, but it keeps giving me the message that plots already exist and are skipped! I've tried different chips and keep getting the same message! Please see below. Any suggestion why this is happening? I cannot find any plots that 'already exist'. >> >> Many thanks, Ina >> >>> expressionQCPipeline(BLData,transFun=logGreenChannelTransform,qcDi r="QC.Dir[iChip]",plotType=".jpeg",horizontal=TRUE,controlProfile=NULL , >> + overWrite=FALSE,nSegments=9,outlierFun=illuminaOutlierMethod,tags ToDetect=list(housekeeping="housekeeping",Biotin="phage_lambda_genome" , >> + Hybridisation="phage_lambda_genome:high"),zlim=c(5,7),positiveCon trolTags=c("housekeeping","phage_lambda_genome"), >> + hybridisationTags=c("phage_lambda_genome:low","phage_lambda_genom e:med","phage_lambda_genome:high"),negativeTag="permuted_negative", >> + boxplotFun=logGreenChannelTransform,imageplotFun=logGreenChannelT ransform) >> Making per-array plots for section 1 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Loading required package: hwriter >> Making per-array plots for section 2 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 3 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 4 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 5 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 6 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 7 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 8 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 9 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 10 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 11 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Making per-array plots for section 12 >> Positive controls >> Positive control plot exists. Skipping to next plot >> Hyb controls >> Hybridisation control plot exists. Skipping to next plot >> Outliers >> Outlier plot exists. Skipping to next plot >> imageplot >> Positive control plot exists. Skipping to next plot >> Creating probe metrics >> Calculating outlier Metrics >> 35901 ?outliers found on the section >> 33408 ?outliers found on the section >> 30521 ?outliers found on the section >> 32013 ?outliers found on the section >> 32032 ?outliers found on the section >> 30939 ?outliers found on the section >> 32048 ?outliers found on the section >> 30359 ?outliers found on the section >> 32026 ?outliers found on the section >> 28328 ?outliers found on the section >> 28367 ?outliers found on the section >> 27180 ?outliers found on the section >>> sessionInfo() >> R version 2.12.1 (2010-12-16) >> Platform: i386-pc-mingw32/i386 (32-bit) >> >> locale: >> [1] LC_COLLATE=English_United States.1252 >> [2] LC_CTYPE=English_United States.1252 >> [3] LC_MONETARY=English_United States.1252 >> [4] LC_NUMERIC=C >> [5] LC_TIME=English_United States.1252 >> >> attached base packages: >> [1] grid ? ? ?stats ? ? graphics ?grDevices utils ? ? datasets ?methods >> [8] base >> >> other attached packages: >> ?[1] hwriter_1.3 ? ? ? ? ?geneplotter_1.28.0 ? lattice_0.19-13 >> ?[4] annotate_1.28.0 ? ? ?AnnotationDbi_1.12.0 RColorBrewer_1.0-2 >> ?[7] gplots_2.8.0 ? ? ? ? caTools_1.11 ? ? ? ? bitops_1.0-4.1 >> [10] gdata_2.8.1 ? ? ? ? ?gtools_2.6.2 ? ? ? ? beadarray_2.0.4 >> [13] Biobase_2.10.0 >> >> loaded via a namespace (and not attached): >> [1] DBI_0.2-5 ? ? limma_3.6.9 ? RSQLite_0.9-4 tools_2.12.1 ?xtable_1.5-6 >>> >> >> >> ----- Original Message ----- >> From: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> >> To: "Mike Smith" <grimbough at="" gmail.com=""> >> Cc: bioconductor at stat.math.ethz.ch >> Sent: Sunday, February 20, 2011 9:20:07 PM >> Subject: Re: [BioC] problems with readIllumina() for many BeadChips >> >> Hi Mike et al., >> ? first thanks for the help, I can now read and process the bead- level data from multiple (HumanHT-12 v4) Expression BeadChips just fine. Unfortunately I've run into the next hurdle, where I have a hard time finding information in the beadarray vignette or manual. The background corrected intensities in the txt files (expectedly) contain some negative values, but I can only find the log2 transformation. When I worked with the old version of beadarray a couple of years ago I used the background correction method of rma (backgroundMethod="normexp" in readIllumina) to obtain positive intensities prior to transformation, but this option seems no longer available? So how does beadarray deal with the negative intensities? Also, is there any opinion on the order of steps - transform bead- level intensities, then summarize, or summarize and then log2 transform, (either followed by normalization)? >> Many thanks ... Ina >> >> >> ----- Original Message ----- >> From: "Mike Smith" <grimbough at="" gmail.com=""> >> To: "Ina Hoeschele" <inah at="" vbi.vt.edu=""> >> Cc: bioconductor at stat.math.ethz.ch >> Sent: Thursday, February 17, 2011 1:59:11 PM >> Subject: Re: [BioC] problems with readIllumina() for many BeadChips >> >> Hi Ina, >> >> Sorry, I think my message has probably created some confusion. >> >> Version 2.0.4 is not the developmental version, its a patched version of the >> release. ?You can install it using biocLite("beadarray"). ?I uploaded the >> patch yesterday, so it may take a day or so to filter through. >> >> If you want to install the developmental version (currently 2.1.11) you >> normally need to be running a developmental version of R, although there was >> some discussion about this a few weeks back on this list. ?This version >> should be considered experimental and potentially buggy, but as you've seen >> that can probably be applied to parts of the release package too! >> >> Mike Smith >> PhD Student >> Computational Biology Group >> Department of Oncology >> Cambridge University >> On 17 Feb 2011 16:52, "Ina Hoeschele" <inah at="" vbi.vt.edu=""> wrote: >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor >> >
ADD REPLY •
0
Entering edit mode
Hi Mark, thanks much for all your emails. Sorry for bothering you with this really silly problem once more. << Regarding your expressionQCPipeline problem; it should be trying to write plots to a directory called "QC.Dir[iChip]". If you run dir("QC.Dir[iChip]") what does it return? >> it created a directory within the current working directory with the name QC.Dir[iChip]. So then I deleted this directory, closed R, and then rerun this code: library(beadarray) library(gplots) library(RColorBrewer) library(geneplotter) nChips <- 8 Chip.Dir <- c("C:/5655508006","C:/5655508014","C:/5655508045","C:/5655 508059","C:/5655735016","C:/5655735017","C:/5655735018","C:/5655735079 ") iChip <- 1 setwd(Chip.Dir[iChip]) BLData = readIllumina(useImages=FALSE, illuminaAnnotation="Humanv4") expressionQCPipeline(BLData,transFun=logGreenChannelTransform,qcDir="Q C",plotType=".jpeg",horizontal=TRUE,controlProfile=NULL, overWrite=FALSE,nSegments=9,outlierFun=illuminaOutlierMethod,t agsToDetect=list(housekeeping="housekeeping",Biotin="phage_lambda_geno me", Hybridisation="phage_lambda_genome:high"),zlim=c(5,7),positive ControlTags=c("housekeeping","phage_lambda_genome"), hybridisationTags=c("phage_lambda_genome:low","phage_lambda_ge nome:med","phage_lambda_genome:high"),negativeTag="permuted_negative", boxplotFun=logGreenChannelTransform,imageplotFun=logGreenChann elTransform) Then it still gives me the message that the plots already exist! It created a sub-directory QC within the C:/5655508006 directory, as it is supposed to. The box-plots look fine, and there are several excel files containing information (detectionMetrics, outlierMetrics etc.). But the plots contained in hyb, imageplot, outliers and poscont are all empty.
ADD REPLY •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 442 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6