Dear Vladimir, sorry for the late reply... I'll give it a try and hope some true expert will correct me if it is nonsense... :-) Q2: rather new in 2011 would mean *probably* 4x44Kv2... the type should be available in the file header if you still have one (?) or otherwise one could guess it from the set of identifiers in column "ProbeName" if you still have one... can you make one file available via web or ftp for a quick look? Visualization should still be feasible, with missing spots missing, of course, and in this case it's a pity the positive controls are missing... IIRC, Agilent FES does produce these plots for their QC -- but I suppose your company did not include them? Q5: ok, so same or similar common reference we are using... and to be useful it should give a reasonable signal for (almost) all probes on the array which is the whole genome -- but only a proportion of these will be expressed in any real biological sample which is why I think that a) MA plots will look pretty unusual for these arrays and b) LOESS normalization will seemingly fix that but actually distort your data. As for the choice of normalization method, since all normalization steps bear the risk of "normalizing" away the biological signal you're interested in, you should do only as much as necessary, using the least stringent method that will produce proper diagnostic plots. For comparison between arrays, density and box plots would be appropriate. I realize this is probably too general to be useful :-) maybe the literature referenced in limma's ?normalizeWithinArrays and ?normalizeBetweenArrays can be of any help? Cheers, - axel Axel Klenk Research Informatician Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / Switzerland From: "Vladimir Krasikov" <v.v.krasikov at="" gmail.com=""> To: axel.klenk at actelion.com Cc: bioconductor at r-project.org, bioconductor-bounces at r-project.org Date: 09.02.2012 13:47 Subject: Re: [BioC] Limma: questions about data pre-processing Dear Axel Once again thanks... Q2: The only thing I know now is that it was rather new Agilent edition of March 2011, however our company stripped away all information in files ( even removed all control spots). Do you think there is still a way to make visualizations? Q5: After reading Rquantile description I now see some rationale about this normalization, when all Red chanels contoined common reference (which is commercial "universal human reference"). However, question remains, what kind of plots, metrics are useful to judge the results of normalizations? On Tue, 07 Feb 2012 15:32:03 +0100, <axel.klenk at="" actelion.com=""> wrote: > Dear Vladimir, > > I'll only answer or comment on some of your questions and leave > the others for the true experts... > > Q2: yes, for example using package arrayQualityMetrics, if you know > the array layout. FES output usually contains columns Col and Row for > spot coordinates but apparently your "service provider" has removed > them. I could send you a coordinates <--> oligo mapping by email if you > can tell me your array type -- is it 1x44K, 4x44K or 4x44Kv2? > Alternatively, > you can try to find that information on Agilent's eArray web site: > earray.chem.agilent.com > > Q5: for a common reference design, dye swaps are not required and > I would not apply a loess normalization -- depending on what you have > hybridized as the common reference, the assumptions may not hold. > As for the between-array normalization, Rquantile may also be an > option for your design and boxplots and density plots may be used > for judging the results. > > Cheers, > > - axel > > > Axel Klenk > Research Informatician > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > Switzerland > > > > > From: > "Vladimir Krasikov" <v.v.krasikov at="" gmail.com=""> > To: > bioconductor at r-project.org > Date: > 07.02.2012 14:27 > Subject: > [BioC] Limma: questions about data pre-processing > Sent by: > bioconductor-bounces at r-project.org > > > > Dear limma experts > > During creating the pipe-line for dissecting differential gene expression > in frame of limma, > several questions have arose. > > Experiment: > I have 62 two-color Agilent human arrays. > The samples are from several human more or less related to each other > disorders and vary in age, sex, disease duration and diagnosis. > Company that made hybridizations performed all hybs in one direction (no > dye-swaps), > where all samples were in G channel and common Ref in R channel, > and unfortunately provided us only excepts of Feature Extraction > which contained info on G, Gb, R, Rb, and FNO (non-uniformity outliers) > and separate gene annotation table. > > I performed generic import of the data and assigned zero-weight to the > FNO > spots: > I analyzed density and MA-plots, box-plots of M-values, G and R channels > and box-plots of background intensities, > and removed from experiment 1 array with aberrant raw G-channel density. > (I will discuss experiment description later, when come to the linear > model) > > Q1: Is there a rationale of down-weighting FNO (around 100-200 spots per > array) for background correction and further normalization? > Q2: Is there way to make image representation of Agilent microarray (for > each channel and backgrounds)? > In another words is there known 'layout' for human 44K Agilent? > > Next I corrected the background with: >> RG.b <- backgroundCorrect(RG.raw, method="minimum", offset=50) > (recommended method=normexp produced shifted curves for five arrays after > taking a look on density plots, > and box-plots for separate G and R channels also look less uniform as > compared with 'minimum' method) > > Q3: I guess it is also possible to remove those 5 arrays from the > experiment. Is it fair? > Q4: What kind of reasoning should be used for the choice between > background subtraction methods? > > Then performed standard loess within array normalization: >> MA.loess <- normalizeWithinArrays(RG.b, method="loess",bc.method="none") > > Q5: Do I need to perform between array normalization? > How to judge which of the methods (non, scale, quantile, Aquantile) > is > best for my experiment? > > For now I decide to stuck with background=minimum, within=loess, and > between=is under the question > > Next I would like to ask questions about > linear model of my experiment, but I will make it in a next help request > > Thanks a lot in advance > > and finally >> sessionInfo() > R version 2.14.1 (2011-12-22) > Platform: i386-pc-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=Dutch_Netherlands.1252 LC_CTYPE=Dutch_Netherlands.1252 > [3] LC_MONETARY=Dutch_Netherlands.1252 LC_NUMERIC=C > [5] LC_TIME=Dutch_Netherlands.1252 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] limma_3.10.2 >> > > With kind regards > Vladimir > -- > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > > > > > The information of this email and in any file transmitted with it is > strictly confidential and may be legally privileged. > It is intended solely for the addressee. If you are not the intended > recipient, any copying, distribution or any other use of this email is > prohibited and may be unlawful. In such case, you should please notify > the sender immediately and destroy this email. > The content of this email is not legally binding unless confirmed by > letter. > Any views expressed in this message are those of the individual sender, > except where the message states otherwise and the sender is authorised > to state them to be the views of the sender's company. For further > information about Actelion please see our website at > http://www.actelion.com > -- Using Opera's revolutionary email client: http://www.opera.com/mail/ The information of this email and in any file transmitted with it is strictly confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any copying, distribution or any other use of this email is prohibited and may be unlawful. In such case, you should please notify the sender immediately and destroy this email. The content of this email is not legally binding unless confirmed by letter. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorised to state them to be the views of the sender's company. For further information about Actelion please see our website at http://www.actelion.com

Dear Vladimir, unfortunately, this turns out to be harder than expected ( as in fortune(211) :-)): yes, all controls have been removed from that file and the set of oligos left is different from any we have ever used (that is, all whole human genome from Agilent's catalogue), so it is probably a custom design... obviously, in order to find out you'll have to ask "that company".... Cheers, - axel Axel Klenk Research Informatician Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / Switzerland From: "Vladimir Krasikov" <v.v.krasikov at="" gmail.com=""> To: axel.klenk at actelion.com Cc: bioconductor at r-project.org, bioconductor-bounces at r-project.org Date: 10.02.2012 16:24 Subject: Re: [BioC] Limma: questions about data pre-processing Dear Axel Thanks for all your explanations and efforts. After your advise to use arrayQualityMetrics package to build a report covering all arrays, I'm exploring its possibilities and find it is indeed produce really nice intuitive report. Q2. Here is one of the arrays link: http://db.tt/ICfiVlUt (I shared it with dropbox) Take a look and may be you will be able to link coordinates to this file, it seems for me that it is stripped of controls, however the order of probes is the same as it should appear in FE. I may guess that company just did export to txt from excel while removing all FE info, controls and most of the columns. Thanks a lot for your time. Q5. After your explanations I will for sure use Rquantile normalisation. Kind regards Vladimir On Thu, 09 Feb 2012 18:02:40 +0100, <axel.klenk at="" actelion.com=""> wrote: > Dear Vladimir, > > sorry for the late reply... I'll give it a try and hope some true expert > will > correct me if it is nonsense... :-) > > Q2: rather new in 2011 would mean *probably* 4x44Kv2... the type should > be available in the file header if you still have one (?) or otherwise > one > could > guess it from the set of identifiers in column "ProbeName" if you still > have > one... can you make one file available via web or ftp for a quick look? > Visualization should still be feasible, with missing spots missing, of > course, > and in this case it's a pity the positive controls are missing... > IIRC, Agilent FES does produce these plots for their QC -- but I suppose > your company did not include them? > > Q5: ok, so same or similar common reference we are using... and to be > useful > it should give a reasonable signal for (almost) all probes on the array > which is > the whole genome -- but only a proportion of these will be expressed in > any > real biological sample which is why I think that a) MA plots will look > pretty > unusual for these arrays and b) LOESS normalization will seemingly fix > that > but actually distort your data. > > As for the choice of normalization method, since all normalization steps > bear > the risk of "normalizing" away the biological signal you're interested > in, > you > should do only as much as necessary, using the least stringent method > that > will produce proper diagnostic plots. For comparison between arrays, > density > and box plots would be appropriate. I realize this is probably too > general > to be > useful :-) maybe the literature referenced in limma's > ?normalizeWithinArrays and > ?normalizeBetweenArrays can be of any help? > > Cheers, > > - axel > > > Axel Klenk > Research Informatician > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / > Switzerland > > > > > From: > "Vladimir Krasikov" <v.v.krasikov at="" gmail.com=""> > To: > axel.klenk at actelion.com > Cc: > bioconductor at r-project.org, bioconductor-bounces at r-project.org > Date: > 09.02.2012 13:47 > Subject: > Re: [BioC] Limma: questions about data pre-processing > > > > > Dear Axel > > Once again thanks... > > Q2: The only thing I know now is that it > was rather new Agilent edition of March 2011, > however our company stripped away all information in files ( even removed > all control spots). > Do you think there is still a way to make visualizations? > > Q5: After reading Rquantile description I now see some rationale about > this normalization, > when all Red chanels contoined common reference (which is commercial > "universal human reference"). > However, question remains, what kind of plots, metrics are useful to > judge > the results of normalizations? > > On Tue, 07 Feb 2012 15:32:03 +0100, <axel.klenk at="" actelion.com=""> wrote: > >> Dear Vladimir, >> >> I'll only answer or comment on some of your questions and leave >> the others for the true experts... >> >> Q2: yes, for example using package arrayQualityMetrics, if you know >> the array layout. FES output usually contains columns Col and Row for >> spot coordinates but apparently your "service provider" has removed >> them. I could send you a coordinates <--> oligo mapping by email if you >> can tell me your array type -- is it 1x44K, 4x44K or 4x44Kv2? >> Alternatively, >> you can try to find that information on Agilent's eArray web site: >> earray.chem.agilent.com >> >> Q5: for a common reference design, dye swaps are not required and >> I would not apply a loess normalization -- depending on what you have >> hybridized as the common reference, the assumptions may not hold. >> As for the between-array normalization, Rquantile may also be an >> option for your design and boxplots and density plots may be used >> for judging the results. >> >> Cheers, >> >> - axel >> >> >> Axel Klenk >> Research Informatician >> Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil / >> Switzerland >> >> >> >> >> From: >> "Vladimir Krasikov" <v.v.krasikov at="" gmail.com=""> >> To: >> bioconductor at r-project.org >> Date: >> 07.02.2012 14:27 >> Subject: >> [BioC] Limma: questions about data pre-processing >> Sent by: >> bioconductor-bounces at r-project.org >> >> >> >> Dear limma experts >> >> During creating the pipe-line for dissecting differential gene > expression >> in frame of limma, >> several questions have arose. >> >> Experiment: >> I have 62 two-color Agilent human arrays. >> The samples are from several human more or less related to each other >> disorders and vary in age, sex, disease duration and diagnosis. >> Company that made hybridizations performed all hybs in one direction (no >> dye-swaps), >> where all samples were in G channel and common Ref in R channel, >> and unfortunately provided us only excepts of Feature Extraction >> which contained info on G, Gb, R, Rb, and FNO (non-uniformity outliers) >> and separate gene annotation table. >> >> I performed generic import of the data and assigned zero-weight to the >> FNO >> spots: >> I analyzed density and MA-plots, box-plots of M-values, G and R channels >> and box-plots of background intensities, >> and removed from experiment 1 array with aberrant raw G-channel density. >> (I will discuss experiment description later, when come to the linear >> model) >> >> Q1: Is there a rationale of down-weighting FNO (around 100-200 spots per >> array) for background correction and further normalization? >> Q2: Is there way to make image representation of Agilent microarray (for >> each channel and backgrounds)? >> In another words is there known 'layout' for human 44K Agilent? >> >> Next I corrected the background with: >>> RG.b <- backgroundCorrect(RG.raw, method="minimum", offset=50) >> (recommended method=normexp produced shifted curves for five arrays > after >> taking a look on density plots, >> and box-plots for separate G and R channels also look less uniform as >> compared with 'minimum' method) >> >> Q3: I guess it is also possible to remove those 5 arrays from the >> experiment. Is it fair? >> Q4: What kind of reasoning should be used for the choice between >> background subtraction methods? >> >> Then performed standard loess within array normalization: >>> MA.loess <- normalizeWithinArrays(RG.b, > method="loess",bc.method="none") >> >> Q5: Do I need to perform between array normalization? >> How to judge which of the methods (non, scale, quantile, Aquantile) >> is >> best for my experiment? >> >> For now I decide to stuck with background=minimum, within=loess, and >> between=is under the question >> >> Next I would like to ask questions about >> linear model of my experiment, but I will make it in a next help request >> >> Thanks a lot in advance >> >> and finally >>> sessionInfo() >> R version 2.14.1 (2011-12-22) >> Platform: i386-pc-mingw32/i386 (32-bit) >> >> locale: >> [1] LC_COLLATE=Dutch_Netherlands.1252 LC_CTYPE=Dutch_Netherlands.1252 >> [3] LC_MONETARY=Dutch_Netherlands.1252 LC_NUMERIC=C >> [5] LC_TIME=Dutch_Netherlands.1252 >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] limma_3.10.2 >>> >> >> With kind regards >> Vladimir >> -- >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: >> http://news.gmane.org/gmane.science.biology.informatics.conductor >> >> >> >> >> The information of this email and in any file transmitted with it is >> strictly confidential and may be legally privileged. >> It is intended solely for the addressee. If you are not the intended >> recipient, any copying, distribution or any other use of this email is >> prohibited and may be unlawful. In such case, you should please notify >> the sender immediately and destroy this email. >> The content of this email is not legally binding unless confirmed by >> letter. >> Any views expressed in this message are those of the individual sender, >> except where the message states otherwise and the sender is authorised >> to state them to be the views of the sender's company. For further >> information about Actelion please see our website at >> http://www.actelion.com >> > > -- Using Opera's revolutionary email client: http://www.opera.com/mail/ The information of this email and in any file transmitted with it is strictly confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any copying, distribution or any other use of this email is prohibited and may be unlawful. In such case, you should please notify the sender immediately and destroy this email. The content of this email is not legally binding unless confirmed by letter. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorised to state them to be the views of the sender's company. For further information about Actelion please see our website at http://www.actelion.com