Questions related to processing large dataset in batches
0
0
Entering edit mode
@james-w-macdonald-5106
Last seen 9 hours ago
United States
If you are only planning on doing rma or gcrma, you might take a look at justRMA and justGCRMA, which use much less memory. I was just informed off-list that you can do 116 of the hgu133plus2 chips with 1 Gb RAM using justRMA. HTH, Jim James W. MacDonald Affymetrix and cDNA Microarray Core University of Michigan Cancer Center 1500 E. Medical Center Drive 7410 CCGC Ann Arbor MI 48109 734-647-5623 >>> "Li, Aiguo (NIH/NCI)" <liai@mail.nih.gov> 06/30/04 03:33PM >>> Hello, everyone. Our project is a beta tester of Affy HGU133 plus 2 chips, which contain 56,000 probes, and the .cel file size in text format is about 32 MB. We currently have more than 100 chips for data process. I tried to read in the .cel files into my machine (1Gb RAM) and it can only read in 19 chips. I have been communicating with several R experts in our mailing list and some of them suggest me to split the data in batches during the probe level data analysis and combine at the probeset level using R cbine/merge function. I think that this probably is the best option for me because I have concerns on data handling capabilities even though I can upgrade my RAM to 4Gb. However, my second concern is whether the solution of batch analysis will have any effects on the final data analysis results. To my opinion, normalization across chips should be done at once across all chips. Can I have probe level normalization during the batch analysis and have an additional normalization at the probeset level across all chips after the data combination using R/bioconductor? Thanks in advance, Aiguo Lee [[alternative HTML version deleted]] _______________________________________________ Bioconductor mailing list Bioconductor@stat.math.ethz.ch https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
Microarray Normalization Cancer hgu133plus2 probe affy gcrma PROcess Microarray Cancer • 694 views
ADD COMMENT

Login before adding your answer.

Traffic: 697 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6