Re: Try to run a lots of chips: 32 bits vs 64 bits
1
0
Entering edit mode
@nicholas-lewin-koh-63
Last seen 10.3 years ago
Hi, I have nothing to add on the 32-64bit issue, However for your memory problem to add on to what i think Laurent was suggesting is that you have to break up your problem into smaller parts. The main problem will be quantile normalization. The other methods work pairwise, so that you could script it in a sequential fashion. maybe you could do the quantile normalization on blocks of oligos across chips? What I do for jobs that hit memory boundaries is 1) work out the procedure on a subset of the data. 2) write a script to do the processing, delete as you go and figure out where you can write output to be loaded back later This will slow things down because of writing to disk. If you can break up the processing you should be able to get the analysis done. Another alternative might be to try using Tony Rossini's lib PVM interface if you have access to multiple processors on a network. Though setting up libpvm for that is a bitch. Nicholas
Normalization GO Network Normalization GO Network • 1.0k views
ADD COMMENT
0
Entering edit mode
@anthony-rossini-10
Last seen 10.3 years ago
Nicholas Lewin-Koh <nikko@hailmail.net> writes: > Another alternative might be to try using Tony Rossini's lib PVM > interface if you have access to multiple processors on a network. Though > setting up libpvm for that is a bitch. Actually, it's much, much better now for configuration. Paper done soon (like the end of the week, at worst). (generalized to Luke's SNOW package, with LAM-MPI, PVM, or sockets... I've got to get it integrated with Bioconductor, as soon as I sort out a few real-life issues...). best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ -------------------- FHCRC:Tu: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX (CHANGE: monday/wednesday/friday locations are completely unpredictable.)
ADD COMMENT

Login before adding your answer.

Traffic: 510 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6