Entering edit mode
Nicholas Lewin-Koh
▴
430
@nicholas-lewin-koh-63
Last seen 10.3 years ago
Hi,
I have nothing to add on the 32-64bit issue, However for your memory
problem to add on to what i think Laurent was suggesting is that you
have to break up your problem into smaller parts. The main problem
will
be quantile normalization. The other methods work pairwise, so that
you
could script it in a sequential fashion. maybe you could do the
quantile
normalization on blocks of oligos across chips?
What I do for jobs that hit memory boundaries is
1) work out the procedure on a subset of the data.
2) write a script to do the processing, delete as you go and figure
out
where you can write output to be loaded back later This will slow
things
down because of writing to disk.
If you can break up the processing you should be able to get the
analysis done.
Another alternative might be to try using Tony Rossini's lib PVM
interface if you have access to multiple processors on a network.
Though
setting up libpvm for that is a bitch.
Nicholas