Question: Re: Try to run a lots of chips: 32 bits vs 64 bits
0
Nicholas Lewin-Koh • 430 wrote:
Hi,
I have nothing to add on the 32-64bit issue, However for your memory
problem to add on to what i think Laurent was suggesting is that you
have to break up your problem into smaller parts. The main problem
will
be quantile normalization. The other methods work pairwise, so that
you
could script it in a sequential fashion. maybe you could do the
quantile
normalization on blocks of oligos across chips?
What I do for jobs that hit memory boundaries is
1) work out the procedure on a subset of the data.
2) write a script to do the processing, delete as you go and figure
out
where you can write output to be loaded back later This will slow
things
down because of writing to disk.
If you can break up the processing you should be able to get the
analysis done.
Another alternative might be to try using Tony Rossini's lib PVM
interface if you have access to multiple processors on a network.
Though
setting up libpvm for that is a bitch.
Nicholas
ADD COMMENT
• link
•
modified 16.8 years ago
by
Anthony Rossini • 530
•
written
16.8 years ago by
Nicholas Lewin-Koh • 430

