CopywriteR job runs out of memory
1
0
Entering edit mode
Ram ▴ 60
@ram
Last seen 8 days ago
Katy

Hello,

I'm looking to run Copywriter on a few Whole Exome BAMs. The CopywriteR() call runs to completion when I run it on just one BAM. When I try multiple BAMs though (say 3), the job errors out with PBS: job killed: vmem 87574020096 exceeded limit 68719476736 even when I give it 64G of RAM (the BAMs put together are 18+18+15 = 51G, so even reading them all into memory should not be a problem. Why am I facing this error?

copywriter cnv • 161 views
ADD COMMENT
2
Entering edit mode
@oscarkrijgsman-6749
Last seen 12 months ago
Netherlands, Amsterdam, the Netherlands…

Dear Ramrs,

I agree with you that system memory should not be the problem. CopywriteR depends on Rsamtools to read and write the bam files you provide. Rsamtools has a limit to the number of reads it can handle. It looks like that at least for one of your samples the number of reads exceeds the Rsamtools limit. An easy solution is to make a new bam file with a subset of sequence reads from your original bam files (1/2 or 2/3 of reads). This will not affect the quality of the DNA copy number profiles because you have far more sequence reads than needed for a good quality profile.

Kind regards, Oscar

ADD COMMENT
0
Entering edit mode

All software runs in limited memory. The solution is to iterate through the file, as illustrated in eg. 'Strategy 4' of ?GenomicAlignments::readGAlignments or implemented in GenomicFiles::reduceBy*() functions.

ADD REPLY
0
Entering edit mode

Thank you, Oscar. I upped the memory to 128G and things ran to completion. A few other BAMs with lower depth worked fine, so I think your solution makes sense.

Is there any way to up the number of reads Rsamtools can handle? Maybe a recompilation of samtools or Rsamtools?

ADD REPLY
0
Entering edit mode

As Martin Morgan suggested I should implement an iterative strategy to decrease memory usage. I will work on this as soon as possible to implement this in the next version of CopywriteR.

ADD REPLY

Login before adding your answer.

Traffic: 417 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6