Search
Question: how to set R memory limit
0
gravatar for wang peter
5.9 years ago by
wang peter2.0k
wang peter2.0k wrote:
i have 64 G memory server only run one R program but it reported cannot relocate 5.2G memory thx who can tell me how to set R memory limit? -- shan gao Room 231(Dr.Fei lab) Boyce Thompson Institute Cornell University Tower Road, Ithaca, NY 14853-1801 Office phone: 1-607-254-1267(day) Official email:sg839 at cornell.edu Facebook:http://www.facebook.com/profile.php?id=100001986532253
ADD COMMENTlink modified 5.9 years ago by Sunny Yu Liu80 • written 5.9 years ago by wang peter2.0k
0
gravatar for Sean Davis
5.9 years ago by
Sean Davis21k
United States
Sean Davis21k wrote:
On Mon, Jan 9, 2012 at 10:49 PM, wang peter <wng.peter at="" gmail.com=""> wrote: > i have 64 G memory server > only run one R program > but it reported > ?cannot relocate 5.2G memory > > thx > who can tell me how to set R memory limit? sessionInfo()? Sean
ADD COMMENTlink written 5.9 years ago by Sean Davis21k
0
gravatar for Steve Lianoglou
5.9 years ago by
Genentech
Steve Lianoglou12k wrote:
Hi, On Mon, Jan 9, 2012 at 10:49 PM, wang peter <wng.peter at="" gmail.com=""> wrote: > i have 64 G memory server > only run one R program > but it reported > ?cannot relocate 5.2G memory Someone sometime ago on R-help (can't remember who) put it best, which I'll paraphrase here: The "cannot relocate XXX memory" error is fired when the "last straw is drawn which breaks the camel's back" So although you have a 64GB server and assuming you are running in 64bit mode, this error doesn't tell you that your server doesn't have 5.4GB memory in it, it means that R (or something else) has been chewing up your memory and R tried to get a 5.4G contiguous block of it, and apparently there isn't any free block of that size left. Use your favorite process monitoring tool for your OS to keep on eye on your CPU's memory usage to confirm. Also, as Sean suggests, sessionInfo output is always helpful. HTH, -steve -- Steve Lianoglou Graduate Student: Computational Systems Biology ?| Memorial Sloan-Kettering Cancer Center ?| Weill Medical College of Cornell University Contact Info: http://cbio.mskcc.org/~lianos/contact
ADD COMMENTlink written 5.9 years ago by Steve Lianoglou12k
Hello, I had such a problem several times ago. Look at the option() parameter there you can set the RAM limit. e.g. options(java.parameters = "-Xmx8192m") to allocate 8GB RAM. the default is 1024MB I think. Best regards Basti 2012/1/10 Steve Lianoglou <mailinglist.honeypot at="" gmail.com="">: > Hi, > > On Mon, Jan 9, 2012 at 10:49 PM, wang peter <wng.peter at="" gmail.com=""> wrote: >> i have 64 G memory server >> only run one R program >> but it reported >> ?cannot relocate 5.2G memory > > Someone sometime ago on R-help (can't remember who) put it best, which > I'll paraphrase here: The "cannot relocate XXX memory" error is fired > when the "last straw is drawn which breaks the camel's back" > > So although you have a 64GB server and assuming you are running in > 64bit mode, this error doesn't tell you that your server doesn't have > 5.4GB memory in it, it means that R (or something else) has been > chewing up your memory and R tried to get a 5.4G contiguous block of > it, and apparently there isn't any free block of that size left. > > Use your favorite process monitoring tool for your OS to keep on eye > on your CPU's memory usage to confirm. > > Also, as Sean suggests, sessionInfo output is always helpful. > > HTH, > > -steve > > -- > Steve Lianoglou > Graduate Student: Computational Systems Biology > ?| Memorial Sloan-Kettering Cancer Center > ?| Weill Medical College of Cornell University > Contact Info: http://cbio.mskcc.org/~lianos/contact > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLYlink written 5.9 years ago by Sebastian Thieme60
On Tue, Jan 10, 2012 at 3:30 AM, Sebastian Thieme <thieme at="" mi.fu-berlin.de=""> wrote: > Hello, > > I had such a problem several times ago. Look at the option() parameter > there you can set the RAM limit. e.g. options(java.parameters = > "-Xmx8192m") to allocate 8GB RAM. the default is 1024MB I think. Hi, Sebastian. This option sets the java heap space and does not affect R memory size. I would not expect setting this option to fix the original poster's problem (but the report was not very detailed), but if one is using a package that uses the JVM, this is a useful parameter to set. Sean > Best regards > > Basti > > 2012/1/10 Steve Lianoglou <mailinglist.honeypot at="" gmail.com="">: >> Hi, >> >> On Mon, Jan 9, 2012 at 10:49 PM, wang peter <wng.peter at="" gmail.com=""> wrote: >>> i have 64 G memory server >>> only run one R program >>> but it reported >>> ?cannot relocate 5.2G memory >> >> Someone sometime ago on R-help (can't remember who) put it best, which >> I'll paraphrase here: The "cannot relocate XXX memory" error is fired >> when the "last straw is drawn which breaks the camel's back" >> >> So although you have a 64GB server and assuming you are running in >> 64bit mode, this error doesn't tell you that your server doesn't have >> 5.4GB memory in it, it means that R (or something else) has been >> chewing up your memory and R tried to get a 5.4G contiguous block of >> it, and apparently there isn't any free block of that size left. >> >> Use your favorite process monitoring tool for your OS to keep on eye >> on your CPU's memory usage to confirm. >> >> Also, as Sean suggests, sessionInfo output is always helpful. >> >> HTH, >> >> -steve >> >> -- >> Steve Lianoglou >> Graduate Student: Computational Systems Biology >> ?| Memorial Sloan-Kettering Cancer Center >> ?| Weill Medical College of Cornell University >> Contact Info: http://cbio.mskcc.org/~lianos/contact >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLYlink written 5.9 years ago by Sean Davis21k
0
gravatar for Sean Davis
5.9 years ago by
Sean Davis21k
United States
Sean Davis21k wrote:
On Tue, Jan 10, 2012 at 9:15 AM, wang peter <wng.peter at="" gmail.com=""> wrote: > thank u to all of your replies > sorry to miss the sessionInfo() > R version 2.13.2 (2011-09-30) > Platform: x86_64-unknown-linux-gnu (64-bit) > > > but i think how to set R memory is nothing related with R version. Memory use is related to R version, architecture, and operating system. These three details are returned by sessionInfo(). > i got many news to set it but donot know which one is correct > > i tried > R --max-vsize=20G > options(java.parameters = "-Xmx8192m") --max-vsize of 20G will limit memory to approximately 20G; I thought you said that you had 64G of memory. Does this make sense? Java parameters do not affect R directly, only JVMs started by the rare R package that runs java, so this will not solve your original problem. > and also restarted the computer to clean the memory > > but i still have no ideas with R memory management > See help('Memory-limits'). However, on 64-bit linux, the original error message you reported is related to not having enough memory to complete the operation; there is generally no need to manually increase memory. As Steve suggested, run 'top' in another window to watch R memory use. Use gc() regularly and be sure to clean up your R workspace. If you still cannot do what you need, let us know what you are trying to do by including code, errors, and the output of sessionInfo(). Perhaps there is a way to do things with less memory use. In the end, you may need more memory, though. Sean -- > shan gao > Room 231(Dr.Fei lab) > Boyce Thompson Institute > Cornell University > Tower Road, Ithaca, NY 14853-1801 > Office phone: 1-607-254-1267(day) > Official email:sg839 at cornell.edu > Facebook:http://www.facebook.com/profile.php?id=100001986532253
ADD COMMENTlink written 5.9 years ago by Sean Davis21k
Hi, Just adding to what Sean said and using some intuition: On Tue, Jan 10, 2012 at 10:20 AM, Sean Davis <sdavis2 at="" mail.nih.gov=""> wrote: [snip] >?If you still cannot do what you need, let us know what you > are trying to do by including code, errors, and the output of > sessionInfo(). ?Perhaps there is a way to do things with less memory > use. ?In the end, you may need more memory, though. I'm guessing you are (1) doing something w/ sequencing data; and (2) trying to load all of your data at once. Assuming that's true (and we all know how prudent making assumptions is), I'd suggest just doing whatever is that you are doing by loading the data for one chromosome at a time. As an aside -- this usage pattern is quite frequent, ie. "get all reads from chromsome X and do Y", I took a stab sometime implementing "iterators" for the foreach package to abstract this idea (get batch of reads and do Y), like: xxx <- foreach(reads=getReadsByChromosome(bam.file), ...) %dopar% { ## compute over `reads`, and profit } Maybe it's worth nailing down into a `seqiterators` package, or something? -steve -- Steve Lianoglou Graduate Student: Computational Systems Biology ?| Memorial Sloan-Kettering Cancer Center ?| Weill Medical College of Cornell University Contact Info: http://cbio.mskcc.org/~lianos/contact
ADD REPLYlink written 5.9 years ago by Steve Lianoglou12k
0
gravatar for Sunny Yu Liu
5.9 years ago by
Sunny Yu Liu80
Sunny Yu Liu80 wrote:
For Windows, try memory.limit( size=XXX) check memory limit: memory.size(max=TRUE) For Mac or Linux, may try mem.limits(nsize=NA, vsize=NA) to check or set memory size maybe this is helpful for the question here -----Original Message----- From: bioconductor-bounces@r-project.org on behalf of Sean Davis Sent: Tue 1/10/2012 10:20 AM To: wang peter Cc: bioconductor@r-project.org Subject: Re: [BioC] how to set R memory limit On Tue, Jan 10, 2012 at 9:15 AM, wang peter <wng.peter@gmail.com> wrote: > thank u to all of your replies > sorry to miss the sessionInfo() > R version 2.13.2 (2011-09-30) > Platform: x86_64-unknown-linux-gnu (64-bit) > > > but i think how to set R memory is nothing related with R version. Memory use is related to R version, architecture, and operating system. These three details are returned by sessionInfo(). > i got many news to set it but donot know which one is correct > > i tried > R --max-vsize=20G > options(java.parameters = "-Xmx8192m") --max-vsize of 20G will limit memory to approximately 20G; I thought you said that you had 64G of memory. Does this make sense? Java parameters do not affect R directly, only JVMs started by the rare R package that runs java, so this will not solve your original problem. > and also restarted the computer to clean the memory > > but i still have no ideas with R memory management > See help('Memory-limits'). However, on 64-bit linux, the original error message you reported is related to not having enough memory to complete the operation; there is generally no need to manually increase memory. As Steve suggested, run 'top' in another window to watch R memory use. Use gc() regularly and be sure to clean up your R workspace. If you still cannot do what you need, let us know what you are trying to do by including code, errors, and the output of sessionInfo(). Perhaps there is a way to do things with less memory use. In the end, you may need more memory, though. Sean -- > shan gao > Room 231(Dr.Fei lab) > Boyce Thompson Institute > Cornell University > Tower Road, Ithaca, NY 14853-1801 > Office phone: 1-607-254-1267(day) > Official email:sg839@cornell.edu > Facebook:http://www.facebook.com/profile.php?id=100001986532253 _______________________________________________ Bioconductor mailing list Bioconductor@r-project.org https://stat.ethz.ch/mailman/listinfo/bioconductor Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor [[alternative HTML version deleted]]
ADD COMMENTlink written 5.9 years ago by Sunny Yu Liu80
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 355 users visited in the last hour