Memory Issue under WinXP x64 (64 bit Windows XP)
4
0
Entering edit mode
@carleton-garrett-52
Last seen 9.6 years ago
Hi I'm currently running Bioconductor version 2.2.0 under Windows XP x64 with 16 Gb RAM and Virtual Memory upto 100 Gb. In trying to combine 67 Affy u133a and 67 Affy u133a_2 cel files I am able to form the initial affy batches using read.affybatch() but get a memory allocation error (below) when I try to combine them with the 'combineAffyBatch()' function Error: cannot allocate vector of size 280,470 Kb Attempts to increase memory size within 'R' are only partly successful. Attempting to increase the memory to 16 Gb (16000 Mb) through using the following command memory.limit(size=16000) produces the error Error in memory.size(size) : don't be silly! : your machine has a 4 Gb limit [not true according to the OS]. Reducing the number of cel files in the two initial affy batches to 16 each (but not 33 each) did allow the combined affy batch to be created so the combneAffyBatch() function appears to work on this platform. Two questions: 1) Does the binary version 2.2.0 of R for Windows have a built-in memory limit of 4 Gb? - and if so, is there a way to circumvent this limit? 2) Is there anything peculiar about the combineAffyBatch() function that makes it use memory inefficiently? I raise the above question because even when the number of cel files was reduced to 33 u133a and 33 u133a_2 there was a memory allocation error when I ran combineAffyBatch() that was the same as with the 67 + 67 cel files (ie, "Error: cannot allocate vector of size 280,470 Kb") and yet the memory.limit() was ~4.2 Gb and the memory.size() was only ~1.4 Gb in a machine with 16 Gb RAM. Any suggestions welcomed. Carl Garrett
affy affy • 2.1k views
ADD COMMENT •
0
Entering edit mode
@kasper-daniel-hansen-2979
Last seen 9 months ago
United States
I am not fully up to date on the 64 bit windows issue, but I would assume that the CRAN build is done for a 32 bit system. If that is the case you need to recompile it - perhaps do a little search to see if someone else has already done it. If you have such a big machine, why are you running windows on it :) /Kasper On May 18, 2006, at 2:01 PM, Carleton Garrett wrote: > Hi > > I'm currently running Bioconductor version 2.2.0 under Windows XP x64 > with 16 Gb RAM and Virtual Memory upto 100 Gb. > > In trying to combine 67 Affy u133a and 67 Affy u133a_2 cel files I am > able to form the initial affy batches using read.affybatch() but get a > memory allocation error (below) when I try to combine them with the > 'combineAffyBatch()' function > > Error: cannot allocate vector of size 280,470 Kb > > Attempts to increase memory size within 'R' are only partly > successful. > Attempting to increase the memory to 16 Gb (16000 Mb) through using > the > following command > > memory.limit(size=16000) > > produces the error > > Error in memory.size(size) : don't be silly! : your machine has a 4 Gb > limit [not true according to the OS]. > > Reducing the number of cel files in the two initial affy batches to 16 > each (but not 33 each) did allow the combined affy batch to be created > so the combneAffyBatch() function appears to work on this platform. > > Two questions: > > 1) Does the binary version 2.2.0 of R for Windows have a built-in > memory limit of 4 Gb? - and if so, is there a way to circumvent > this limit? > > 2) Is there anything peculiar about the combineAffyBatch() function > that makes it use memory inefficiently? > > I raise the above question because even when the number of cel > files > was reduced to 33 u133a and 33 u133a_2 there was a memory allocation > error when I ran combineAffyBatch() that was the same as with the 67 + > 67 cel files (ie, "Error: cannot allocate vector of size 280,470 Kb") > and yet the memory.limit() was ~4.2 Gb and the memory.size() was only > ~1.4 Gb in a machine with 16 Gb RAM. > > Any suggestions welcomed. > > Carl Garrett > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/ > gmane.science.biology.informatics.conductor
ADD COMMENT •
0
Entering edit mode
On Thu, 2006-05-18 at 14:23 -0700, Kasper Daniel Hansen wrote: > I am not fully up to date on the 64 bit windows issue, but I would > assume that the CRAN build is done for a 32 bit system. If that is > the case you need to recompile it - perhaps do a little search to see > if someone else has already done it. That is unfortunately not that easy. The MingW environment that is used to build R does not have a 64 bit version (neither does Cygwin). I do not know if there are other windows compilers which would be able to build R properly. I kind of gave up once I realized it (and told the windows user to reboot in linux), maybe someone more persistent will figure it out. > If you have such a big machine, why are you running windows on it :) I was kind of wondering the same thing, but to each his own. Francois
ADD REPLY •
0
Entering edit mode
@kasper-daniel-hansen-2979
Last seen 9 months ago
United States
Actually in the R for Windows FAQ (which you of course consulted before sending your email), it says under 8.1 "Note that building under Windows 64 is not supported as no suitable compiler is available" This means you need to switch OS if you want R to use the many GB of memory you have. /Kasper On May 18, 2006, at 2:01 PM, Carleton Garrett wrote: > Hi > > I'm currently running Bioconductor version 2.2.0 under Windows XP x64 > with 16 Gb RAM and Virtual Memory upto 100 Gb. > > In trying to combine 67 Affy u133a and 67 Affy u133a_2 cel files I am > able to form the initial affy batches using read.affybatch() but get a > memory allocation error (below) when I try to combine them with the > 'combineAffyBatch()' function > > Error: cannot allocate vector of size 280,470 Kb > > Attempts to increase memory size within 'R' are only partly > successful. > Attempting to increase the memory to 16 Gb (16000 Mb) through using > the > following command > > memory.limit(size=16000) > > produces the error > > Error in memory.size(size) : don't be silly! : your machine has a 4 Gb > limit [not true according to the OS]. > > Reducing the number of cel files in the two initial affy batches to 16 > each (but not 33 each) did allow the combined affy batch to be created > so the combneAffyBatch() function appears to work on this platform. > > Two questions: > > 1) Does the binary version 2.2.0 of R for Windows have a built-in > memory limit of 4 Gb? - and if so, is there a way to circumvent > this limit? > > 2) Is there anything peculiar about the combineAffyBatch() function > that makes it use memory inefficiently? > > I raise the above question because even when the number of cel > files > was reduced to 33 u133a and 33 u133a_2 there was a memory allocation > error when I ran combineAffyBatch() that was the same as with the 67 + > 67 cel files (ie, "Error: cannot allocate vector of size 280,470 Kb") > and yet the memory.limit() was ~4.2 Gb and the memory.size() was only > ~1.4 Gb in a machine with 16 Gb RAM. > > Any suggestions welcomed. > > Carl Garrett > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/ > gmane.science.biology.informatics.conductor
ADD COMMENT •
0
Entering edit mode
@carleton-garrett-52
Last seen 9.6 years ago
Francois, Kasper Thanks for the info and your insights. I've been looking into setting up the computer to dual boot in Windows/Linux in order to run 'R' without the Windows constraints. (Sorry about missing the point in the Windows FAQ). Regards Carl Garrett ----- Original Message ----- From: "Kasper Daniel Hansen" <khansen@stat.berkeley.edu> To: "Carleton Garrett" <ctgarret at="" hsc.vcu.edu=""> Cc: <bioconductor at="" stat.math.ethz.ch=""> Sent: Thursday, May 18, 2006 5:25 PM Subject: Re: [BioC] Memory Issue under WinXP x64 (64 bit Windows XP) > Actually in the R for Windows FAQ (which you of course consulted before > sending your email), it says under 8.1 > "Note that building under Windows 64 is not supported as no suitable > compiler is available" > > This means you need to switch OS if you want R to use the many GB of > memory you have. > > /Kasper > > On May 18, 2006, at 2:01 PM, Carleton Garrett wrote: > >> Hi >> >> I'm currently running Bioconductor version 2.2.0 under Windows XP x64 >> with 16 Gb RAM and Virtual Memory upto 100 Gb. >> >> In trying to combine 67 Affy u133a and 67 Affy u133a_2 cel files I am >> able to form the initial affy batches using read.affybatch() but get a >> memory allocation error (below) when I try to combine them with the >> 'combineAffyBatch()' function >> >> Error: cannot allocate vector of size 280,470 Kb >> >> Attempts to increase memory size within 'R' are only partly successful. >> Attempting to increase the memory to 16 Gb (16000 Mb) through using the >> following command >> >> memory.limit(size=16000) >> >> produces the error >> >> Error in memory.size(size) : don't be silly! : your machine has a 4 Gb >> limit [not true according to the OS]. >> >> Reducing the number of cel files in the two initial affy batches to 16 >> each (but not 33 each) did allow the combined affy batch to be created >> so the combneAffyBatch() function appears to work on this platform. >> >> Two questions: >> >> 1) Does the binary version 2.2.0 of R for Windows have a built-in >> memory limit of 4 Gb? - and if so, is there a way to circumvent this >> limit? >> >> 2) Is there anything peculiar about the combineAffyBatch() function >> that makes it use memory inefficiently? >> >> I raise the above question because even when the number of cel files >> was reduced to 33 u133a and 33 u133a_2 there was a memory allocation >> error when I ran combineAffyBatch() that was the same as with the 67 + >> 67 cel files (ie, "Error: cannot allocate vector of size 280,470 Kb") >> and yet the memory.limit() was ~4.2 Gb and the memory.size() was only >> ~1.4 Gb in a machine with 16 Gb RAM. >> >> Any suggestions welcomed. >> >> Carl Garrett >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at stat.math.ethz.ch >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/ >> gmane.science.biology.informatics.conductor > > >
ADD COMMENT •
0
Entering edit mode
@carleton-garrett-52
Last seen 9.6 years ago
An embedded and charset-unspecified text was scrubbed... Name: not available Url: https://stat.ethz.ch/pipermail/bioconductor/attachments/20060530/ 765b2e4c/attachment.pl
ADD COMMENT •
0
Entering edit mode
Hi, Thanks for the bug report. Carleton Garrett wrote: > Hi > > I'm currently running R version 2.2.0 under Windows XP with 2 Gb RAM. > > I'm working through the GOstats vignette using the GOstats.Rnw file to > obtain the R code (for a description of the package see at end of this > E-mail) > > The first objective of this section of the vignette is to extract all of > the probe sets in the hgu95av2 chip that are associated with > transcription factor GO identifiers using: > > TF2 <- get("GO:0003700", hgu95av2GO2ALLPROBES) > > FYI- length(TF2) = 834 > > The next step gets the locus links (Entrez Gene IDs) associated with > these probe sets thus: > > LLs <- getLL(TF2, "hgu95av2") > > FYI - length(LLs) = 834 > > The third step gets a vector of probe sets that have been selected that > show some level of expression and some variation in expression across > samples. The data is contained within the exprSet = esetSub. > > gN = geneNames(esetSub) > > FYI - length(gN) = 2391 > > So far so good. The next objective is to get probe sets that are common > to both TF2 and gN and uses the following code: > > hv <- match(gN, TF2, 0) > > hv contains 159 non zero terms > > length(hv[!(hv ==0 )]) > [1] 159 > > HOWEVER, THESE NON ZERO TERMS ARE THE INDEX VALUES THAT LOCATE THE PROBE > SETS in TF2 - NOT in gN!!! The next part of the code is where the error > occurs and this error is propagated in the subsequent code for this section. > No need to shout, yes it does seem to be the wrong way around. We will fix it and push a fix out in the next days. Thanks again (it is often helpful to get the output of sessionInfo, when reporting bugs as it lets us make sure we are talking about the same thing). You will need to update to the most recent version of R/Bioconductor to get the benefit of our fixes as we do not have the resources to patch outdated releases. thanks again, Robert > oTF2 <- gN[hv] > > As one would expect from the above, the length(oTF2) does equal 159. > However, VERY FEW of these probe sets in oTF2 belong to the vector of > probes selected on the basis of an association with GO::0003700 - that > is - very few of them (only 14) are actually part of TF2. Thus: > > > length(oTF2[oTF2 %in% TF2]) > [1] 14 > > whereas all values of oTF2 should be in TF2. > > If one revises the above code thus: > > hvcorr <- hv[!(hv==0)] > > oTF2corr<- TF2[hvcorr] > > One again gets length(oTF2corr) = 159 but now the probe sets are in both > TF2 and gN: > > > length(TF2[TF2 %in% oTF2corr]) > [1] 159 > > > length(gN[gN %in% oTF2corr]) > [1] 159 > > Thus, all subsequent calculations in this section of the vignette that > depend on oTF2 are in error. > > This problem has probably been raised before and I just now > rediscovering it. If so, I would appreciate your pointing me to the > thread or location of the correction. nope, it hasn't > > Thanks > > Carl Garrett > > > > > ====================================================================== > Description > Package: GOstats > Title: Tools for manipulating GO and microarrays. > Version: 1.4.0 > Date: 20 Jan 2005 > Author: R. Gentleman > Description: A set of tools for interacting with GO and microarray > data. A variety of basic manipulation tools for graphs, > hypothesis testing and other simple calculations. > biocViews: Statistics, Annotation, GO, MultipleComparisons > Depends: graph, GO, annotate, RBGL, xtable, Biobase, genefilter, > multtest > Suggests: hgu95av2 (>= 1.6.0) > Maintainer: R. Gentleman <rgentlem at="" fhcrc.org=""> > License: GPL2.0 > Packaged: Wed Oct 12 21:34:06 2005; biocbuild > Built: R 2.2.0; ; 2005-10-12 21:34:10; windows > > > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > -- Robert Gentleman, PhD Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M2-B876 PO Box 19024 Seattle, Washington 98109-1024 206-667-7700 rgentlem at fhcrc.org
ADD REPLY •
0
Entering edit mode
Hi Carleton, it appears that you are using old and probably outdated versions of the software and the vignette. Please update to the latest release of Bioconductor (1.8), or the current development version. It will make more sense for this list to discuss current versions. Also, please include the output of sessionInfo() to your posting, because version numbers in particular of the data / annotation packages are important to answer your question. Best wishes Wolfgang. ------------------------------------- Wolfgang Huber European Bioinformatics Institute European Molecular Biology Laboratory Cambridge CB10 1SD England Phone: +44 1223 494642 Fax: +44 1223 494486 Http: www.ebi.ac.uk/huber ------------------------------------- Carleton Garrett wrote: > Hi > > I'm currently running R version 2.2.0 under Windows XP with 2 Gb RAM. > > I'm working through the GOstats vignette using the GOstats.Rnw file to > obtain the R code (for a description of the package see at end of this > E-mail) > > The first objective of this section of the vignette is to extract all of > the probe sets in the hgu95av2 chip that are associated with > transcription factor GO identifiers using: > > TF2 <- get("GO:0003700", hgu95av2GO2ALLPROBES) > > FYI- length(TF2) = 834 > > The next step gets the locus links (Entrez Gene IDs) associated with > these probe sets thus: > > LLs <- getLL(TF2, "hgu95av2") > > FYI - length(LLs) = 834 > > The third step gets a vector of probe sets that have been selected that > show some level of expression and some variation in expression across > samples. The data is contained within the exprSet = esetSub. > > gN = geneNames(esetSub) > > FYI - length(gN) = 2391 > > So far so good. The next objective is to get probe sets that are common > to both TF2 and gN and uses the following code: > > hv <- match(gN, TF2, 0) > > hv contains 159 non zero terms > > length(hv[!(hv ==0 )]) > [1] 159 > > HOWEVER, THESE NON ZERO TERMS ARE THE INDEX VALUES THAT LOCATE THE PROBE > SETS in TF2 - NOT in gN!!! The next part of the code is where the error > occurs and this error is propagated in the subsequent code for this section. > > oTF2 <- gN[hv] > > As one would expect from the above, the length(oTF2) does equal 159. > However, VERY FEW of these probe sets in oTF2 belong to the vector of > probes selected on the basis of an association with GO::0003700 - that > is - very few of them (only 14) are actually part of TF2. Thus: > > > length(oTF2[oTF2 %in% TF2]) > [1] 14 > > whereas all values of oTF2 should be in TF2. > > If one revises the above code thus: > > hvcorr <- hv[!(hv==0)] > > oTF2corr<- TF2[hvcorr] > > One again gets length(oTF2corr) = 159 but now the probe sets are in both > TF2 and gN: > > > length(TF2[TF2 %in% oTF2corr]) > [1] 159 > > > length(gN[gN %in% oTF2corr]) > [1] 159 > > Thus, all subsequent calculations in this section of the vignette that > depend on oTF2 are in error. > > This problem has probably been raised before and I just now > rediscovering it. If so, I would appreciate your pointing me to the > thread or location of the correction. > > Thanks > > Carl Garrett > > > > > ====================================================================== > Description > Package: GOstats > Title: Tools for manipulating GO and microarrays. > Version: 1.4.0 > Date: 20 Jan 2005 > Author: R. Gentleman > Description: A set of tools for interacting with GO and microarray > data. A variety of basic manipulation tools for graphs, > hypothesis testing and other simple calculations. > biocViews: Statistics, Annotation, GO, MultipleComparisons > Depends: graph, GO, annotate, RBGL, xtable, Biobase, genefilter, > multtest > Suggests: hgu95av2 (>= 1.6.0) > Maintainer: R. Gentleman <rgentlem at="" fhcrc.org=""> > License: GPL2.0 > Packaged: Wed Oct 12 21:34:06 2005; biocbuild > Built: R 2.2.0; ; 2005-10-12 21:34:10; windows
ADD REPLY •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 513 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6