The maximum number of hgu133plus2 arrays that can be processed using expresso-like tools in R on a 32-bit Windows machine is about 30. The maximum number of hgu95a arrays that can be processed using expresso-like tools in R on a 32-bit Windows machine is about 90. The rough computation is simple: each hgu133plus 2 array contains 1164*1164 = 1,354,896 probes. Allowing at least 10 bytes to store the probe intenisty as a double precision float, the probe-level data for one array requires about 13.5 MB. Thus, 30 arrays requires at least 400 MB just to store the probe level data. Because R implements pass-by-value rather than pass-by-reference (and because BioConductor has followed the usual good programming practice of writing and reusing modular functions), you can easily get as many as 3 copies of this data structure floating around in memory at the same time, putting us at the 1.2 GB level. This doesn't include any additional overhead for other parts of the AffyBatch structure or the amount of space needed to store the code objects, and anything else in your current session. Now, a 32-bit machine is limited to adressing 4GB of RAM, but as far as I can tell, Windows won't let a single program see more than 2GB, so you can begin to see why expresso can't handle more than about 30 of these arrays at once. The specialized functions like justrma get around this problem by passing control over to routines written in C, which presimably use pass-by-reference to avoid making extra copies of the data. -- Kevin --On Wednesday, January 11, 2006 10:30 AM +0100 Benjamin Otto <b.otto at="" uke.uni-hamburg.de=""> wrote: > Hi Nick, > > I'm currently working on that chip type myself on a system with 512MB RAM > and had to increase the memory assigned to R too. However > "--max-mem-size=1024Mb" seems to be ok in my case. Besides the problem of > how much memory is necessary for the hgu133plus2 maybe somebody out there > could answer me this two related questions: > * I'm running R under WinXp AND under Linux on the same machine. However > there has never been any memory problem in linux. So is the memory > allocation or the assignment of the max. memory size that can be used by > R different in the two systems? > * With my 512MB RAM system the old maximum memory value under WinXP was > 512MB. Is this just a coincidence or does this maximum value rise to > 1024MB if I upgrade my system to 1024MB RAM? > > regards, > Benjamin > >> -----Original Message----- >> From: bioconductor-bounces at stat.math.ethz.ch >> [mailto:bioconductor-bounces at stat.math.ethz.ch]On Behalf Of James W. >> MacDonald >> Sent: 10 January 2006 20:02 >> To: Ettinger, Nicholas >> Cc: bioconductor at stat.math.ethz.ch >> Subject: Re: [BioC] affy & problems with memory >> >> >> Ettinger, Nicholas wrote: >> > Hello all! >> > >> > I have 16 Affy Hg-U133plus2 cel files that I am trying to analyze. I >> > am having difficulties generating expression data because of memory >> > problems. I am working on a Dell desktop running WinXP. I can >> > successfully generate expression values if I use "RMA" but if I try >> > MAS5 or GCRMA or LiWong instead then I consistently get an error >> > message with "Cannot allocate vector of size 169362 Kb." To try to >> > increase available memory I have already done two things: (a) I >> > right-clicked on the R icon and changed the box labeled 'Target' to >> > read "C:\Program Files\R\R-2.2.0\bin\Rgui.exe" --sdi >> > --max-mem-size=2000M" and (b) I have increased the virtual memory >> > Total Page File size to 2000MB. But it still runs out of memory. >> > >> > Help! Any suggestions -- I don't really have access to any bigger >> > computers? >> >> You don't need a bigger computer, you need more RAM, which is pretty >> cheap these days. I can't imagine it would take more than 512 Mb to do >> 16 U133 plus 2 chips, so you might look into adding a 512 Mb stick. >> >> Best, >> >> Jim >> >> >> > >> > Thanks. >> > ---Nick >> > >> > _______________________________________________ >> > Bioconductor mailing list >> > Bioconductor at stat.math.ethz.ch >> > https://stat.ethz.ch/mailman/listinfo/bioconductor >> >> >> -- >> James W. MacDonald >> Affymetrix and cDNA Microarray Core >> University of Michigan Cancer Center >> 1500 E. Medical Center Drive >> 7410 CCGC >> Ann Arbor MI 48109 >> 734-647-5623 >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at stat.math.ethz.ch >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor

I forgot: If concerning GCRMA and RMA you could use justGCRMA and justRMA which are less memory consuming... regards, Benjamin > > > Hi Nick, > > I'm currently working on that chip type myself on a system with 512MB RAM > and had to increase the memory assigned to R too. However > "--max-mem-size=1024Mb" seems to be ok in my case. Besides the problem of > how much memory is necessary for the hgu133plus2 maybe somebody out there > could answer me this two related questions: > * I'm running R under WinXp AND under Linux on the same machine. However > there has never been any memory problem in linux. So is the memory > allocation or the assignment of the max. memory size that can be > used by R > different in the two systems? > * With my 512MB RAM system the old maximum memory value under WinXP was > 512MB. Is this just a coincidence or does this maximum value rise > to 1024MB > if I upgrade my system to 1024MB RAM? > > regards, > Benjamin > > > -----Original Message----- > > From: bioconductor-bounces at stat.math.ethz.ch > > [mailto:bioconductor-bounces at stat.math.ethz.ch]On Behalf Of James W. > > MacDonald > > Sent: 10 January 2006 20:02 > > To: Ettinger, Nicholas > > Cc: bioconductor at stat.math.ethz.ch > > Subject: Re: [BioC] affy & problems with memory > > > > > > Ettinger, Nicholas wrote: > > > Hello all! > > > > > > I have 16 Affy Hg-U133plus2 cel files that I am trying to > analyze. I am > > > having difficulties generating expression data because of memory > > > problems. I am working on a Dell desktop running WinXP. I can > > > successfully generate expression values if I use "RMA" but if > I try MAS5 > > > or GCRMA or LiWong instead then I consistently get an error > message with > > > "Cannot allocate vector of size 169362 Kb." To try to increase > > > available memory I have already done two things: (a) I > right-clicked on > > > the R icon and changed the box labeled 'Target' to read "C:\Program > > > Files\R\R-2.2.0\bin\Rgui.exe" --sdi --max-mem-size=2000M" and (b) I > > > have increased the virtual memory Total Page File size to 2000MB. But > > > it still runs out of memory. > > > > > > Help! Any suggestions -- I don't really have access to any bigger > > > computers? > > > > You don't need a bigger computer, you need more RAM, which is pretty > > cheap these days. I can't imagine it would take more than 512 Mb to do > > 16 U133 plus 2 chips, so you might look into adding a 512 Mb stick. > > > > Best, > > > > Jim > > > > > > > > > > Thanks. > > > ---Nick > > > > > > _______________________________________________ > > > Bioconductor mailing list > > > Bioconductor at stat.math.ethz.ch > > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > > > > > -- > > James W. MacDonald > > Affymetrix and cDNA Microarray Core > > University of Michigan Cancer Center > > 1500 E. Medical Center Drive > > 7410 CCGC > > Ann Arbor MI 48109 > > 734-647-5623 > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor at stat.math.ethz.ch > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor >

Hi, would it be a solution to use RMAexpress to obtain expression values for a good amount of arrays and then import them into your R session? I haven?t got much experience on this program but it is said on the RMAexpress webpage that the program can process upto 250 arrays simultaneously ( I guess not U133plus) and in the history section it is stated that the 0.4alpha3 version can run 200 arrays. Any information on U133plus? David > My apologies for the previous (empty) mail, it slipped me :) > > > I'm currently working on that chip type myself on a system with 512MB RAM > > and had to increase the memory assigned to R too. However > > "--max-mem-size=1024Mb" seems to be ok in my case. Besides the problem of > > how much memory is necessary for the hgu133plus2 maybe somebody out there > > could answer me this two related questions: > > * I'm running R under WinXp AND under Linux on the same machine. However > > there has never been any memory problem in linux. So is the memory > > allocation or the assignment of the max. memory size that can be used by R > > different in the two systems? > > This is due to the much more flexible memory management of Linux as > opposed to Windows. Theoretical limit on a 32-bit Windows machine as > 2Gb; 4 Gb under Linux. Linux has also a more flexible way of swapping > (finding memory on your local harddrive) then Windows (the /swp > filesystem). > > > > * With my 512MB RAM system the old maximum memory value under WinXP was > > 512MB. Is this just a coincidence or does this maximum value rise to 1024MB > > if I upgrade my system to 1024MB RAM? > > No, this is not a coincidence since the maximum value is related to the > amount available on your system. > > Cheers, > R. > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor >