Hi, all; I know there has been a lot of discussions on memory usage in R. However, I have some odd situation here. Basically, I have a rare opportunity to run R in a system with 64GB memory without any limit on memory usage for any person or process. However, I encountered the memory problem error message like this: Error: cannot allocate vector of size 594075 Kb I got this error message while I was trying to apply dChip preprocessing procedures for 150 Affymetrix U133v2 chips that has > 22,000 probe sets on them. The actual codes I ran was like this: > Data <- ReadAffy(filenames = paste(HOME, "CelData/", fname, sep="")) > mem.limits() nsize vsize NA NA > gc() used (Mb) gc trigger (Mb) Ncells 530216 14.2 899071 24.1 Vcells 76196137 581.4 243983468 1861.5 > eset <- expresso(Data, normalize.method="invariantset", bg.correct=FALSE, pmc\ orrect.method="pmonly", summary.method="liwong") normalization: invariantset PM/MM correction : pmonly expression values: liwong normalizing...Error: cannot allocate vector of size 594075 Kb > gc() used (Mb) gc trigger (Mb) Ncells 797983 21.4 1710298 45.7 Vcells 76716811 585.4 305954068 2334.3 > object.size(Data) [1] 608355664 > memory.profile() NILSXP SYMSXP LISTSXP CLOSXP ENVSXP PROMSXP LANGSXP 1 30484 372373 4845 420 180 127274 SPECIALSXP BUILTINSXP CHARSXP LGLSXP INTSXP 203 1168 111434 5296 0 0 44649 REALSXP CPLXSXP STRSXP DOTSXP ANYSXP VECSXP EXPRSXP 13382 9 60173 0 0 26002 0 BCODESXP EXTPTRSXP WEAKREFSXP 0 106 0 Although I have no idea of memory allocation in R, apparently something's wrong with this. The memory problem must have nothing to do with physical memory. My question is this. Is this memory problem due to some non- optimal configuration of memory usage? If so, then what will be the optimal configuration for this? If not, then there must be problems on actual implementations of functions I used here, right? The reason I am asking this is that, according to the reference manual, the error message I got can be brought up by roughly three reasons. First, when the system is unable to provide the R requested memory. Second, when the requested memory size exceeds the address-space limit for a process. Finally, when the length of a vector is larger than 2^31-1. I wonder the problem has anything to do with the third case. (If so, then I think I am hopeless unless the internal implementations change...)

Dear Bioconductorians, My sincere apologies if this email is duplicated.I have searched the mailing list and could not find a way to solve my problem: Find out the probe intensity (PM adn MM) for HG_U95Av2 which fall in (200 to 20000 units) range and extract their respective PM sequences. I have been able to filter the probes but how do i extract the sequence data?! I am including the small script which i have written for filtering, now how do i extract sequence information. Thanks in advance. Hrishi library(affy) Data <- ReadAffy() pmi <- pm(Data) mmi <- mm(Data) pbn <- probeNames(Data) rng.pmi <- apply(pmi,1,range) rng.mmi <- apply(mmi,1,range) out.of.boundspm <- ((rng.pmi[1,] >=200) & (rng.pmi[1,] <=20000)) out.of.boundsmm <- ((rng.mmi[1,] >=200) & (rng.mmi[1,] <=20000)) in.bounds <- (out.of.boundspm & out.of.boundsmm) length(pmi[,1]) ac1 <- 1:201800 ac2 <- ac1[in.bounds] h.pbn <- pbn[ac2] h.pmi <- pmi[ac2,] h.mmi <- mmi[ac2,] normal.data <- cbind(h.pbn,h.pmi,h.mmi) names(normal.data) <- c('probesetnames','PMintensity','MMintensity') write.table(normal.data, file="normaldata.txt",quote =F,row.names=F,col.names=T,sep = " ") __________________________________ Send holiday email and support a worthy cause. Do good.

Tae-Hoon Chung <thchung@tgen.org> writes: > Hi, all; > > I know there has been a lot of discussions on memory usage in R. > However, I have some odd situation here. Basically, I have a rare > opportunity to run R in a system with 64GB memory without any limit on > memory usage for any person or process. However, I encountered the memory > problem error message like this: > > Error: cannot allocate vector of size 594075 Kb .... > Although I have no idea of memory allocation in R, apparently something's > wrong with this. The memory problem must have nothing to do with physical > memory. My question is this. Is this memory problem due to some non- optimal > configuration of memory usage? If so, then what will be the optimal > configuration for this? If not, then there must be problems on actual > implementations of functions I used here, right? The reason I am asking this > is that, according to the reference manual, the error message I got can be > brought up by roughly three reasons. First, when the system is unable to > provide the R requested memory. Second, when the requested memory size > exceeds the address-space limit for a process. Finally, when the length of a > vector is larger than 2^31-1. Hmm, the length issue should not kick in before the length exceeds 2 billion or so and you are not beyond 75 or 150 million (counting 8 or 4 bytes per elements). > I wonder the problem has anything to do with > the third case. (If so, then I think I am hopeless unless the internal > implementations change...) Well, revolutionaries often find themselves just below the cutting edge... Just a sanity check: this is using a 64-bit compiled R on a 64-bit operating system, right? -- O__ ---- Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - (p.dalgaard@biostat.ku.dk) FAX: (+45) 35327907