IDAT files upload error, MethyLumiM object: IlluminaHumanMethylation450k
2
0
Entering edit mode
@magdalena-wozniak-5004
Last seen 9.6 years ago
Hi, I must admit I'm quite new to both methylation data analysis and R. I am currently analyzing IlluminaHumanMethylation450k data starting from IDAT files. The command: > barcodes <- c('6055432006_R04C01',+ '6055432006_R04C01',+ '6042316163_R06C02',+ '6042316163_R06C02',...{other barcodes omitted}) works well when I upload only few samples but gives the following error when I am trying to upload all the samples I want to analyse: > lumi450k <- methylumIDAT (barcodes)Warning: filtering out duplicate barcodes0 HumanMethylation27 samples found294 HumanMethylation450 samples foundError in annotation(NChannelSet) : error in evaluating the argument 'object' in selecting a method for function 'annotation': Error: cannot allocate vector of size 8.0 Mb How could I increase the size allocated to a vector? When I submit only 2 samples the initial code works well and I get the MethylumiSetObject, however, I don't know how to change beta values to m-values, and MethyLumiSet to MethyLumiM object as when I want to do color balance adjustment, I obtain the following error: > lumiMethy.c.adj <- lumiMethyC(lumi450k)Error in lumiMethyC(lumi450k) : The object should be class "MethyLumiM" inherited! I would be very grateful for help. I attach session info below: > sessionInfo()R version 2.14.0 (2011-10-31) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C [5] LC_TIME=English_United Kingdom.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] IlluminaHumanMethylation450k.db_1.4.6 org.Hs.eg.db_2.6.4 [3] RSQLite_0.11.1 DBI_0.2-5 [5] AnnotationDbi_1.16.9 matrixStats_0.4.2 [7] R.methodsS3_1.2.1 limma_3.10.0 [9] lumi_2.6.0 nleqslv_1.9.1 [11] methylumi_2.0.1 Biobase_2.14.0 loaded via a namespace (and not attached): [1] affy_1.32.0 affyio_1.22.0 annotate_1.32.0 BiocInstaller_1.2.1 [5] grid_2.14.0 hdrcde_2.15 IRanges_1.12.4 KernSmooth_2.23-7 [9] lattice_0.20-0 MASS_7.3-16 Matrix_1.0-2 mgcv_1.7-11 [13] nlme_3.1-102 preprocessCore_1.16.0 tools_2.14.0 xtable_1.6-0 [17] zlibbioc_1.0.0 > Best wishes, Magdalena [[alternative HTML version deleted]]
• 1.6k views
ADD COMMENT •
0
Entering edit mode
Tim Triche ★ 4.2k
@tim-triche-3561
Last seen 3.6 years ago
United States
How much RAM do you have on your machine? It looks like it is running out... Unfortunately the current approach to solving this problem (both in minfi and methylumi) is "buy more RAM".... :-( On Fri, Dec 9, 2011 at 6:50 AM, Magdalena Wozniak < wozniak.magdalena@gmail.com> wrote: > Hi, > > I must admit I'm quite new to both methylation data analysis and R. I am > currently analyzing IlluminaHumanMethylation450k data starting from IDAT > files. > The command: > > > barcodes <- c('6055432006_R04C01',+ '6055432006_R04C01',+ '6042316163_R06C02',+ > '6042316163_R06C02',...{other barcodes omitted}) > > works well when I upload only few samples but gives the following error > when I am trying to upload all the samples I want to analyse: > > > lumi450k <- methylumIDAT (barcodes)Warning: filtering out duplicate > barcodes0 HumanMethylation27 samples found294 HumanMethylation450 samples > foundError in annotation(NChannelSet) : > error in evaluating the argument 'object' in selecting a method for > function 'annotation': Error: cannot allocate vector of size 8.0 Mb > > How could I increase the size allocated to a vector? > > When I submit only 2 samples the initial code works well and I get the > MethylumiSetObject, however, I don't know how to change beta values to > m-values, and MethyLumiSet to MethyLumiM object as when I want to do > color balance adjustment, I obtain the following error: > > > > lumiMethy.c.adj <- lumiMethyC(lumi450k)Error in lumiMethyC(lumi450k) : > The object should be class "MethyLumiM" inherited! > > > I would be very grateful for help. I attach session info below: > > > sessionInfo()R version 2.14.0 (2011-10-31) > Platform: i386-pc-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United > Kingdom.1252 > [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C > [5] LC_TIME=English_United Kingdom.1252 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] IlluminaHumanMethylation450k.db_1.4.6 org.Hs.eg.db_2.6.4 > [3] RSQLite_0.11.1 DBI_0.2-5 > [5] AnnotationDbi_1.16.9 matrixStats_0.4.2 > [7] R.methodsS3_1.2.1 limma_3.10.0 > [9] lumi_2.6.0 nleqslv_1.9.1 > [11] methylumi_2.0.1 Biobase_2.14.0 > > loaded via a namespace (and not attached): > [1] affy_1.32.0 affyio_1.22.0 annotate_1.32.0 > BiocInstaller_1.2.1 > [5] grid_2.14.0 hdrcde_2.15 IRanges_1.12.4 > KernSmooth_2.23-7 > [9] lattice_0.20-0 MASS_7.3-16 Matrix_1.0-2 > mgcv_1.7-11 > [13] nlme_3.1-102 preprocessCore_1.16.0 tools_2.14.0 > xtable_1.6-0 > [17] zlibbioc_1.0.0 > > > > > > > Best wishes, > Magdalena > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > -- If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is. John von Neumann<http: www-groups.dcs.st-="" and.ac.uk="" ~history="" biographies="" von_neumann.html=""> [[alternative HTML version deleted]]
ADD COMMENT •
0
Entering edit mode
Tim, do you have any metrics to the RAM issue? eg: is it 450kb per array/sample for instance? cheers On 10/12/2011, at 7:21 AM, Tim Triche, Jr. wrote: > How much RAM do you have on your machine? It looks like it is running > out... > > Unfortunately the current approach to solving this problem (both in minfi > and methylumi) is "buy more RAM".... :-( > > > > On Fri, Dec 9, 2011 at 6:50 AM, Magdalena Wozniak < > wozniak.magdalena at gmail.com> wrote: > >> Hi, >> >> I must admit I'm quite new to both methylation data analysis and R. I am >> currently analyzing IlluminaHumanMethylation450k data starting from IDAT >> files. >> The command: >> >>> barcodes <- c('6055432006_R04C01',+ '6055432006_R04C01',+ '6042316163_R06C02',+ >> '6042316163_R06C02',...{other barcodes omitted}) >> >> works well when I upload only few samples but gives the following error >> when I am trying to upload all the samples I want to analyse: >> >>> lumi450k <- methylumIDAT (barcodes)Warning: filtering out duplicate >> barcodes0 HumanMethylation27 samples found294 HumanMethylation450 samples >> foundError in annotation(NChannelSet) : >> error in evaluating the argument 'object' in selecting a method for >> function 'annotation': Error: cannot allocate vector of size 8.0 Mb >> >> How could I increase the size allocated to a vector? >> >> When I submit only 2 samples the initial code works well and I get the >> MethylumiSetObject, however, I don't know how to change beta values to >> m-values, and MethyLumiSet to MethyLumiM object as when I want to do >> color balance adjustment, I obtain the following error: >> >> >>> lumiMethy.c.adj <- lumiMethyC(lumi450k)Error in lumiMethyC(lumi450k) : >> The object should be class "MethyLumiM" inherited! >> >> >> I would be very grateful for help. I attach session info below: >> >>> sessionInfo()R version 2.14.0 (2011-10-31) >> Platform: i386-pc-mingw32/i386 (32-bit) >> >> locale: >> [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United >> Kingdom.1252 >> [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C >> [5] LC_TIME=English_United Kingdom.1252 >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] IlluminaHumanMethylation450k.db_1.4.6 org.Hs.eg.db_2.6.4 >> [3] RSQLite_0.11.1 DBI_0.2-5 >> [5] AnnotationDbi_1.16.9 matrixStats_0.4.2 >> [7] R.methodsS3_1.2.1 limma_3.10.0 >> [9] lumi_2.6.0 nleqslv_1.9.1 >> [11] methylumi_2.0.1 Biobase_2.14.0 >> >> loaded via a namespace (and not attached): >> [1] affy_1.32.0 affyio_1.22.0 annotate_1.32.0 >> BiocInstaller_1.2.1 >> [5] grid_2.14.0 hdrcde_2.15 IRanges_1.12.4 >> KernSmooth_2.23-7 >> [9] lattice_0.20-0 MASS_7.3-16 Matrix_1.0-2 >> mgcv_1.7-11 >> [13] nlme_3.1-102 preprocessCore_1.16.0 tools_2.14.0 >> xtable_1.6-0 >> [17] zlibbioc_1.0.0 >> >> >>> >> >> >> Best wishes, >> Magdalena >> >> [[alternative HTML version deleted]] >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: >> http://news.gmane.org/gmane.science.biology.informatics.conductor >> > > > > -- > If people do not believe that mathematics is simple, > it is only because they do not realize how complicated life is. John von > Neumann<http: www-groups.dcs.st-="" and.ac.uk="" ~history="" biographies="" von_neumann.html=""> > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > > If you have any question, please contact MCRI IT Helpdesk for further assistance. > ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com
ADD REPLY •
0
Entering edit mode
14MB of IDATs per sample (7MB Cy3 + 7MB Cy5); Kasper has said that, in 'minfi', he got it down to about 1MB per sample after processing. A MethyLumiSet that I'm looking at right now with 192 samples and about 6,000 covariates per sample, 20 or so per feature, clocks in at 8.5MB per sample with all of the feature and sample annotations. I assume that this could be optimized somewhat. I asked Kasper what he thought about optimizing for small memory machines and the answer was (in so many words) "you should have a very good reason to bother with that". The memory footprint (uncompressed) is the real problem. In theory, using an environment for the assayData should reduce that. In practice... One thing that could probably be optimized for memory consumption is the way that the samples are converted into assayData after being read off the disk. I almost always work on a machine with 8GB or more of RAM; on machines with 32GB, I have processed up to 595 samples at a time, keeping all of the data in both channels for preprocessing purposes. (this is why I added a parallel=TRUE option to methylumIDAT). A vignette covering all of this and a bit more, working on some reasonable-sized (a couple hundred) samples, should be up sometime next week. On Fri, Dec 9, 2011 at 12:53 PM, Nicholas Wong <nick.wong@mcri.edu.au>wrote: > Tim, do you have any metrics to the RAM issue? > > eg: is it 450kb per array/sample for instance? > > cheers > > > > On 10/12/2011, at 7:21 AM, Tim Triche, Jr. wrote: > > > How much RAM do you have on your machine? It looks like it is running > > out... > > > > Unfortunately the current approach to solving this problem (both in minfi > > and methylumi) is "buy more RAM".... :-( > > > > > > > > On Fri, Dec 9, 2011 at 6:50 AM, Magdalena Wozniak < > > wozniak.magdalena@gmail.com> wrote: > > > >> Hi, > >> > >> I must admit I'm quite new to both methylation data analysis and R. I am > >> currently analyzing IlluminaHumanMethylation450k data starting from IDAT > >> files. > >> The command: > >> > >>> barcodes <- c('6055432006_R04C01',+ '6055432006_R04C01',+ '6042316163 > _R06C02',+ > >> '6042316163_R06C02',...{other barcodes omitted}) > >> > >> works well when I upload only few samples but gives the following error > >> when I am trying to upload all the samples I want to analyse: > >> > >>> lumi450k <- methylumIDAT (barcodes)Warning: filtering out duplicate > >> barcodes0 HumanMethylation27 samples found294 HumanMethylation450 > samples > >> foundError in annotation(NChannelSet) : > >> error in evaluating the argument 'object' in selecting a method for > >> function 'annotation': Error: cannot allocate vector of size 8.0 Mb > >> > >> How could I increase the size allocated to a vector? > >> > >> When I submit only 2 samples the initial code works well and I get the > >> MethylumiSetObject, however, I don't know how to change beta values to > >> m-values, and MethyLumiSet to MethyLumiM object as when I want to do > >> color balance adjustment, I obtain the following error: > >> > >> > >>> lumiMethy.c.adj <- lumiMethyC(lumi450k)Error in lumiMethyC(lumi450k) : > >> The object should be class "MethyLumiM" inherited! > >> > >> > >> I would be very grateful for help. I attach session info below: > >> > >>> sessionInfo()R version 2.14.0 (2011-10-31) > >> Platform: i386-pc-mingw32/i386 (32-bit) > >> > >> locale: > >> [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United > >> Kingdom.1252 > >> [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C > >> [5] LC_TIME=English_United Kingdom.1252 > >> > >> attached base packages: > >> [1] stats graphics grDevices utils datasets methods base > >> > >> other attached packages: > >> [1] IlluminaHumanMethylation450k.db_1.4.6 org.Hs.eg.db_2.6.4 > >> [3] RSQLite_0.11.1 DBI_0.2-5 > >> [5] AnnotationDbi_1.16.9 matrixStats_0.4.2 > >> [7] R.methodsS3_1.2.1 limma_3.10.0 > >> [9] lumi_2.6.0 nleqslv_1.9.1 > >> [11] methylumi_2.0.1 Biobase_2.14.0 > >> > >> loaded via a namespace (and not attached): > >> [1] affy_1.32.0 affyio_1.22.0 annotate_1.32.0 > >> BiocInstaller_1.2.1 > >> [5] grid_2.14.0 hdrcde_2.15 IRanges_1.12.4 > >> KernSmooth_2.23-7 > >> [9] lattice_0.20-0 MASS_7.3-16 Matrix_1.0-2 > >> mgcv_1.7-11 > >> [13] nlme_3.1-102 preprocessCore_1.16.0 tools_2.14.0 > >> xtable_1.6-0 > >> [17] zlibbioc_1.0.0 > >> > >> > >>> > >> > >> > >> Best wishes, > >> Magdalena > >> > >> [[alternative HTML version deleted]] > >> > >> _______________________________________________ > >> Bioconductor mailing list > >> Bioconductor@r-project.org > >> https://stat.ethz.ch/mailman/listinfo/bioconductor > >> Search the archives: > >> http://news.gmane.org/gmane.science.biology.informatics.conductor > >> > > > > > > > > -- > > If people do not believe that mathematics is simple, > > it is only because they do not realize how complicated life is. John von > > Neumann< > http://www-groups.dcs.st- and.ac.uk/~history/Biographies/Von_Neumann.html> > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > > > > ______________________________________________________________________ > > This email has been scanned by the Symantec Email Security.cloud service. > > For more information please visit http://www.symanteccloud.com > > > > If you have any question, please contact MCRI IT Helpdesk for further > assistance. > > ______________________________________________________________________ > > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ > -- If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is. John von Neumann<http: www-groups.dcs.st-="" and.ac.uk="" ~history="" biographies="" von_neumann.html=""> [[alternative HTML version deleted]]
ADD REPLY •
0
Entering edit mode
@magdalena-wozniak-5004
Last seen 9.6 years ago
> Hi Tim, > > Thanks for your answer. I currently have 4GB of RAM but have access to > 12GB machine so will try to run it there. > Do you know whether there is a command to change change beta values to > m-values, and MethyLumiSet to MethyLumiM object to avoid "The object > should be class "MethyLumiM" inherited!" error? > > Thanks a lot. > Best wishes, > Magdalena > > > 2011/12/9 Tim Triche, Jr. <tim.triche@gmail.com> > > How much RAM do you have on your machine? It looks like it is running >> out... >> >> Unfortunately the current approach to solving this problem (both in minfi >> and methylumi) is "buy more RAM".... :-( >> >> >> >> On Fri, Dec 9, 2011 at 6:50 AM, Magdalena Wozniak < >> wozniak.magdalena@gmail.com> wrote: >> >>> Hi, >>> >>> I must admit I'm quite new to both methylation data analysis and R. I am >>> currently analyzing IlluminaHumanMethylation450k data starting from IDAT >>> files. >>> The command: >>> >>> > barcodes <- c('6055432006_R04C01',+ '6055432006_R04C01',+ '6042316163_R06C02',+ >>> '6042316163_R06C02',...{other barcodes omitted}) >>> >>> >>> works well when I upload only few samples but gives the following error >>> when I am trying to upload all the samples I want to analyse: >>> >>> > lumi450k <- methylumIDAT (barcodes)Warning: filtering out duplicate >>> barcodes0 HumanMethylation27 samples found294 HumanMethylation450 samples >>> foundError in annotation(NChannelSet) : >>> >>> error in evaluating the argument 'object' in selecting a method for >>> function 'annotation': Error: cannot allocate vector of size 8.0 Mb >>> >>> How could I increase the size allocated to a vector? >>> >>> When I submit only 2 samples the initial code works well and I get the >>> MethylumiSetObject, however, I don't know how to change beta values to >>> m-values, and MethyLumiSet to MethyLumiM object as when I want to do >>> color balance adjustment, I obtain the following error: >>> >>> >>> > lumiMethy.c.adj <- lumiMethyC(lumi450k)Error in lumiMethyC(lumi450k) : >>> >>> The object should be class "MethyLumiM" inherited! >>> >>> >>> I would be very grateful for help. I attach session info below: >>> >>> > sessionInfo()R version 2.14.0 (2011-10-31) >>> >>> Platform: i386-pc-mingw32/i386 (32-bit) >>> >>> locale: >>> [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United >>> Kingdom.1252 >>> [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C >>> [5] LC_TIME=English_United Kingdom.1252 >>> >>> attached base packages: >>> [1] stats graphics grDevices utils datasets methods base >>> >>> other attached packages: >>> [1] IlluminaHumanMethylation450k.db_1.4.6 org.Hs.eg.db_2.6.4 >>> [3] RSQLite_0.11.1 DBI_0.2-5 >>> [5] AnnotationDbi_1.16.9 matrixStats_0.4.2 >>> [7] R.methodsS3_1.2.1 limma_3.10.0 >>> [9] lumi_2.6.0 nleqslv_1.9.1 >>> [11] methylumi_2.0.1 Biobase_2.14.0 >>> >>> loaded via a namespace (and not attached): >>> [1] affy_1.32.0 affyio_1.22.0 annotate_1.32.0 >>> BiocInstaller_1.2.1 >>> [5] grid_2.14.0 hdrcde_2.15 IRanges_1.12.4 >>> KernSmooth_2.23-7 >>> [9] lattice_0.20-0 MASS_7.3-16 Matrix_1.0-2 >>> mgcv_1.7-11 >>> [13] nlme_3.1-102 preprocessCore_1.16.0 tools_2.14.0 >>> xtable_1.6-0 >>> [17] zlibbioc_1.0.0 >>> >>> >>> > >>> >>> >>> Best wishes, >>> Magdalena >>> >>> [[alternative HTML version deleted]] >>> >>> _______________________________________________ >>> Bioconductor mailing list >>> Bioconductor@r-project.org >>> https://stat.ethz.ch/mailman/listinfo/bioconductor >>> Search the archives: >>> http://news.gmane.org/gmane.science.biology.informatics.conductor >>> >> >> >> >> -- >> If people do not believe that mathematics is simple, >> it is only because they do not realize how complicated life is. John von >> Neumann<http: www-groups.dcs.st-="" and.ac.uk="" %7ehistory="" biographies="" von_neumann.html=""> >> > > [[alternative HTML version deleted]]
ADD COMMENT •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 684 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6