Hi Thanks to respond. I checked and the file path is correctly because i could open a similar file but about 38 Mb, that's why i thought that the memory was the problem, here is: > control <- readFastq("ControlB_2A8_130_50.250k.fq", qualityType="SFastqQuality") > seedling <- readFastq("s6_plantula.fq", qualityType="SFastqQuality") Error en .local(dirPath, pattern, ...) : cannot open file s6_plantula.fq > control class: ShortReadQ length: 250000 reads; width: 36 cycles Here the sessionInfo(), but i'm not sure that it has to give me the info about the hardware andf software > sessionInfo() R version 2.10.1 (2009-12-14) i486-pc-linux-gnu locale: [1] LC_CTYPE=es_MX.utf8 LC_NUMERIC=C [3] LC_TIME=es_MX.utf8 LC_COLLATE=es_MX.utf8 [5] LC_MONETARY=C LC_MESSAGES=es_MX.utf8 [7] LC_PAPER=es_MX.utf8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=es_MX.utf8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] ShortRead_1.4.0 lattice_0.18-3 BSgenome_1.14.2 Biostrings_2.14.12 [5] IRanges_1.4.16 loaded via a namespace (and not attached): [1] Biobase_2.6.1 grid_2.10.1 hwriter_1.2 On Fri, Aug 20, 2010 at 11:23 AM, Martin Morgan <mtmorgan@fhcrc.org> wrote: > On 08/20/2010 08:19 AM, Marco Ortiz wrote: > > * Hi All, ** ** I am new to the world of R and Bioconductor and I had > > the** following error when I try to read a fastaq file of 2 Gb**** > > First I try to run it in Linux, but there i had an error that tell me > > * > > > > * <** > > > > Error: (dirPath, pattern, ...) cannot open file > > > > * > > > > * Then i try to run it in Windows and there i had this error* > > > > * > > * > > > > * > > * > > > > * > library(ShortRead)* > > > > *** > plantula <- readFastq("s6_plantula.fq", > qualityType="SFastqQuality")* > > > > *** Error: cannot allocate vector of size 649.8 Mb* > > This is likely limited memory problem; R is saying 'I allocated some > memory, and now I'm trying to allocate 649.8Mb more, and that additional > memory allocation fails'. Memory limits on 32-bit windows systems are > hard to get around; you're better using a 64-bit Windows or Linux > system. Your Linux problem seems really straight-forward -- you haven't > specified the file path correctly. Your R likely has command completion, so > > readFastq("~/ > > and then use the tab key to complete the path. > > Please include sessionInfo() when you're posting to the list; it makes > clear what software and hardware you're using. > > Martin > > > > * > > * > > > > * * > > > > * Then i **set memory.size() at 3000 as well as memory.limit() but i'm > > already having the problem* > > > > * i has a processor of 32 bits, a virtual memory of 3754 Mb* > > > > * ** Can someone tell me if this is due to an improper configuration, > > lack** of sufficient memory, etc?* > > > > > > > -- > Martin Morgan > Computational Biology / Fred Hutchinson Cancer Research Center > 1100 Fairview Ave. N. > PO Box 19024 Seattle, WA 98109 > > Location: Arnold Building M1 B861 > Phone: (206) 667-2793 > -- Marco Ortiz Sexta generacion Licenciatura en Ciencias Genómicas [[alternative HTML version deleted]]

Yes, i already checked the permissions $ ls -l s6_plantula.fq -rwxrwxrwx 1 marco marco 2618477626 2010-08-11 18:38 s6_plantula.fq any other idea? On Fri, Aug 20, 2010 at 1:45 PM, Martin Morgan <mtmorgan@fhcrc.org> wrote: > On 08/20/2010 11:22 AM, Marco Ortiz wrote: > > Hi > > > > Thanks to respond. I checked and the file path is correctly because i > could > > open a similar file but about 38 Mb, that's why i thought that the > memory > > was the problem, here is: > > > > > > > >> control <- readFastq("ControlB_2A8_130_50.250k.fq", > > qualityType="SFastqQuality") > >> seedling <- readFastq("s6_plantula.fq", qualityType="SFastqQuality") > > Error en .local(dirPath, pattern, ...) : cannot open file s6_plantula.fq > > From the shell, do you have read permissions for this file? > > % ls -l s6_plantula.fq > > Martin > > >> control > > class: ShortReadQ > > length: 250000 reads; width: 36 cycles > > > > > > > > > > Here the sessionInfo(), but i'm not sure that it has to give me the info > > about the hardware andf software > > > > > > > >> sessionInfo() > > R version 2.10.1 (2009-12-14) > > i486-pc-linux-gnu > > > > locale: > > [1] LC_CTYPE=es_MX.utf8 LC_NUMERIC=C > > [3] LC_TIME=es_MX.utf8 LC_COLLATE=es_MX.utf8 > > [5] LC_MONETARY=C LC_MESSAGES=es_MX.utf8 > > [7] LC_PAPER=es_MX.utf8 LC_NAME=C > > [9] LC_ADDRESS=C LC_TELEPHONE=C > > [11] LC_MEASUREMENT=es_MX.utf8 LC_IDENTIFICATION=C > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > > other attached packages: > > [1] ShortRead_1.4.0 lattice_0.18-3 BSgenome_1.14.2 > > Biostrings_2.14.12 > > [5] IRanges_1.4.16 > > > > loaded via a namespace (and not attached): > > [1] Biobase_2.6.1 grid_2.10.1 hwriter_1.2 > > > > > > > > > > > > > > > > On Fri, Aug 20, 2010 at 11:23 AM, Martin Morgan <mtmorgan@fhcrc.org> > wrote: > > > >> On 08/20/2010 08:19 AM, Marco Ortiz wrote: > >>> * Hi All, ** ** I am new to the world of R and Bioconductor and I had > >>> the** following error when I try to read a fastaq file of 2 Gb**** > >>> First I try to run it in Linux, but there i had an error that tell me > >>> * > >>> > >>> * <** > >>> > >>> Error: (dirPath, pattern, ...) cannot open file > >>> > >>> * > >>> > >>> * Then i try to run it in Windows and there i had this error* > >>> > >>> * > >>> * > >>> > >>> * > >>> * > >>> > >>> * > library(ShortRead)* > >>> > >>> *** > plantula <- readFastq("s6_plantula.fq", > >> qualityType="SFastqQuality")* > >>> > >>> *** Error: cannot allocate vector of size 649.8 Mb* > >> > >> This is likely limited memory problem; R is saying 'I allocated some > >> memory, and now I'm trying to allocate 649.8Mb more, and that additional > >> memory allocation fails'. Memory limits on 32-bit windows systems are > >> hard to get around; you're better using a 64-bit Windows or Linux > >> system. Your Linux problem seems really straight-forward -- you haven't > >> specified the file path correctly. Your R likely has command completion, > so > >> > >> readFastq("~/ > >> > >> and then use the tab key to complete the path. > >> > >> Please include sessionInfo() when you're posting to the list; it makes > >> clear what software and hardware you're using. > >> > >> Martin > >>> > >>> * > >>> * > >>> > >>> * * > >>> > >>> * Then i **set memory.size() at 3000 as well as memory.limit() but i'm > >>> already having the problem* > >>> > >>> * i has a processor of 32 bits, a virtual memory of 3754 Mb* > >>> > >>> * ** Can someone tell me if this is due to an improper configuration, > >>> lack** of sufficient memory, etc?* > >>> > >>> > >> > >> > >> -- > >> Martin Morgan > >> Computational Biology / Fred Hutchinson Cancer Research Center > >> 1100 Fairview Ave. N. > >> PO Box 19024 Seattle, WA 98109 > >> > >> Location: Arnold Building M1 B861 > >> Phone: (206) 667-2793 > >> > > > > > > > > > -- > Martin Morgan > Computational Biology / Fred Hutchinson Cancer Research Center > 1100 Fairview Ave. N. > PO Box 19024 Seattle, WA 98109 > > Location: Arnold Building M1 B861 > Phone: (206) 667-2793 > -- Marco Ortiz Sexta generacion Licenciatura en Ciencias Genómicas [[alternative HTML version deleted]]

On 08/20/2010 12:53 PM, Martin Morgan wrote: > On 08/20/2010 12:12 PM, Marco Ortiz wrote: >> Yes, i already checked the permissions >> >> $ ls -l s6_plantula.fq >> -rwxrwxrwx 1 marco marco 2618477626 2010-08-11 18:38 s6_plantula.fq >> >> any other idea? > > I doubt it's a memory problem. I'll revise that, and say that the problem occurs when the file is being opened, not when it is being read in. It could be that on your 32-bit system >>>> i486-pc-linux-gnu _representing_ the file _size_ causes the file open to fail. I think your best bets will be to either split the file using standard Linux utilities such as 'split' (though you'll need to make sure your split is on a fastq record boundary) or to locate a 64bit machine wiith ample memory. Martin > Do both of these work in R > > readLines("s6_plantula.fq", 10) > readLines(gzcon(file("s_6_plantula.fq")), 10) > > The permissions are weird (I would have guessed -rw-r--r-- or something) > but I doubt this is a problem. I guess this is supposed to be a plain > text file; is it? e.g., > > head s6_plantula.fq > > giving something reasonable. Is it mounted on an unusual file system, > e.g., can you read another file from the same location? What about just > a small fraction of this file, e.g., > > head -n 1024 s6_plantula.fq > some_file.fq > > Also not likely to be a problem, but updating to a current R / ShortRead. > > Martin > >> >> >> On Fri, Aug 20, 2010 at 1:45 PM, Martin Morgan <mtmorgan at="" fhcrc.org=""> wrote: >> >>> On 08/20/2010 11:22 AM, Marco Ortiz wrote: >>>> Hi >>>> >>>> Thanks to respond. I checked and the file path is correctly because i >>> could >>>> open a similar file but about 38 Mb, that's why i thought that the >>> memory >>>> was the problem, here is: >>>> >>>> >>>> >>>>> control <- readFastq("ControlB_2A8_130_50.250k.fq", >>>> qualityType="SFastqQuality") >>>>> seedling <- readFastq("s6_plantula.fq", qualityType="SFastqQuality") >>>> Error en .local(dirPath, pattern, ...) : cannot open file s6_plantula.fq >>> >>> From the shell, do you have read permissions for this file? >>> >>> % ls -l s6_plantula.fq >>> >>> Martin >>> >>>>> control >>>> class: ShortReadQ >>>> length: 250000 reads; width: 36 cycles >>>> >>>> >>>> >>>> >>>> Here the sessionInfo(), but i'm not sure that it has to give me the info >>>> about the hardware andf software >>>> >>>> >>>> >>>>> sessionInfo() >>>> R version 2.10.1 (2009-12-14) >>>> i486-pc-linux-gnu >>>> >>>> locale: >>>> [1] LC_CTYPE=es_MX.utf8 LC_NUMERIC=C >>>> [3] LC_TIME=es_MX.utf8 LC_COLLATE=es_MX.utf8 >>>> [5] LC_MONETARY=C LC_MESSAGES=es_MX.utf8 >>>> [7] LC_PAPER=es_MX.utf8 LC_NAME=C >>>> [9] LC_ADDRESS=C LC_TELEPHONE=C >>>> [11] LC_MEASUREMENT=es_MX.utf8 LC_IDENTIFICATION=C >>>> >>>> attached base packages: >>>> [1] stats graphics grDevices utils datasets methods base >>>> >>>> other attached packages: >>>> [1] ShortRead_1.4.0 lattice_0.18-3 BSgenome_1.14.2 >>>> Biostrings_2.14.12 >>>> [5] IRanges_1.4.16 >>>> >>>> loaded via a namespace (and not attached): >>>> [1] Biobase_2.6.1 grid_2.10.1 hwriter_1.2 >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Aug 20, 2010 at 11:23 AM, Martin Morgan <mtmorgan at="" fhcrc.org=""> >>> wrote: >>>> >>>>> On 08/20/2010 08:19 AM, Marco Ortiz wrote: >>>>>> * Hi All, ** ** I am new to the world of R and Bioconductor and I had >>>>>> the** following error when I try to read a fastaq file of 2 Gb**** >>>>>> First I try to run it in Linux, but there i had an error that tell me >>>>>> * >>>>>> >>>>>> * <** >>>>>> >>>>>> Error: (dirPath, pattern, ...) cannot open file >>>>>> >>>>>> * >>>>>> >>>>>> * Then i try to run it in Windows and there i had this error* >>>>>> >>>>>> * >>>>>> * >>>>>> >>>>>> * >>>>>> * >>>>>> >>>>>> * > library(ShortRead)* >>>>>> >>>>>> *** > plantula <- readFastq("s6_plantula.fq", >>>>> qualityType="SFastqQuality")* >>>>>> >>>>>> *** Error: cannot allocate vector of size 649.8 Mb* >>>>> >>>>> This is likely limited memory problem; R is saying 'I allocated some >>>>> memory, and now I'm trying to allocate 649.8Mb more, and that additional >>>>> memory allocation fails'. Memory limits on 32-bit windows systems are >>>>> hard to get around; you're better using a 64-bit Windows or Linux >>>>> system. Your Linux problem seems really straight-forward -- you haven't >>>>> specified the file path correctly. Your R likely has command completion, >>> so >>>>> >>>>> readFastq("~/ >>>>> >>>>> and then use the tab key to complete the path. >>>>> >>>>> Please include sessionInfo() when you're posting to the list; it makes >>>>> clear what software and hardware you're using. >>>>> >>>>> Martin >>>>>> >>>>>> * >>>>>> * >>>>>> >>>>>> * * >>>>>> >>>>>> * Then i **set memory.size() at 3000 as well as memory.limit() but i'm >>>>>> already having the problem* >>>>>> >>>>>> * i has a processor of 32 bits, a virtual memory of 3754 Mb* >>>>>> >>>>>> * ** Can someone tell me if this is due to an improper configuration, >>>>>> lack** of sufficient memory, etc?* >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Martin Morgan >>>>> Computational Biology / Fred Hutchinson Cancer Research Center >>>>> 1100 Fairview Ave. N. >>>>> PO Box 19024 Seattle, WA 98109 >>>>> >>>>> Location: Arnold Building M1 B861 >>>>> Phone: (206) 667-2793 >>>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Martin Morgan >>> Computational Biology / Fred Hutchinson Cancer Research Center >>> 1100 Fairview Ave. N. >>> PO Box 19024 Seattle, WA 98109 >>> >>> Location: Arnold Building M1 B861 >>> Phone: (206) 667-2793 >>> >> >> >> > > -- Martin Morgan Computational Biology / Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: Arnold Building M1 B861 Phone: (206) 667-2793