Question: rhdf5 segfault
0
gravatar for Brad Friedman
5.6 years ago by
Brad Friedman40 wrote:
Dear Bernd and List Members: I'm getting a segmentation fault reading in a character vector (attached) written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read this without throwing the error I would appreciate it. The file appears to be written correctly since h5dump can read it. I got the same error with the same vector on both my mac laptop and our linux server, in both cases using the latest version of rhdf5 (2.7.4). Brad > library(rhdf5) > load("charvec.rda") ## provides "charvec" > str(charvec) chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > hdf5.file <- tempfile() > h5createFile(hdf5.file) [1] TRUE > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") Error: segfault from C stack overflow > sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin10.8.0 (64-bit) locale: [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rhdf5_2.7.4 loaded via a namespace (and not attached): [1] zlibbioc_1.6.0
rhdf5 • 1.0k views
ADD COMMENTlink modified 4.5 years ago by imadcat0 • written 5.6 years ago by Brad Friedman40
Answer: rhdf5 segfault
1
gravatar for Ben Bolstad
5.6 years ago by
Ben Bolstad1.2k
Ben Bolstad1.2k wrote:
You might try changing your stack size limits in your shell, before launching R. e.g. ulimit -s unlimited I know I encountered a similar issue with rhdf5 not to long ago reading from a file with a fairly large number of strings, and that was a suitable work around. Note within R Cstack_info() will also tell you stack size. At the time I looked into it, at least in my case I tracked it down to a segfault occuring in the H5Dread_helper_STRING function of file H5D.C at line 244, but it looked like the issue might be with the allocation: char bufSTR[n][size] in the previous line and I can only assume that n*size is too large for the C stack (in my instance, and perhaps yours) at the default settings in your shell. Best, Ben On Dec 23, 2013, at 4:11 PM, Brad Friedman <friedman.brad@gene.com> wrote: > Dear Bernd and List Members: > > I'm getting a segmentation fault reading in a character vector (attached) > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > this without throwing the error I would appreciate it. The file appears to > be written correctly since h5dump can read it. > > I got the same error with the same vector on both my mac laptop and our > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > Brad > >> library(rhdf5) >> load("charvec.rda") ## provides "charvec" > > >> str(charvec) > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >> hdf5.file <- tempfile() >> h5createFile(hdf5.file) > [1] TRUE >> h5write(charvec, hdf5.file, "foo") >> got <- h5read(hdf5.file, "foo") > Error: segfault from C stack overflow >> sessionInfo() > R version 3.0.1 (2013-05-16) > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > locale: > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.6.0 > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor [[alternative HTML version deleted]]
ADD COMMENTlink written 5.6 years ago by Ben Bolstad1.2k
On Dec 23, 2013, at 10:56 PM, Ben Bolstad <bmb at="" bmbolstad.com=""> wrote: > You might try changing your stack size limits in your shell, before launching R. e.g. > > ulimit -s unlimited I don?t think that is possible, at least as you put it. The soft limit tends to be smaller than the hard limit and is the only thing that can be increased. You may not get unlimited, but this should work and might help: limit -Ss `ulimit -Hs` > I know I encountered a similar issue with rhdf5 not to long ago reading from a file with a fairly large number of strings, and that was a suitable work around. Note within R Cstack_info() will also tell you stack size. > > At the time I looked into it, at least in my case I tracked it down to a segfault occuring in the H5Dread_helper_STRING function of file H5D.C at line 244, but it looked like the issue might be with the allocation: > > char bufSTR[n][size] > > in the previous line and I can only assume that n*size is too large for the C stack (in my instance, and perhaps yours) at the default settings in your shell. > > > Best, > > Ben > > > > On Dec 23, 2013, at 4:11 PM, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > >> Dear Bernd and List Members: >> >> I'm getting a segmentation fault reading in a character vector (attached) >> written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read >> this without throwing the error I would appreciate it. The file appears to >> be written correctly since h5dump can read it. >> >> I got the same error with the same vector on both my mac laptop and our >> linux server, in both cases using the latest version of rhdf5 (2.7.4). >> >> Brad >> >>> library(rhdf5) >>> load("charvec.rda") ## provides "charvec" >> >> >>> str(charvec) >> chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >>> hdf5.file <- tempfile() >>> h5createFile(hdf5.file) >> [1] TRUE >>> h5write(charvec, hdf5.file, "foo") >>> got <- h5read(hdf5.file, "foo") >> Error: segfault from C stack overflow >>> sessionInfo() >> R version 3.0.1 (2013-05-16) >> Platform: x86_64-apple-darwin10.8.0 (64-bit) >> >> locale: >> [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] rhdf5_2.7.4 >> >> loaded via a namespace (and not attached): >> [1] zlibbioc_1.6.0 >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLYlink written 5.6 years ago by Harris A. Jaffee590
Answer: rhdf5 segfault
0
gravatar for Wolfgang Huber
5.6 years ago by
EMBL European Molecular Biology Laboratory
Wolfgang Huber13k wrote:
Dear Brad thank you for the feedback, much appreciated. I could not run your code since the file ?charvec.rda? is presumably only on your harddrive, but a similar piece of code (below) worked fine for me. This is on Mac OS 10.9.1, I built R and rhdf5 from source using system libraries from Mac Ports. Since this seems to be a problem specific for certain installations, we?ll need a reproducible example, exact system description, and more detailed debugging information at the C level (afaIu there are people locally at your workplace who can help with that). Kind regards Wolfgang -------------------------- library("rhdf5") ## load("charvec.rda") ## provides "charvec" charvec = paste(1:16630) str(charvec) hdf5.file <- tempfile() h5createFile(hdf5.file) h5write(charvec, hdf5.file, "foo") got <- h5read(hdf5.file, "foo") stopifnot(all(got==charvec), length(got)==length(charvec)) sessionInfo() ---------------------------------------------- > sessionInfo() R Under development (unstable) (2013-12-25 r64540) Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) locale: [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rhdf5_2.7.4 fortunes_1.5-2 loaded via a namespace (and not attached): [1] zlibbioc_1.8.0 On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > Dear Bernd and List Members: > > I'm getting a segmentation fault reading in a character vector (attached) > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > this without throwing the error I would appreciate it. The file appears to > be written correctly since h5dump can read it. > > I got the same error with the same vector on both my mac laptop and our > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > Brad > >> library(rhdf5) >> load("charvec.rda") ## provides "charvec" > > >> str(charvec) > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >> hdf5.file <- tempfile() >> h5createFile(hdf5.file) > [1] TRUE >> h5write(charvec, hdf5.file, "foo") >> got <- h5read(hdf5.file, "foo") > Error: segfault from C stack overflow >> sessionInfo() > R version 3.0.1 (2013-05-16) > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > locale: > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.6.0 > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD COMMENTlink written 5.6 years ago by Wolfgang Huber13k
Hi, Wolfgang Thanks for your response. I attached a file charvec.rda to the original posting....maybe I did this wrong? I'm attaching it here. Do you mind trying one more time if you can reproduce my error? Thanks, Brad On Wed, Dec 25, 2013 at 11:06 AM, Wolfgang Huber <whuber at="" embl.de=""> wrote: > Dear Brad > > thank you for the feedback, much appreciated. I could not run your code > since the file ?charvec.rda? is presumably only on your harddrive, but a > similar piece of code (below) worked fine for me. This is on Mac OS > 10.9.1, I built R and rhdf5 from source using system libraries from Mac > Ports. > > Since this seems to be a problem specific for certain installations, we?ll > need a reproducible example, exact system description, and more detailed > debugging information at the C level (afaIu there are people locally at > your workplace who can help with that). > > Kind regards > Wolfgang > > -------------------------- > library("rhdf5") > > ## load("charvec.rda") ## provides "charvec" > charvec = paste(1:16630) > > str(charvec) > > hdf5.file <- tempfile() > h5createFile(hdf5.file) > > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") > > stopifnot(all(got==charvec), length(got)==length(charvec)) > > sessionInfo() > > ---------------------------------------------- > > sessionInfo() > R Under development (unstable) (2013-12-25 r64540) > Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) > > locale: > [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 fortunes_1.5-2 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.8.0 > > > > > On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > > > Dear Bernd and List Members: > > > > I'm getting a segmentation fault reading in a character vector (attached) > > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to > read > > this without throwing the error I would appreciate it. The file appears > to > > be written correctly since h5dump can read it. > > > > I got the same error with the same vector on both my mac laptop and our > > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > > > Brad > > > >> library(rhdf5) > >> load("charvec.rda") ## provides "charvec" > > > > > >> str(charvec) > > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > >> hdf5.file <- tempfile() > >> h5createFile(hdf5.file) > > [1] TRUE > >> h5write(charvec, hdf5.file, "foo") > >> got <- h5read(hdf5.file, "foo") > > Error: segfault from C stack overflow > >> sessionInfo() > > R version 3.0.1 (2013-05-16) > > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > > > locale: > > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > > other attached packages: > > [1] rhdf5_2.7.4 > > > > loaded via a namespace (and not attached): > > [1] zlibbioc_1.6.0 > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor at r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > >
ADD REPLYlink written 5.6 years ago by Brad Friedman40
Dear Brad thank you. When I use the instance of ?charvec? loaded from your charvec.rda file, then I can reproduce the error you observe: "Error: segfault from C stack overflow? and return to R prompt after running the below code once, "Segmentation fault: 11? and termination of the R process after the second time. So this might be related to the content or size of the character vector. I leave the rest to Bernd (package maintainer). This should be enough information to get going. Btw, your original post as received in my inbox did not contain an attachment, and it appears that attachments are also stripped from here: https://stat.ethz.ch/pipermail/bioconductor/2013-December/056760.html ?? library("rhdf5") if (TRUE) load("charvec.rda") else charvec = paste(1:16630) hdf5.file <- tempfile() h5createFile(hdf5.file) h5write(charvec, hdf5.file, "foo") got <- h5read(hdf5.file, "foo") stopifnot(all(got==charvec), length(got)==length(charvec)) sessionInfo() ?? Kind regards Wolfgang On 26 Dec 2013, at 17:49, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > Hi, Wolfgang > > Thanks for your response. I attached a file charvec.rda to the original posting....maybe I did this wrong? I'm attaching it here. Do you mind trying one more time if you can reproduce my error? > > Thanks, > > Brad > > > On Wed, Dec 25, 2013 at 11:06 AM, Wolfgang Huber <whuber at="" embl.de=""> wrote: > Dear Brad > > thank you for the feedback, much appreciated. I could not run your code since the file ?charvec.rda? is presumably only on your harddrive, but a similar piece of code (below) worked fine for me. This is on Mac OS 10.9.1, I built R and rhdf5 from source using system libraries from Mac Ports. > > Since this seems to be a problem specific for certain installations, we?ll need a reproducible example, exact system description, and more detailed debugging information at the C level (afaIu there are people locally at your workplace who can help with that). > > Kind regards > Wolfgang > > -------------------------- > library("rhdf5") > > ## load("charvec.rda") ## provides "charvec" > charvec = paste(1:16630) > > str(charvec) > > hdf5.file <- tempfile() > h5createFile(hdf5.file) > > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") > > stopifnot(all(got==charvec), length(got)==length(charvec)) > > sessionInfo() > > ---------------------------------------------- > > sessionInfo() > R Under development (unstable) (2013-12-25 r64540) > Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) > > locale: > [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 fortunes_1.5-2 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.8.0 > > > > > On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > > > Dear Bernd and List Members: > > > > I'm getting a segmentation fault reading in a character vector (attached) > > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > > this without throwing the error I would appreciate it. The file appears to > > be written correctly since h5dump can read it. > > > > I got the same error with the same vector on both my mac laptop and our > > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > > > Brad > > > >> library(rhdf5) > >> load("charvec.rda") ## provides "charvec" > > > > > >> str(charvec) > > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > >> hdf5.file <- tempfile() > >> h5createFile(hdf5.file) > > [1] TRUE > >> h5write(charvec, hdf5.file, "foo") > >> got <- h5read(hdf5.file, "foo") > > Error: segfault from C stack overflow > >> sessionInfo() > > R version 3.0.1 (2013-05-16) > > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > > > locale: > > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > > other attached packages: > > [1] rhdf5_2.7.4 > > > > loaded via a namespace (and not attached): > > [1] zlibbioc_1.6.0 > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor at r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > > <charvec.rda>
ADD REPLYlink written 5.6 years ago by Wolfgang Huber13k
Answer: rhdf5 segfault
0
gravatar for imadcat
4.5 years ago by
imadcat0
Shanghai, China
imadcat0 wrote:

The solution (ulimit -s unlimited) given by Ben Bolstad works on Linux machine, but not on Mac OS. ulimit -s won's accept unlimited as a valid parameter, instead you have to specify the size in number such as  ulimit -s 20000.

It appears that in Mac OS 10.10 ulimit no longer works, you should instead use launchctl limit stack to set the stack size, but the size can not exceed the system stack hard limit.

So first run a  

launchctl limit stack  

to check the 2nd number (in my case 67104768).  Then 

sudo launchctl limit stack 67104768

to set the current stack limit to the hard limit.

Restart your terminal or rstudio, now h5read should work without segment fault.

ADD COMMENTlink modified 4.5 years ago • written 4.5 years ago by imadcat0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 114 users visited in the last hour