rhdf5 segfault
3
0
Entering edit mode
Last seen 8.4 years ago
Dear Bernd and List Members: I'm getting a segmentation fault reading in a character vector (attached) written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read this without throwing the error I would appreciate it. The file appears to be written correctly since h5dump can read it. I got the same error with the same vector on both my mac laptop and our linux server, in both cases using the latest version of rhdf5 (2.7.4). Brad > library(rhdf5) > load("charvec.rda") ## provides "charvec" > str(charvec) chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > hdf5.file <- tempfile() > h5createFile(hdf5.file) [1] TRUE > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") Error: segfault from C stack overflow > sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin10.8.0 (64-bit) locale: [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rhdf5_2.7.4 loaded via a namespace (and not attached): [1] zlibbioc_1.6.0
rhdf5 rhdf5 • 1.9k views
1
Entering edit mode
Last seen 5.4 years ago
0
Entering edit mode
On Dec 23, 2013, at 10:56 PM, Ben Bolstad <bmb at="" bmbolstad.com=""> wrote: > You might try changing your stack size limits in your shell, before launching R. e.g. > > ulimit -s unlimited I don?t think that is possible, at least as you put it. The soft limit tends to be smaller than the hard limit and is the only thing that can be increased. You may not get unlimited, but this should work and might help: limit -Ss ulimit -Hs > I know I encountered a similar issue with rhdf5 not to long ago reading from a file with a fairly large number of strings, and that was a suitable work around. Note within R Cstack_info() will also tell you stack size. > > At the time I looked into it, at least in my case I tracked it down to a segfault occuring in the H5Dread_helper_STRING function of file H5D.C at line 244, but it looked like the issue might be with the allocation: > > char bufSTR[n][size] > > in the previous line and I can only assume that n*size is too large for the C stack (in my instance, and perhaps yours) at the default settings in your shell. > > > Best, > > Ben > > > > On Dec 23, 2013, at 4:11 PM, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > >> Dear Bernd and List Members: >> >> I'm getting a segmentation fault reading in a character vector (attached) >> written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read >> this without throwing the error I would appreciate it. The file appears to >> be written correctly since h5dump can read it. >> >> I got the same error with the same vector on both my mac laptop and our >> linux server, in both cases using the latest version of rhdf5 (2.7.4). >> >> Brad >> >>> library(rhdf5) >>> load("charvec.rda") ## provides "charvec" >> >> >>> str(charvec) >> chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >>> hdf5.file <- tempfile() >>> h5createFile(hdf5.file) >> [1] TRUE >>> h5write(charvec, hdf5.file, "foo") >>> got <- h5read(hdf5.file, "foo") >> Error: segfault from C stack overflow >>> sessionInfo() >> R version 3.0.1 (2013-05-16) >> Platform: x86_64-apple-darwin10.8.0 (64-bit) >> >> locale: >> [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] rhdf5_2.7.4 >> >> loaded via a namespace (and not attached): >> [1] zlibbioc_1.6.0 >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
0
Entering edit mode
@wolfgang-huber-3550
Last seen 5 days ago
EMBL European Molecular Biology Laborat…
0
Entering edit mode
0
Entering edit mode
0
Entering edit mode
Last seen 8.1 years ago
Shanghai, China

The solution (ulimit -s unlimited) given by Ben Bolstad works on Linux machine, but not on Mac OS. ulimit -s won's accept unlimited as a valid parameter, instead you have to specify the size in number such as  ulimit -s 20000.

It appears that in Mac OS 10.10 ulimit no longer works, you should instead use launchctl limit stack to set the stack size, but the size can not exceed the system stack hard limit.

So first run a

launchctl limit stack

to check the 2nd number (in my case 67104768).  Then

sudo launchctl limit stack 67104768

to set the current stack limit to the hard limit.

Restart your terminal or rstudio, now h5read should work without segment fault.