rHDF5 problem writing strings in attributes
1
0
Entering edit mode
@spencermahaffey-7505
Last seen 9.1 years ago
United States

I'm converting some existing R scripts over from using h5r to rHDF5.  I want to keep all of the file structure and datatypes the same as these files are also read into some of our Java code.  I am having a problem creating an attribute with a string datatype.  Information is written about some of the methods run on the data as an attribute of the group the dataset is stored in.  I can't seem to create an attribute of the same type.  Here's an example.  We're reading in more data and doing calculations but the relevant rHDF5 code is here.

 

h5 <- H5Fopen (InputFile,flags= "H5F_ACC_RDWR")
gVersion<-H5Gopen(h5, "v6")
gFilter<-H5Gopen(gVersion,"Filters")
gDay<-H5Gopen(gFilter,"09132013")
gFVer<-H5Gopen(gDay, "140558")

...Processing, etc.

Procedure <- paste('Function=statistics.OneWay.ANOVA.R',';','Stat.method = one-way ANOVA',';','pvalue of interest=',pvalue,sep = '')

if(H5Aexists (gFVer, "statMethod")){
    H5Adelete (gFVer, "statMethod")
}
sid <- H5Screate_simple (nchar(Procedure))
tid <- H5Tcopy("H5T_STRING")

gSM <- H5Acreate (gFVer, "statMethod",tid,sid)
H5Awrite(gSM,Procedure)
H5Aclose(gSM)

 

Trying to use the H5T_STRING type I get this error:

> tid <- H5Tcopy("H5T_STRING")
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5T.c line 1676 in H5Tcopy(): not a datatype or dataset
    major: Invalid arguments to routine
    minor: Inappropriate type

but running:

> h5const("H5T")
 [1] "H5T_IEEE_F32BE"          "H5T_IEEE_F32LE"          "H5T_IEEE_F64BE"          "H5T_IEEE_F64LE"          "H5T_STD_I8BE"            "H5T_STD_I8LE"           
 [7] "H5T_STD_I16BE"           "H5T_STD_I16LE"           "H5T_STD_I32BE"           "H5T_STD_I32LE"           "H5T_STD_I64BE"           "H5T_STD_I64LE"          
[13] "H5T_STD_U8BE"            "H5T_STD_U8LE"            "H5T_STD_U16BE"           "H5T_STD_U16LE"           "H5T_STD_U32BE"           "H5T_STD_U32LE"          
[19] "H5T_STD_U64BE"           "H5T_STD_U64LE"           "H5T_STD_B8BE"            "H5T_STD_B8LE"            "H5T_STD_B16BE"           "H5T_STD_B16LE"          
[25] "H5T_STD_B32BE"           "H5T_STD_B32LE"           "H5T_STD_B64BE"           "H5T_STD_B64LE"           "H5T_NATIVE_CHAR"         "H5T_NATIVE_SCHAR"       
[31] "H5T_NATIVE_UCHAR"        "H5T_NATIVE_SHORT"        "H5T_NATIVE_USHORT"       "H5T_NATIVE_INT"          "H5T_NATIVE_UINT"         "H5T_NATIVE_LONG"        
[37] "H5T_NATIVE_ULONG"        "H5T_NATIVE_LLONG"        "H5T_NATIVE_ULLONG"       "H5T_NATIVE_FLOAT"        "H5T_NATIVE_DOUBLE"       "H5T_NATIVE_LDOUBLE"     
[43] "H5T_NATIVE_B8"           "H5T_NATIVE_B16"          "H5T_NATIVE_B32"          "H5T_NATIVE_B64"          "H5T_NATIVE_OPAQUE"       "H5T_NATIVE_HADDR"       
[49] "H5T_NATIVE_HSIZE"        "H5T_NATIVE_HSSIZE"       "H5T_NATIVE_HERR"         "H5T_NATIVE_HBOOL"        "H5T_NATIVE_INT8"         "H5T_NATIVE_UINT8"       
[55] "H5T_NATIVE_INT_LEAST8"   "H5T_NATIVE_UINT_LEAST8"  "H5T_NATIVE_INT_FAST8"    "H5T_NATIVE_UINT_FAST8"   "H5T_NATIVE_INT16"        "H5T_NATIVE_UINT16"      
[61] "H5T_NATIVE_INT_LEAST16"  "H5T_NATIVE_UINT_LEAST16" "H5T_NATIVE_INT_FAST16"   "H5T_NATIVE_UINT_FAST16"  "H5T_NATIVE_INT32"        "H5T_NATIVE_UINT32"      
[67] "H5T_NATIVE_INT_LEAST32"  "H5T_NATIVE_UINT_LEAST32" "H5T_NATIVE_INT_FAST32"   "H5T_NATIVE_UINT_FAST32"  "H5T_NATIVE_INT64"        "H5T_NATIVE_UINT64"      
[73] "H5T_NATIVE_INT_LEAST64"  "H5T_NATIVE_UINT_LEAST64" "H5T_NATIVE_INT_FAST64"   "H5T_NATIVE_UINT_FAST64"  "H5T_NATIVE_DOUBLE"       "H5T_STRING"             
[79] "H5T_C_S1"                "H5T_FORTRAN_S1"

H5T_STRING is listed.

I can use rHDF5 to read in a previous attribute of this type:

> gA<-H5Aopen(gFVer,"statMethod")
> gA
HDF5 ATTR

     objName /v6/Filters/09132013/140558
    filename 
    attrName statMethod
        type HST_STRING
        rank 2
        size 1 x 1
     maxsize -1 x -1

> temp2<-H5Aread (gA)
> temp2
     [,1]                                                                                     
[1,] "Function=statistics.OneWay.ANOVA.R;Stat.method = one-way ANOVA;pvalue of interest=Model"

 

So is a variable length string not supported in an attribute by rHDF5 or have I missed something?  I'll keep trying to get one of the fixed length options to work, so far I still don't seem to be specifying the right encoding, but I'm worried that will require changing all of the java code and trying to detect old vs new entries in these files.  If possible I'd like to keep the datatypes the same.

I appreciate any input/suggestions anyone has.  Thank you.

rhdf5 • 2.5k views
ADD COMMENT
1
Entering edit mode
Bernd Fischer ▴ 550
@bernd-fischer-5348
Last seen 7.3 years ago
Germany / Heidelberg / DKFZ

Here is a solution to read and write attributes.

You can write an attribute to a group by the following code:

library(rhdf5)

h5createFile("file.h5")
h5createGroup(file = "file.h5", group = "/mygroup")

Procedure <- paste('Function=statistics.OneWay.ANOVA.R',';','Stat.method = one-way ANOVA',';','pvalue of interest=',sep = '')

fid = H5Fopen("file.h5", flags= "H5F_ACC_RDWR")
h5g = H5Gopen(fid, "/mygroup")
h5writeAttribute(attr = Procedure, h5obj = h5g, name = "statMethod")
H5Gclose(h5g)
H5Fclose(fid)

And two alternatives to read the attributes again:

Alternative 1, reading only attributes:

Attr = h5readAttributes(file = "file.h5", name="/mygroup")
Attr

Alternative 2: reading the group including attributes

R = h5read(file = "file.h5", name="/mygroup", read.attributes = TRUE)
R

If you have problem reading a specific attribute in one of your files, please provide an example that can be executed.

 

ADD COMMENT
0
Entering edit mode

Thank you for your help, I was trying to use the attribute interface directly, but this method makes it much easier.  It works and the difference in datatype (not a variable length string) doesn't seem to cause problems with reading the rHDF5 modified file in Java.

ADD REPLY

Login before adding your answer.

Traffic: 625 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6