Search
Question: rHDF5 problem writing strings in attributes
0
gravatar for spencer.mahaffey
2.7 years ago by
United States
spencer.mahaffey0 wrote:

I'm converting some existing R scripts over from using h5r to rHDF5.  I want to keep all of the file structure and datatypes the same as these files are also read into some of our Java code.  I am having a problem creating an attribute with a string datatype.  Information is written about some of the methods run on the data as an attribute of the group the dataset is stored in.  I can't seem to create an attribute of the same type.  Here's an example.  We're reading in more data and doing calculations but the relevant rHDF5 code is here.

 

h5 <- H5Fopen (InputFile,flags= "H5F_ACC_RDWR")
gVersion<-H5Gopen(h5, "v6")
gFilter<-H5Gopen(gVersion,"Filters")
gDay<-H5Gopen(gFilter,"09132013")
gFVer<-H5Gopen(gDay, "140558")

...Processing, etc.

Procedure <- paste('Function=statistics.OneWay.ANOVA.R',';','Stat.method = one-way ANOVA',';','pvalue of interest=',pvalue,sep = '')

if(H5Aexists (gFVer, "statMethod")){
    H5Adelete (gFVer, "statMethod")
}
sid <- H5Screate_simple (nchar(Procedure))
tid <- H5Tcopy("H5T_STRING")

gSM <- H5Acreate (gFVer, "statMethod",tid,sid)
H5Awrite(gSM,Procedure)
H5Aclose(gSM)

 

Trying to use the H5T_STRING type I get this error:

> tid <- H5Tcopy("H5T_STRING")
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5T.c line 1676 in H5Tcopy(): not a datatype or dataset
    major: Invalid arguments to routine
    minor: Inappropriate type

but running:

> h5const("H5T")
 [1] "H5T_IEEE_F32BE"          "H5T_IEEE_F32LE"          "H5T_IEEE_F64BE"          "H5T_IEEE_F64LE"          "H5T_STD_I8BE"            "H5T_STD_I8LE"           
 [7] "H5T_STD_I16BE"           "H5T_STD_I16LE"           "H5T_STD_I32BE"           "H5T_STD_I32LE"           "H5T_STD_I64BE"           "H5T_STD_I64LE"          
[13] "H5T_STD_U8BE"            "H5T_STD_U8LE"            "H5T_STD_U16BE"           "H5T_STD_U16LE"           "H5T_STD_U32BE"           "H5T_STD_U32LE"          
[19] "H5T_STD_U64BE"           "H5T_STD_U64LE"           "H5T_STD_B8BE"            "H5T_STD_B8LE"            "H5T_STD_B16BE"           "H5T_STD_B16LE"          
[25] "H5T_STD_B32BE"           "H5T_STD_B32LE"           "H5T_STD_B64BE"           "H5T_STD_B64LE"           "H5T_NATIVE_CHAR"         "H5T_NATIVE_SCHAR"       
[31] "H5T_NATIVE_UCHAR"        "H5T_NATIVE_SHORT"        "H5T_NATIVE_USHORT"       "H5T_NATIVE_INT"          "H5T_NATIVE_UINT"         "H5T_NATIVE_LONG"        
[37] "H5T_NATIVE_ULONG"        "H5T_NATIVE_LLONG"        "H5T_NATIVE_ULLONG"       "H5T_NATIVE_FLOAT"        "H5T_NATIVE_DOUBLE"       "H5T_NATIVE_LDOUBLE"     
[43] "H5T_NATIVE_B8"           "H5T_NATIVE_B16"          "H5T_NATIVE_B32"          "H5T_NATIVE_B64"          "H5T_NATIVE_OPAQUE"       "H5T_NATIVE_HADDR"       
[49] "H5T_NATIVE_HSIZE"        "H5T_NATIVE_HSSIZE"       "H5T_NATIVE_HERR"         "H5T_NATIVE_HBOOL"        "H5T_NATIVE_INT8"         "H5T_NATIVE_UINT8"       
[55] "H5T_NATIVE_INT_LEAST8"   "H5T_NATIVE_UINT_LEAST8"  "H5T_NATIVE_INT_FAST8"    "H5T_NATIVE_UINT_FAST8"   "H5T_NATIVE_INT16"        "H5T_NATIVE_UINT16"      
[61] "H5T_NATIVE_INT_LEAST16"  "H5T_NATIVE_UINT_LEAST16" "H5T_NATIVE_INT_FAST16"   "H5T_NATIVE_UINT_FAST16"  "H5T_NATIVE_INT32"        "H5T_NATIVE_UINT32"      
[67] "H5T_NATIVE_INT_LEAST32"  "H5T_NATIVE_UINT_LEAST32" "H5T_NATIVE_INT_FAST32"   "H5T_NATIVE_UINT_FAST32"  "H5T_NATIVE_INT64"        "H5T_NATIVE_UINT64"      
[73] "H5T_NATIVE_INT_LEAST64"  "H5T_NATIVE_UINT_LEAST64" "H5T_NATIVE_INT_FAST64"   "H5T_NATIVE_UINT_FAST64"  "H5T_NATIVE_DOUBLE"       "H5T_STRING"             
[79] "H5T_C_S1"                "H5T_FORTRAN_S1"

H5T_STRING is listed.

I can use rHDF5 to read in a previous attribute of this type:

> gA<-H5Aopen(gFVer,"statMethod")
> gA
HDF5 ATTR

     objName /v6/Filters/09132013/140558
    filename 
    attrName statMethod
        type HST_STRING
        rank 2
        size 1 x 1
     maxsize -1 x -1

> temp2<-H5Aread (gA)
> temp2
     [,1]                                                                                     
[1,] "Function=statistics.OneWay.ANOVA.R;Stat.method = one-way ANOVA;pvalue of interest=Model"

 

So is a variable length string not supported in an attribute by rHDF5 or have I missed something?  I'll keep trying to get one of the fixed length options to work, so far I still don't seem to be specifying the right encoding, but I'm worried that will require changing all of the java code and trying to detect old vs new entries in these files.  If possible I'd like to keep the datatypes the same.

I appreciate any input/suggestions anyone has.  Thank you.

ADD COMMENTlink modified 2.7 years ago by Bernd Fischer540 • written 2.7 years ago by spencer.mahaffey0
1
gravatar for Bernd Fischer
2.7 years ago by
Bernd Fischer540
Germany / Heidelberg / DKFZ
Bernd Fischer540 wrote:

Here is a solution to read and write attributes.

You can write an attribute to a group by the following code:

library(rhdf5)

h5createFile("file.h5")
h5createGroup(file = "file.h5", group = "/mygroup")

Procedure <- paste('Function=statistics.OneWay.ANOVA.R',';','Stat.method = one-way ANOVA',';','pvalue of interest=',sep = '')

fid = H5Fopen("file.h5", flags= "H5F_ACC_RDWR")
h5g = H5Gopen(fid, "/mygroup")
h5writeAttribute(attr = Procedure, h5obj = h5g, name = "statMethod")
H5Gclose(h5g)
H5Fclose(fid)

And two alternatives to read the attributes again:

Alternative 1, reading only attributes:

Attr = h5readAttributes(file = "file.h5", name="/mygroup")
Attr

Alternative 2: reading the group including attributes

R = h5read(file = "file.h5", name="/mygroup", read.attributes = TRUE)
R

If you have problem reading a specific attribute in one of your files, please provide an example that can be executed.

 

ADD COMMENTlink modified 2.7 years ago • written 2.7 years ago by Bernd Fischer540

Thank you for your help, I was trying to use the attribute interface directly, but this method makes it much easier.  It works and the difference in datatype (not a variable length string) doesn't seem to cause problems with reading the rHDF5 modified file in Java.

ADD REPLYlink written 2.7 years ago by spencer.mahaffey0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 224 users visited in the last hour