[CF-metadata] packed flags

Ed Hartnett ed at unidata.ucar.edu
Tue Nov 14 08:10:21 MST 2006


Jonathan Gregory <j.m.gregory at reading.ac.uk> writes:

> Dear Richard
>
> I am not sure I quite understand what you are saying. Is the problem that
> you want to use less than a byte for your flags and pack them into words,
> but that CF does not provide a convention for packing that handles less
> than a byte? Does HDF allow you to define datatypes of n bits? If so, perhaps
> netCDF is going to support this at some point.

Sorry, no. HDF5 does not support the creation of types other than 8,
16, 32, and 64 bits. (There is a way to "fake" it, but behind the
scenes, if you define an 11-bit data type, HDF5 will round up the size
and store each value in 16 bits anyway.

HDF5 (and netCDF-4) has a way of defining data types of n bytes (the
opaque type), but not n bits.

>
> This strikes me as a problem it would be awkward to deal with in CF. It's not
> a metadata limitation, but one which refers to file format. CF is based on
> netCDF variables, and in this case you want to pack several variables into one,
> which "breaks" the logical model.
>

I didn't quite follow the original message either. Is the intention to
do this to save disk space?

I have to wonder whether the use of compression might be almost as
useful for saving space, but considerably easier for everyone to
program with!

One could use gzip on netCDF-3 files, and one can turn on compression
in netCDF-4 on a per variable basis (that is, turn it on for some and
not for others.)

I also wonder if the netCDF-4 compound type might be useful here. It
allows a user-defined data type like a C struct...

Thanks,

Ed
-- 
Ed Hartnett  -- ed at unidata.ucar.edu




More information about the CF-metadata mailing list