Re: looking for a file format for time series data other software can easily consume
- Date: Thu, 2 Nov 2017 09:00:02 +1300
- From: Ben Caradoc-Davies <ben@xxxxxxxxxxxx>
- Subject: Re: looking for a file format for time series data other software can easily consume
On 01/11/17 20:38, Dan Hitt wrote:
The NetCDF does seem to do what i want. As far as i can tell, it
allows you to set metadata at the file level as well as attributes
attached to specific arrays in your data. It can be installed in
debian (packages libnetcdf-dev, netcdf-doc, netcdf-bin). The api
doesn't look too complicated, at least if i understand it correctly.
The packages come with some simple example programs. And the 'file'
command recognizes the data that it produces. And there's some kind
of mailing list. So if i can make a go of it, i think that's what
i'll be doing.
What language would you like to use? There are bindings for many popular
languages including Python, and a Java implementation for JVM languages.
Timeseries representation is straightforward: a one-dimensional
"coordinate variable" for time with an attribute like units="seconds
since [your epoch goes here]" and data variables of the same shape. The
udunits library will even convert units for you. If you make your time
dimension UNLIMITED, new time values are written at the end of the file
(rather than rewriting the whole file).
Ben Caradoc-Davies <ben@xxxxxxxxxxxx>
Transient Software Limited <http://transient.nz/>