result.netcdf¶

The netCDF files of the results: compressed netCDF4, the attributes as JSON.

A result is written as netCDF4 (HDF5) through h5netcdf, the one file format which xarray, netCDF-C, R, MATLAB and Julia read with the dimensions, coordinates and attributes of the result. The attributes of a result are nested, they are written as one JSON attribute, NETCDF_ATTRS.

The numeric variables are compressed (#412): the results of simulations compress well, the HCTZ model with its 191 selections over 5000 points from 1551 MB to 136 MB. zstd (the default) is the HDF5 filter 32015 of hdf5plugin with the shuffle filter of HDF5, which writes about twice as long as no compression and reads about as fast; netCDF-C 4.9 reads it, a python reader outside of sbmlsim imports hdf5plugin first. gzip is the filter every HDF5 reads, at about five times the time of writing. A variable is stored in chunks of whole timecourses (its last dimension) of about CHUNK_BYTES, which compress better than the chunks h5netcdf would choose.

Arrow, Zarr and npz were measured as well (#412). Arrow IPC writes and reads fastest, but it is a table: the dimensions, coordinates and attributes would be an encoding of our own which no other tool reads as such. Zarr is N-dimensional and native to xarray, but a directory writes a file per chunk, a zip file duplicate entries, and it adds its dependencies. npz writes slowest.

write_netcdf ¶

write_netcdf(ds, path, compression='zstd')

Write a dataset as netCDF4, its attributes as JSON.

Parameters:

Name	Type	Description	Default
`ds`	`Dataset`	the dataset, whose attributes must be JSON serializable; numpy scalars and arrays are written as numbers and lists.	required
`path`	`str \| Path`	the file.	required
`compression`	`Compression`	the compression of the numeric variables, see the module.	`'zstd'`

Raises:

Type	Description
`ValueError`	for an unknown compression.

read_netcdf ¶

read_netcdf(path)

Read a dataset of write_netcdf into memory, with its attributes.

Raises:

Type	Description
`ValueError`	if the file has no attributes of a result of sbmlsim.

chunk_shape ¶

chunk_shape(shape, itemsize)

Get the chunks of a variable: whole rows of its last dimension.

A chunk holds about CHUNK_BYTES: the whole last dimension (a timecourse) and as many of the dimensions before it as fit, from the last of them on. A last dimension larger than a chunk is cut.

Parameters:

Name	Type	Description	Default
`shape`	`tuple[int, ...]`	the shape of the variable, every size at least one.	required
`itemsize`	`int`	the bytes of a value.	required

json_attributes ¶

json_attributes(attrs)

Get attributes as the JSON types which read_netcdf reads them as.

Attributes in memory and attributes read from a file compare equal in this form, e.g. a tuple and a numpy array are lists.