sciml.network¶

The architecture and the arrays of a neural network.

Network holds the architecture of a network as the NNModel of petab_sciml, i.e. the content of the NN YAML, and the arrays of its layers in the PyTorch layout. Network.forward evaluates it with numpy.

Every element of an array has an id, <net>__<layer>__<array>__<index> with the PyTorch index of the element and _ between the axes, e.g. net1__layer1__weight__0_1. The units of the nodes of the forward pass, the inputs and the outputs have ids of the same form:

entity	id	function
element of an array	`net1__layer1__weight__0_1`	`element_id`
unit of a node	`net1__tanh_1__3`	`unit_id`
input	`net1__input0__1`, `net1__input0` for the array	`input_id`
output	`net1__output0__0`	`output_id`

Every id is a valid SBML SId, the compilation of a network into a model and the hybridization of a problem share these functions.

Network `dataclass` ¶

Network(sid, model, parameters=dict())

The architecture and the arrays of one network.

A network is validated when it is created: the id, the forward pass and the nominal values are checked against the architecture, and every layer needs an implementation. A network does not change afterwards: its attributes cannot be assigned, the mappings of parameters cannot be changed and its arrays cannot be written, so the structures derived from them (array_specs, used_layers, parameter_ids) are computed once and stay true. The arrays are copies of the ones the network was built from. The model is shared and not copied, it must not be changed after the network was created, the derived structures would not follow. A network with other nominal values is a new network, e.g. dataclasses.replace(network, parameters=...). A copy of a network (copy.deepcopy, pickle of any protocol) is created again from its id, architecture and arrays, so it is validated and frozen like the original.

Two networks are equal when they have the same id, the same architecture and the same arrays, which is what the round trip of a problem and the pickling of a fit compare. The hash is the one of the id only, networks of one id are equal or differ in the architecture or the arrays.

Attributes:

Name	Type	Description
`sid`	`str`	id of the network, an SBML `SId` which is the `nn_model_id` of the model.
`model`	`NNModel`	the architecture, i.e. the content of the NN YAML.
`parameters`	`FrozenParameters`	the nominal values of the arrays in the PyTorch layout, layer id -> array name -> values.

from_files `classmethod` ¶

from_files(yaml_path, array_path=None, sid=None)

Read a network from its NN YAML and its array file.

The arrays of a file with metadata/pytorch_format false are stored in the column major layout, i.e. with the axes in the reverse order, and are permuted into the PyTorch layout.

Parameters:

Name	Type	Description	Default
`yaml_path`	`Path`	the NN YAML.	required
`array_path`	`Path \| None`	the HDF5 file with the arrays of the network. Without it the network has no values, which a problem sets.	`None`
`sid`	`str \| None`	id of the network, the `nn_model_id` of the YAML when `None`. The arrays are read from the group of this id.	`None`

Returns:

Type	Description
`Network`	The network.

Raises:

Type	Description
`NetworkImportError`	if a file does not exist or cannot be read, if the network is not valid (see `Network`), if the array file has no arrays for the network, or if an array does not belong to a layer or does not have the shape of the layer.
`UnsupportedLayerError`	if the network has a layer without an implementation.

read_arrays ¶

read_arrays(array_path)

Read the arrays of the network from an array file.

An empty array is an array the file does not provide, which is how a file leaves out the arrays a problem sets.

Parameters:

Name	Type	Description	Default
`array_path`	`Path`	the HDF5 file.	required

Returns:

Type	Description
`NetworkParameters`	The arrays in the PyTorch layout, not checked against the
`NetworkParameters`	architecture; a network created with them checks them.

Raises:

Type	Description
`NetworkImportError`	if the file does not exist, cannot be read or has no arrays for the network.

check_forward ¶

check_forward()

Check the forward pass against the layers.

Raises:

Type	Description
`NetworkImportError`	if a `call_module` node calls a layer the network does not have, or if an output node does not have one argument, the output or the list of the outputs.

array_specs ¶

array_specs()

Get the arrays of every layer of the network.

Returns:

Type	Description
`dict[str, dict[str, ArraySpec]]`	The arrays of the layers, layer id -> array name -> shape and
`dict[str, dict[str, ArraySpec]]`	kind, in the order of the layers. The layers were checked when the
`dict[str, dict[str, ArraySpec]]`	network was created.

used_layers ¶

used_layers()

Get the ids of the layers the forward pass calls, in its order.

backends ¶

backends()

Get the backends which evaluate every node of the forward pass.

This is about the evaluation of the nodes: the layers and functions of the forward pass, not the layers the network defines and does not call.

Returns:

Type	Description
`frozenset[BackendKind]`	The backends all layers and functions of the forward pass support.

Raises:

Type	Description
`UnsupportedLayerError`	if a node has no implementation.

check_arrays ¶

check_arrays(parameters, complete=True)

Check arrays against the architecture.

Parameters:

Name	Type	Description	Default
`parameters`	`Mapping[str, Mapping[str, ndarray]]`	the arrays, layer id -> array name -> values.	required
`complete`	`bool`	whether every required array of a layer of the forward pass must be there.	`True`

Raises:

Type	Description
`NetworkImportError`	if an array does not belong to a layer of the network, does not have the shape of the layer, holds a value which is not finite or, with `complete`, is missing.

forward ¶

forward(*inputs, parameters=None)

Evaluate the network with numpy, in evaluation mode.

Parameters:

Name	Type	Description	Default
`*inputs`	`ndarray`	the inputs in the PyTorch layout, one per input of the forward pass.	`()`
`parameters`	`NetworkParameters \| None`	the arrays the network is evaluated with, the nominal values when `None`.	`None`

Returns:

Type	Description
`ndarray`	The outputs of the network, arrays which share no memory with the
`...`	inputs.

Raises:

Type	Description
`NetworkImportError`	if an array of a layer is missing or has the wrong shape.
`UnsupportedLayerError`	if a layer or function has no implementation.
`ValueError`	if the inputs do not fit the network.

parameter_ids ¶

parameter_ids()

Get the ids of the elements of the arrays which are parameters.

The ids follow from the architecture, so they exist for an array without values as well. The running statistics of a normalization layer are not parameters.

Returns:

Type	Description
`ParameterIds`	id of the element -> layer id, array name and PyTorch index, in
`ParameterIds`	the order of the layers, the arrays and the row major order of the
`ParameterIds`	elements.

Raises:

Type	Description
`NetworkImportError`	if two elements have the same id, which the ids of two layers such as `block.0` and `block_0` cause.

with_values ¶

with_values(values)

Get the arrays with the values of some elements replaced.

Parameters:

Name	Type	Description	Default
`values`	`Mapping[str, float]`	id of the element -> value.	required

Returns:

Type	Description
`NetworkParameters`	A copy of the nominal values with the elements replaced. The
`NetworkParameters`	network is not changed.

Raises:

Type	Description
`NetworkImportError`	if an id is not the id of an element of the network, if a value is not finite, or if an element of an array without nominal values is set.

element_id ¶

element_id(network, layer, array, index)

Get the id of an element of an array.

Parameters:

Name	Type	Description	Default
`network`	`str`	id of the network.	required
`layer`	`str`	id of the layer.	required
`array`	`str`	name of the array, e.g. `weight`.	required
`index`	`tuple[int, ...]`	the PyTorch index of the element.	required

Returns:

Type	Description
`str`	The id, e.g. `net1__layer1__weight__0_1`. A character which is not
`str`	part of an SBML `SId` is replaced by `_`, e.g. the dot in the id of a
`str`	layer of a nested module.

index_id ¶

index_id(index)

Get the part of an id which is the index of an element.

Parameters:

Name	Type	Description	Default
`index`	`tuple[int, ...]`	the PyTorch index of the element, empty for a value without axes.	required

Returns:

Type	Description
`str`	The axes joined by `_`, e.g. `0_1`, and `0` for a value without axes.

Raises:

Type	Description
`ValueError`	if an axis of the index is negative.

unit_id ¶

unit_id(network, node, index)

Get the id of a unit of a node of the forward pass.

Parameters:

Name	Type	Description	Default
`network`	`str`	id of the network.	required
`node`	`str`	name of the node.	required
`index`	`tuple[int, ...]`	the index of the unit in the value of the node.	required

Returns:

Type	Description
`str`	The id, e.g. `net1__tanh_1__3`. A character which is not part of an
`str`	SBML `SId` is replaced by `_`.

input_id ¶

input_id(network, k, index=None)

Get the id of an input of a network or of an element of it.

Parameters:

Name	Type	Description	Default
`network`	`str`	id of the network.	required
`k`	`int`	the position of the input in the inputs of the forward pass.	required
`index`	`tuple[int, ...] \| None`	the index of the element, `None` for the input as an array.	`None`

Returns:

Type	Description
`str`	The id, e.g. `net1__input0__1` for an element and `net1__input0` for
`str`	the array.

Raises:

Type	Description
`ValueError`	if the position or an axis of the index is negative.

output_id ¶

output_id(network, k, index)

Get the id of an element of an output of a network.

Parameters:

Name	Type	Description	Default
`network`	`str`	id of the network.	required
`k`	`int`	the position of the output in the outputs of the forward pass.	required
`index`	`tuple[int, ...]`	the index of the element.	required

Returns:

Type	Description
`str`	The id, e.g. `net1__output0__0`.

Raises:

Type	Description
`ValueError`	if the position or an axis of the index is negative.

parse_io_id ¶

parse_io_id(network, kind, sid)

Read the position and the index from the id of an input or an output.

Parameters:

Name	Type	Description	Default
`network`	`str`	id of the network.	required
`kind`	`str`	`input` or `output`.	required
`sid`	`str`	the id, e.g. `net1__input0__1` or `net1__input0`.	required

Returns:

Type	Description
`int`	The position and the index of the element, `None` as index for an
`tuple[int, ...] \| None`	id without one, i.e. for an input as an array.

Raises:

Type	Description
`ValueError`	if the id is not the id of an input or output of the network.

copy_parameters ¶

copy_parameters(parameters)

Copy the arrays of a network.

Parameters:

Name	Type	Description	Default
`parameters`	`Mapping[str, Mapping[str, ndarray]]`	the arrays, layer id -> array name -> values.	required

Returns:

Type	Description
`NetworkParameters`	A copy which shares no array with the original.

load_array_data ¶

load_array_data(path)

Read an array file.

Parameters:

Name	Type	Description	Default
`path`	`Path`	the HDF5 file.	required

Returns:

Type	Description
`ArrayData`	The arrays of the file.

Raises:

Type	Description
`NetworkImportError`	if the file does not exist, cannot be read or is not an array file.

sciml.network¶

Network dataclass ¶

from_files classmethod ¶

read_arrays ¶

check_forward ¶

array_specs ¶

used_layers ¶

backends ¶

check_arrays ¶

forward ¶

parameter_ids ¶

with_values ¶

element_id ¶

index_id ¶

unit_id ¶

input_id ¶

output_id ¶

parse_io_id ¶

copy_parameters ¶

load_array_data ¶

Network `dataclass` ¶

from_files `classmethod` ¶