Skip to content

sciml.backend

The backends the layers and functions of a network are written against.

A layer is implemented once. It uses the array operations of numpy (@, reshape, sum, concatenate), which work on arrays of numbers and on object arrays of expressions alike, and takes everything which differs between the two from a Backend: the elementwise functions, the functions with a condition and the normalization of softmax.

NumpyBackend works on float arrays and is the forward pass. SympyBackend works on object arrays of sympy expressions and is the first half of the compilation of a network into an SBML model, see sbmlsim.sciml.compiler. The layers are the same for both.

BackendKind

Bases: StrEnum

The kinds of backends a layer declares its support for.

Backend

Bases: ABC

The elementwise functions and the array type of a forward pass.

Every method takes and returns arrays of the dtype of the backend and works elementwise, with the broadcasting of numpy.

Attributes:

Name Type Description
kind BackendKind

the kind of the backend, which a layer declares its support for.
dtype type

the dtype of the arrays, float or object.

asarray 

asarray(values)

Convert values to an array of the backend.

Parameters:

Name Type Description Default
values Any

an array, a nested sequence or a scalar.

 required

Returns:

Type Description
ndarray

The array with the dtype of the backend.

exp abstractmethod 

exp(x)

Calculate the exponential function.

log abstractmethod 

log(x)

Calculate the natural logarithm.

tanh abstractmethod 

tanh(x)

Calculate the hyperbolic tangent.

sqrt abstractmethod 

sqrt(x)

Calculate the square root.

erf abstractmethod 

erf(x)

Calculate the error function.

absolute abstractmethod 

absolute(x)

Calculate the absolute value.

select abstractmethod 

select(x, threshold, below, above)

Choose between two values by a condition on x.

This is the one function with a condition, every piecewise activation is written with it.

Parameters:

Name Type Description Default
x ndarray

the values the condition is evaluated on.

 required
threshold float

the threshold of the condition.

 required
below ndarray | float

the result where x <= threshold.

 required
above ndarray | float

the result where x > threshold.

 required

Returns:

Type Description
ndarray

above where x > threshold and below elsewhere.

stabilizer abstractmethod 

stabilizer(x, axis)

Get a shift which keeps the exponentials of softmax finite.

softmax and log_softmax do not change when a value which is constant along axis is subtracted from x. Both backends return the maximum along the axis, a backend on expressions as the expression Max, so that a model with the network does not overflow where the forward pass does not.

Parameters:

Name Type Description Default
x ndarray

the input of softmax.

 required
axis int

the axis softmax normalizes over.

 required

Returns:

Type Description
ndarray

The shift, which broadcasts against x.

softmax 

softmax(x, axis)

Evaluate exp(x) / sum(exp(x)) along an axis.

The exponentials are shifted by the stabilizer, which is what PyTorch does.

Parameters:

Name Type Description Default
x ndarray

the values.

 required
axis int

the axis the values are normalized over.

 required

Returns:

Type Description
ndarray

The values of the softmax, of the shape of x.

NumpyBackend

Bases: Backend

The backend on arrays of numbers, i.e. the forward pass.

exp 

exp(x)

Calculate the exponential function.

An overflow is inf and not a warning: the branch of a select which is not chosen is evaluated as well.

log 

log(x)

Calculate the natural logarithm.

tanh 

tanh(x)

Calculate the hyperbolic tangent.

sqrt 

sqrt(x)

Calculate the square root.

erf 

erf(x)

Calculate the error function.

absolute 

absolute(x)

Calculate the absolute value.

select 

select(x, threshold, below, above)

Choose between two values by a condition on x.

stabilizer 

stabilizer(x, axis)

Get the maximum along the axis.

SympyBackend

Bases: Backend

The backend on object arrays of sympy expressions.

The elements of the arrays are symbols, numbers and expressions of them. A function with a condition is a sympy.Piecewise, which is the piecewise of the MathML of SBML. erf is evaluated as sympy.erf, which the MathML of SBML does not have: the compiler rejects an expression with it.

exp 

exp(x)

Calculate the exponential function.

log 

log(x)

Calculate the natural logarithm.

tanh 

tanh(x)

Calculate the hyperbolic tangent.

sqrt 

sqrt(x)

Calculate the square root.

erf 

erf(x)

Calculate the error function.

absolute 

absolute(x)

Calculate the absolute value.

select 

select(x, threshold, below, above)

Choose between two values by a condition on x, as a Piecewise.

softmax 

softmax(x, axis)

Evaluate the softmax as 1 / sum_j exp(x_j - x_i) along an axis.

The expression needs no maximum: an exponential overflows only in the sum of a unit whose value is below the smallest number, and the unit is its limit 0. A maximum would be a part of every exponential, and roadrunner inlines the assignment rules: the rules of a softmax over n units would grow with n**3 instead of n**2.

Parameters:

Name Type Description Default
x ndarray

the expressions.

 required
axis int

the axis the values are normalized over.

 required

Returns:

Type Description
ndarray

The expressions of the softmax, of the shape of x.

stabilizer 

stabilizer(x, axis)

Get the maximum along the axis, as the expression Max.