Skip to content

testsuite.comparison

Comparison of the results of a case with the expected results.

The SBML Test Suite states when a simulation is correct: a value u is within the tolerances of the expected value c if |c - u| <= abs_tol + rel_tol * |c| with the tolerances of the case, see the README of the semantic cases. sbmlsim.comparison.diff.within_tolerance is that criterion, this module applies it to a case.

CaseComparison dataclass 

CaseComparison(
    cid,
    valid,
    missing=list(),
    n_points=0,
    n_violations=0,
    worst_variable="",
    worst_expected=float("nan"),
    worst_observed=float("nan"),
    worst_time=float("nan"),
)

The comparison of the results of a case with the expected results.

Attributes:

Name Type Description
cid str

identifier of the case.
valid bool

whether every point of every variable is within the tolerances.
missing list[str]

variables the simulation did not produce.
n_points int

number of compared points, i.e. rows times variables.
n_violations int

number of points outside the tolerances.
worst_variable str

variable with the largest violation, empty if none.
worst_expected float

expected value at the largest violation.
worst_observed float

simulated value at the largest violation.
worst_time float

time of the largest violation.

summary property 

summary

Get a one line description of the comparison.

compare_case 

compare_case(case, observed)

Compare the simulation of a case with its expected results.

Both tables are the time followed by one column per variable of the case, in the order of the case: that is how the results of the suite are written and how roadrunner answers selections. The columns are therefore matched by position and not by name, which a case cannot be trusted on, since its model may carry a parameter named time and two variables may differ in case alone; cases 01820 and 01821 have the parameters time, Time and TIME and used to compare a variable against every column at once.

Parameters:

Name Type Description Default
case SemanticCase

the case which was simulated.

 required
observed TimecourseResult

results of the simulation, the time in the first column.

 required

Returns:

Type Description
CaseComparison

The comparison, valid if every point is within the tolerances.