Skip to content

testsuite.runner

Running a semantic case of the SBML Test Suite.

run_case reads the model of a case, simulates it as its settings ask and compares the results with the expected ones. It answers with a CaseResult rather than raising: a case whose model cannot be read or whose integration fails is a result of the suite like any other, and the report groups the cases by exactly these outcomes.

CaseStatus

Bases: StrEnum

The outcome of a case of the suite.

pass : the results are within the tolerances of the case.

tolerance : the case was simulated and its results are outside the tolerances, i.e. the simulation is wrong.

not_read : the model could not be loaded, e.g. an SBML construct which the simulator does not support.

simulation_error : the model was loaded and the integration failed.

missing_variable : the simulation ran and did not produce a variable the case compares, e.g. a symbol the simulator does not expose.

The last three are failures of the simulator and not of its numerics, so a report separates them: they say something different about what is missing.

CaseResult dataclass 

CaseResult(
    cid,
    status,
    message="",
    duration=0.0,
    comparison=None,
    component_tags=frozenset(),
    test_tags=frozenset(),
    encoding="",
)

The result of running a case of the suite.

Attributes:

Name Type Description
cid str

identifier of the case.
status CaseStatus

outcome of the case.
message str

what went wrong, empty for a case which passed.
duration float

seconds the case took, including reading the model.
comparison CaseComparison | None

comparison with the expected results, None if the case was never simulated.
component_tags frozenset[str]

SBML components of the case, for the report.
test_tags frozenset[str]

what the case tests, for the report.
encoding str

SBML encoding which was simulated, e.g. l3v2.

passed property 

passed

Check whether the case passed.

to_dict 

to_dict()

Convert to a dictionary of JSON serializable values.

simulate_case 

simulate_case(case, simulator)

Simulate a case on a simulator which has its model loaded.

Parameters:

Name Type Description Default
case SemanticCase

the case to simulate.

 required
simulator SimulatorSerial

simulator with the model of the case.

 required

Returns:

Type Description
DataFrame

The results with a time column and one column per selection.

run_suite 

run_suite(suite, cids=None)

Run the cases of a suite.

Parameters:

Name Type Description Default
suite SemanticSuite

the suite to run.

 required
cids Iterable[str] | None

identifiers to run, all timecourse cases by default.

 None

Returns:

Type Description
list[CaseResult]

The results, in the order of the case identifiers.

run_case 

run_case(case)

Run a case of the suite and compare it with the expected results.

Every failure is a result: a model which cannot be read, an integration which fails and results outside the tolerances are the outcomes the report is grouped by, so nothing is raised.

Parameters:

Name Type Description Default
case SemanticCase

the case to run.

 required

Returns:

Type Description
CaseResult

The result of the case with its status and its comparison.