experiment.runner¶

Runner for SimulationExperiments.

The ExperimentRunner is used to execute simulation experiments. This includes - loading of datasets - loading of models - running tasks (simulation on models) - creating outputs

The experiments of a run are independent of each other, a run of several experiments runs them in a pool of worker processes (n_workers of ExperimentRunner.run_experiments): every worker initializes an experiment again from its class, loads its models once and keeps them for the next experiment, runs it serially (its scans included, a worker starts no pool) and writes its outputs; the parent keeps its own initialized experiments, which the report is created from, and logs the messages of the workers in the order of the experiments. A run is serial when an experiment cannot be sent to a worker, see ExperimentRunner._serial_reason.

The models of the experiments are loaded in the engine of the simulator of the runner, roadrunner without one, see ExperimentRunner.engine: a simulator of another engine loads them again, and a worker loads them in the engine of the run.

A run shows its progress on the console and ends with a table of the seconds every experiment took in every phase, see sbmlsim.experiment.progress: the phases are measured where they run, in a worker too, and the result of every experiment carries them (ExperimentResult.timings). The workers of a run with progress send what their experiments do through a queue, which a thread of the parent forwards to the display.

ModelKey ¶

Bases: NamedTuple

The model of an experiment in an engine, see model_key.

It decides whether two experiments can share one loaded model.

ExperimentRunner ¶

ExperimentRunner(
    experiment_classes,
    base_path=None,
    data_path=None,
    simulator=None,
    ureg=None,
    on_error="log",
    jit_optimization=None,
    **kwargs,
)

Class for running simulation experiments.

Initialize the runner.

Parameters:

Name	Type	Description	Default
`experiment_classes`	`type[SimulationExperiment] \| Iterable[type[SimulationExperiment]]`	the simulation experiments.	required
`base_path`	`Path \| None`	base path of the simulation experiments, `None` for none, e.g. for experiments whose models are absolute paths.	`None`
`data_path`	`Path \| Iterable[Path] \| None`	path or paths of the datasets of the simulation experiments, `None` for experiments without datasets of files.	`None`
`simulator`	`Simulator \| None`	the simulator of the tasks, the models are loaded in its engine, see `engine`.	`None`
`ureg`	`UnitRegistry \| None`	the unit registry of the experiments.	`None`
`on_error`	`Literal['raise', 'log']`	what an error in the definition of an experiment does (its models, datasets, simulations, tasks, data, figures, fit mappings and their checks). `"log"` (the default) logs it with its traceback and goes on with the next experiment: the experiment is in `failed` and not in `experiments`, and `run_experiments` reports it as failed without running it. `"raise"` raises it, which is what a fit uses.	`'log'`
`jit_optimization`	`JitOptimization \| str \| None`	the optimization level every model is compiled with, see `RoadrunnerSBMLModel`.	`None`
`**kwargs`	`Any`	settings of the experiments, `SimulationExperiment.settings`.	`{}`

engine `property` ¶

engine

Get the engine the models are loaded in: the one of the simulator.

Roadrunner without a simulator.

set_simulator ¶

set_simulator(simulator)

Set the simulator on the runner and the experiments.

A simulator of another engine than the one of the loaded models loads them again in its engine, the models of the runner and of every experiment, see sbmlsim.model.model_sbml.load_model; a state set on a model of the other engine is not carried over. A simulator of the same engine keeps them. None keeps the simulator.

initialize ¶

initialize(experiment_classes, **kwargs)

Initialize ExperimentRunner.

Initialization is required in addition to construction to allow serialization of information for parallelization.

run_experiments ¶

run_experiments(
    output_path,
    show_figures=False,
    save_results=False,
    figure_formats=None,
    reduced_selections=True,
    keep_results=False,
    raise_on_failure=False,
    figure_settings=None,
    n_workers=None,
    progress=None,
)

Run the experiments and write their outputs.

The run shows its progress on the console of the package, a bar over the experiments and a row per running experiment with what it does, and ends with a table of the seconds every experiment took in every phase, see sbmlsim.experiment.progress. The seconds are in the results as well, ExperimentResult.timings.

Parameters:

Name	Type	Description	Default
`output_path`	`Path`	directory of the outputs, one directory per experiment.	required
`show_figures`	`bool`	show the matplotlib figures.	`False`
`save_results`	`bool`	write the results of the tasks.	`False`
`figure_formats`	`list[str] \| None`	formats of the figures.	`None`
`reduced_selections`	`bool`	simulate only the selections an experiment uses.	`True`
`keep_results`	`bool`	keep the results of every experiment after its outputs are written. By default they are released, so a run holds only the results of the experiment it runs and not of all experiments until its end; `SimulationExperiment.results` then raises.	`False`
`raise_on_failure`	`bool`	raise an `ExperimentRunError` after every experiment ran when an experiment or a figure failed. By default the failures are logged and recorded in the results (`ExperimentResult.failed`) only.	`False`
`figure_settings`	`FigureSettings \| Mapping[str, Any] \| None`	the settings of the figures of every experiment, a `FigureSettings` or overrides of its defaults, which the `figure_settings` of an experiment and the settings of a figure override.	`None`
`n_workers`	`int \| None`	the processes which run the experiments: `1` runs them in the calling process, a number in a pool of that many workers (at most one per experiment), `None` in a pool of a worker per `EXPERIMENTS_PER_WORKER` experiments, at most one per CPU, i.e. serially for a few experiments, and serially in a worker. In a pool every experiment runs serially in its worker, its scans included; a run with `show_figures`, `keep_results`, a unit registry of its own or an experiment which cannot be pickled is serial, see the module. The pool starts worker processes, a script which runs it needs the guard `if __name__ == "__main__":`.	`None`
`progress`	`bool \| None`	show the progress and the timings on the console; `None` shows them on a terminal and in a notebook and not when the output is redirected, see `progress.show_progress`. `False` prints nothing, the progress bars of the scans of the tasks included. A run without progress inside of `progress.reporting` reports its events to the listener of that context, also from its workers.	`None`

Returns:

Type	Description
`list[ExperimentResult]`	The results of the experiments, which the report is created from.

Raises:

Type	Description
`ExperimentRunError`	with `raise_on_failure`, if something failed.
`ValueError`	if a key of `figure_settings` is no setting or `n_workers` is less than 1, before any experiment runs.

model_key ¶

model_key(abstract_model, engine=ROADRUNNER)

Get the key a loaded model is cached under.

Two experiments share a loaded model when they resolve to the same source, the same language, the same changes, the same added parameters and the same selections in the same engine; the changes and the selections are part of it because they are applied to the model when it is loaded. The source of a file is its resolved path, the one of inline SBML its content and, for a hierarchical model, the directory its external models are resolved against, see SBMLModel. The key is the description of the model and not the object: an experiment builds its own AbstractModel, which has no equality of its own, so caching on the object would never find anything and every experiment would load and compile the model again.

Parameters:

Name	Type	Description	Default
`abstract_model`	`AbstractModel`	the model of an experiment.	required
`engine`	`Engine \| str`	the engine the model is loaded in.	`ROADRUNNER`

Returns:

Type	Description
`ModelKey`	The key, hashable and equal for two descriptions of the same model in
`ModelKey`	the same engine.

run_experiments ¶

run_experiments(
    experiments,
    output_path,
    base_path=None,
    data_path=None,
    show_report=False,
    raise_on_failure=False,
    figure_settings=None,
    n_workers=None,
    progress=None,
    simulator=None,
    figure_formats=None,
    save_results=False,
    reduced_selections=True,
    jit_optimization=None,
    uncertainty=None,
)

Run simulation experiments and write their report to the output path.

Parameters:

Name	Type	Description	Default
`experiments`	`type[SimulationExperiment] \| list[type[SimulationExperiment]]`	the simulation experiments.	required
`output_path`	`Path`	directory of the results and the report.	required
`base_path`	`Path \| None`	base path of the simulation experiments.	`None`
`data_path`	`Path \| Iterable[Path] \| None`	path or paths of the datasets of the simulation experiments.	`None`
`show_report`	`bool`	open the report in a web browser.	`False`
`raise_on_failure`	`bool`	raise an `ExperimentRunError` when an experiment or a figure failed, after every experiment ran and the report was written.	`False`
`figure_settings`	`FigureSettings \| Mapping[str, Any] \| None`	the settings of the figures of every experiment, see `ExperimentRunner.run_experiments`.	`None`
`n_workers`	`int \| None`	the processes which run the experiments, see `ExperimentRunner.run_experiments`.	`None`
`progress`	`bool \| None`	show the progress and the timings of the run, see `ExperimentRunner.run_experiments`.	`None`
`simulator`	`Simulator \| None`	the simulator of the tasks, `Simulator()` without one; the models are loaded in its engine, e.g. `Simulator(engine="copasi")` or `Simulator(engine="amici")`.	`None`
`figure_formats`	`list[str] \| None`	the formats of the figures, see `ExperimentRunner.run_experiments`.	`None`
`save_results`	`bool`	write the results of the tasks.	`False`
`reduced_selections`	`bool`	simulate only the selections an experiment uses.	`True`
`jit_optimization`	`JitOptimization \| str \| None`	the optimization level every model is compiled with, see `ExperimentRunner`.	`None`
`uncertainty`	`UncertaintyRun \| None`	run every experiment over the draws of the fitted parameters, its setting `uncertainty`, see `sbmlsim.experiment.draws`; `None` runs them once.	`None`

Returns:

Type	Description
`list[ExperimentResult]`	The results of the experiments.

Raises:

Type	Description
`ExperimentRunError`	with `raise_on_failure`, if something failed.

experiment.runner¶

ModelKey ¶

ExperimentRunner ¶

engine property ¶

set_simulator ¶

initialize ¶

run_experiments ¶

model_key ¶

run_experiments ¶

engine `property` ¶