Analyses¶

python -m sbmlsim.run runs the analyses of a model package from the command line: its simulation experiments, fits, profile likelihoods, scans, sensitivity and uncertainty analyses, the build of its models and the actions of its own. It runs an analysis of a project from a spec, a small YAML or JSON file of its settings, into an output directory. The same analysis runs whole on one machine or in units on several machines, which merge to the result of the whole run, so a fit of 64 repeats or a scan of 10000 points can be spread over a cluster with simrun and give the same numbers as on a laptop.

Quick start¶

The HCTZ example of the fitting has a project, examples.hctz_fitting.project:PROJECT. Run from the root of the repository, which python -m puts on the path, list its analyses:

python -m sbmlsim.run list examples.hctz_fitting.project:PROJECT

kind             name   description
simulate         all    the simulation experiments of the project
fit              PK     fit of the problem 'PK'
identifiability  PK     profile likelihoods of the fit of the problem 'PK'
fisher           PK     Fisher information of the fit of the problem 'PK'
report           PK     report of the fit of the problem 'PK'
petab            PK     PEtab v2 problem of the fit problem 'PK'
fit              IV     fit of the problem 'IV'
...
scan             dose   oral doses of 12.5, 25 and 50 mg over one day
scan             sobol  Sobol design of the dissolution and the renal excretion of a 25 mg oral dose
sensitivity      sobol  sobol indices of the scan 'sobol'
uncertainty      cl     the AUC of a 25 mg oral dose under a renal excretion uncertain by 30 %

A spec names the project and the analysis:

project: examples.hctz_fitting.project:PROJECT
subset: PK
repeats: 8
seed: 1

and the run fits the problem, then profiles the parameters of the fit and writes the report of both:

python -m sbmlsim.run fit fit.yaml --out results/fit
python -m sbmlsim.run identifiability profiles.yaml --out results/profiles --input fit=results/fit
python -m sbmlsim.run report report.yaml --out results/report --input fit=results/profiles

where profiles.yaml and report.yaml hold the same project and subset. The fit uses every CPU it may use, see CPU quota, --workers 4 limits it.

The command¶

python -m sbmlsim.run <kind> <spec> --out DIR [--workers N] [--resume] [--provenance FILE] [--input PORT=PATH ...] [--summary FILE]
python -m sbmlsim.run plan <kind> <spec> [-o FILE]
python -m sbmlsim.run unit <kind> <spec> <index> --out DIR [--input PORT=PATH ...]
python -m sbmlsim.run merge <kind> <spec> --out DIR [--input PORT=PATH ...] [--summary FILE]
python -m sbmlsim.run list <project> [--json] [--experiments]
python -m sbmlsim.run verify <subcommand> ...

The first form runs an analysis whole (its command is run, which may be left out). plan, unit and merge run it in units, list lists the analyses of a project as a table or, with --json, as JSON; with --experiments it also lists the groups of the simulation experiments and every experiment (kind simulate, which runs them by these names), which imports the experiments. verify is a family of commands that compare the results of two checkouts, see Verify. --profile, anywhere in the command line, runs a command which writes into --out under the sampling profiler of python 3.15, see Profiling.

plan writes the planned spec as JSON to stdout or to -o FILE; the messages of every command go to stderr. unit and merge take no --workers, --resume or --provenance: a unit runs serially in its process, a unit or a merge into an output with files continues it, and the provenance of their sources is the file of the environment variable SIMRUN_PROVENANCE, if set.

Kinds¶

A spec forbids unknown fields. The fit kinds name a fit problem, the other kinds an analysis of the project:

fit¶

The plan refuses a fit whose upper bounds let a remainder of a model of the project become negative, an initial assignment f_c = 1 - f_a - f_b whose fitted fractions and the values of the others sum above 1 (sbmlsim.fit.project_fits.remainder_problems), and checks the start before a repeat runs.

identifiability¶

The settings are the ones of the profile likelihood. A profile is computed on the problem of the studies the fit was restricted to and with the settings the fit ran with, both read from its records; the same holds for fisher, report and petab.

fisher, report and petab¶

adopt adopts the fit of its input fit, the out of a fit or a records directory: the id of its result, which has the form of the id of a fit of a subset of the project, names the subset; it needs the profiles or the Fisher information of the fit and writes its record into fits/<fit id> and adopted.json with the fit adopted for its subset, as fit with adopt does. | petab | required_extension | true | mark the extension sbmlsim as required |

simulate, scan, sensitivity and uncertainty¶

The scan, its observables and its design are defined in the project, so the spec of a scan, a sensitivity or an uncertainty analysis only names it, e.g. the Sobol indices of the scan sobol and the propagation cl with a seed of its own:

project: examples.hctz_fitting.project:PROJECT
analysis: sobol

project: examples.hctz_fitting.project:PROJECT
analysis: cl
seed: 7

python -m sbmlsim.run sensitivity sobol.yaml --out results/sobol
python -m sbmlsim.run uncertainty cl.yaml --out results/cl

A sensitivity analysis of the project over the groups of its simulations (Project.sensitivities, see Sensitivity analysis) is run by the same kind; the spec names it and may choose its analyses:

project: my_model:PROJECT
analysis: decay
analyses: [local, sampling]

python -m sbmlsim.run sensitivity decay.yaml --out results/sensitivity

Every analysis is a unit, which writes its folder of the output, and the merge writes the table of the parameters; a resumed run keeps the analyses whose folder was written by the same plan. The plan fixes the numbers of samples: a unit run with another SBMLSIM_SENSITIVITY_N than its plan fails.

The output¶

Every run writes the same files describing it into its output, besides its results:

The results of the kinds:

The input fit of a kind is the output of a fit run, whose records are in its directory <opid>, or a directory of records itself, e.g. the output of an identifiability or fisher run, which copy the records of their fit, so a report shows the fit and its analyses from one directory. Every file is written atomically: a reader sees the old or the new file and never a partial one. A run never writes into the project: even adopt writes adopted.json and the record fits/<opid> into the output, which you copy into the directory of the fit records of the project (fits_dir) and commit to adopt the fit. A report of a project has its context: the title of the project and the adopted fits the fit kept fixed, as its records name them.

The results are the files of the classes of sbmlsim, which read them back: OptimizationResult.from_json reads optimization_result.json of a fit, IdentifiabilityResult.from_json reads identifiability.json, ScanResult.from_netcdf reads result.nc of a scan, SensitivityResult.from_netcdf reads sensitivity.nc and UncertaintyResult.from_netcdf reads the directory uncertainty/. The fitted parameters of parameters.json are the input of a report or of the identifiability tools.

Units and merges¶

A run of a split kind computes units, each of its own files, and merges them. python -m sbmlsim.run <kind> does all of it in one process and its pool. The commands plan, unit and merge do it in separate processes, which may run at the same time, on one machine or on several machines which share the output directory. First write the scan of three doses as a spec:

project: examples.hctz_fitting.project:PROJECT
analysis: dose

plan completes the spec into what every unit computes alike and sets the number of units, units:

python -m sbmlsim.run plan scan dose.yaml -o dose-planned.json

{
  "units": 3,
  "project": "examples.hctz_fitting.project:PROJECT",
  "analysis": "dose",
  "scan_hash": "cdbdb9ebc0c25da8b9a67857cd198d315c47ffb183a6330148b3ed4b4f3ce080"
}

The units are numbered from 0 to units - 1. Compute each one into the shared output, in any order, here one after another (& and wait, or one machine per unit, run them in parallel), then merge them:

for i in 0 1 2; do python -m sbmlsim.run unit scan dose-planned.json $i --out results/dose; done
python -m sbmlsim.run merge scan dose-planned.json --out results/dose

The merge writes result.nc, the same result as a whole run, python -m sbmlsim.run scan dose.yaml --out results/dose-whole. A unit which is missing makes the merge exit with 3 and name it, e.g. 1 of 3 units are missing: 1, so compute that unit and merge again. A fit splits the same way, and so does a profile likelihood, whose unit and merge need the records of the fit as their input:

python -m sbmlsim.run plan fit fit.yaml -o fit-planned.json
for i in $(seq 0 7); do python -m sbmlsim.run unit fit fit-planned.json $i --out results/fit; done
python -m sbmlsim.run merge fit fit-planned.json --out results/fit

python -m sbmlsim.run plan identifiability profiles.yaml -o profiles-planned.json
python -m sbmlsim.run unit identifiability profiles-planned.json 0 --out results/profiles --input fit=results/fit
python -m sbmlsim.run merge identifiability profiles-planned.json --out results/profiles --input fit=results/fit

with one unit per parameter of pids in profiles-planned.json. The rules:

• A fit, a scan and a propagation of a given number of draws are cut into at most 64 units of consecutive items, max(1, ceil(n / 64)) repeats, points or draws each; a profile likelihood has one unit per parameter, and the adaptive number of draws of a propagation is one unit, whose rounds are its checkpoints.
• The plan of a fit and of a propagation draws the seed when the spec has none, so every unit draws the same start values or the same points. The plan of a scan records the hash of the identity of the scan, which every unit and the merge compare with the scan they build: its record without the numbers a design draws and the references it reads (scan_identity: the method, the distributions and the options of a design with its seed and its number of points, the labels, the targets, and the whole record of the simulation and of a dimension of given values), so a unit on another platform than the plan, whose numerics differ in the last bits, builds the same scan. The plan builds the scan twice and compares the whole records; a scan which builds other points every time, e.g. a design without a seed, is refused.
• A whole run computes the same units on the workers of the run: the consecutive blocks of a scan or of a propagation of a given number of draws which have no file run in spans of as many blocks as the run has workers, a span one run of the simulator, so it is parallel whatever the size of a block, and the result of a span is cut into the files of its blocks, each the file its unit writes (a block whose messages of failed points the span does not keep, more than ten failed points before them, runs alone).
• The first unit writes spec.json into the output, and a unit or a merge of another plan is refused, so the units of two plans never mix. A unit writes no run.json, the merge does.
• A unit computed again, e.g. after its machine was lost, writes the same complete files again, and a unit whose files exist is not computed again; the merge names the units it misses and exits with 3.
• The merged result equals the result of the whole run bitwise with roadrunner, apart from the times and durations of the records: the blocks of a scan combine to the ScanResult of the whole scan (ScanResult.combine), the repeats of a fit to its OptimizationResult, the profiles to the IdentifiabilityResult (IdentifiabilityResult.combine) and the blocks of draws to the UncertaintyResult, every block taking its rows of the points of the design of all the draws, so a Latin hypercube stays one hypercube.

Checkpoints and resume¶

The units are the checkpoints of a run: a repeat of a fit is written once it finished, a profile once both of its scans finished, a block of points or draws once it ran, in a whole run once its span of about workers blocks ran. A run which was interrupted, by Ctrl-C, a lost machine or a time limit, continues with --resume into the same output, which computes only what is missing:

python -m sbmlsim.run fit fit.yaml --out results/fit --resume

A resume runs the planned spec of spec.json, so a drawn seed stays, and is accepted only for the spec file the run was started with (and not for spec.json, the planned spec); its inputs are left out of the comparison, so they may be staged at another path. A run into an output which is not empty without --resume is refused, and so is a resume with another spec, as both would mix the results of two runs. A resume of a finished run runs it again, keeps every result and merges again; --resume into an empty output starts a run, and progress.jsonl collects the events of every attempt.

Progress and provenance¶

progress.jsonl has one JSON line per step of the analysis, which a program outside of the run can follow, e.g. tail -f:

{"time": "2026-11-06T13:03:03.374+00:00", "event": "scan.chunk", "points": 1, "done": 1, "total": 1}
{"time": "2026-11-06T13:03:03.490+00:00", "event": "scan.block", "analysis": "dose", "start": 0, "stop": 1, "points": 3}

The provenance of a run is the git revision of the root of the project, recorded in run.json. A run on a snapshot of the sources, e.g. on a cluster, has no checkout: --provenance FILE, or the file the environment variable SIMRUN_PROVENANCE names, is the JSON list of its sources (name, branch, commit, uncommitted changes), which run.json, the records of a fit (FitProvenance.sources) and the methods of its report record instead of the git revision.

Exit codes¶

An output which a run may not run into raises sbmlsim.run.OutInUse, a ConfigurationError with the output (out) and why (reason: not_a_directory, not_empty without --resume, no_record and unreadable for a resume without a readable run.json, other_run for the record of another kind or spec). sbmlsim.run.check_out(kind, spec, out, resume=...) makes these checks and the validation of the spec without running and without writing anything, so a wrapper refuses before it starts:

from sbmlsim import run

try:
    run.check_out("release", {"project": "my_model:PROJECT", "version": "0.8.0"}, out)
except run.OutInUse as err:
    print(err.reason, err.out)

Verify¶

python -m sbmlsim.run verify <subcommand> checks that a change of a model changes only what it is meant to change: a refactoring, an update of sbmlsim or a refresh of the study data should leave the simulations of the model as they were. It is a family of local commands like plan and list, not a kind: it compares the results of checkouts on this machine, writes only where it is told to (--out) and has no units, no merge and no simrun task type. The result of a command goes to stdout, the progress and the problems are logged on stderr. The exit code is 0 when nothing differs, 1 when a difference is found and 2 for a command that cannot be run as it was asked (an unknown project or experiment, a bad option, a missing directory).

git worktree add --detach ../base develop
# the experiments simulate the models of the project, which are generated:
# create them with the factory of each checkout right before its dump
python -m sbmlsim.run verify dump --root ../base --project my_models.hctz:PROJECT --out base.pkl --atol 1e-16 --rtol 1e-12 --grid 1000
python -m sbmlsim.run verify dump --root . --project my_models.hctz:PROJECT --out new.pkl --order base.pkl --atol 1e-16 --rtol 1e-12 --grid 1000
python -m sbmlsim.run verify compare base.pkl new.pkl

--project is the reference module:attribute of the project and --root the directory put first on the python path, e.g. a checkout (the project must be imported from below it). The variant of the model a task runs on is stored with every task: the variant SimulationExperiment.variant_of(simulation) names, which a framework that builds a model per group of subjects implements, else the variant of the project whose file is the source of the model of the task; a study that moves to another variant is a difference of its own. The renames file has the parts metadata (fields, members and added, the renamed metadata of the fit mappings), strings and ids (renamed texts and ids), sids (true when the ids became SIds), figure_fields (fields of a figure left out of both dumps) and expected (patterns of the keys per part of the dump, results, datasets, mappings, figures, ranges and builds, whose differences are expected: they are printed and not counted, and an expected difference that does not occur is counted).

CPU quota and threads¶

The number of processes of a run is the number of CPUs the process may use, bounded by the CPU quota of its cgroup, so a run in a container with a quota of 4 CPUs on a host of 64 starts 4 workers and not 64. With cgroup v2 the quota is the smallest one of the cgroup of the process and its parents (cpu.max), with cgroup v1 the one quota of the cgroup as the container sees it (cpu.cfs_quota_us over cpu.cfs_period_us), rounded down and at least 1; the pools of sbmlsim (Simulator, a fit, the experiments) bound their default size the same way.

A run uses its CPUs through the processes of its pools, so the numerical libraries run with one thread each: the command sets OMP_NUM_THREADS, OPENBLAS_NUM_THREADS and MKL_NUM_THREADS to 1 unless they are set, starting itself once more with them, and draws its figures with the matplotlib backend Agg (MPLBACKEND unless set).

Profiling¶

--profile runs a command which writes into --out (run, unit, merge) under the statistical sampling profiler of python 3.15 (profiling.sampling, PEP 799), which samples the stack of the command from a process of its own, so the command runs at nearly its normal speed:

python -m sbmlsim.run simulate simulate.yaml --out results/simulate --profile

After the command the profile is written into profiling/<command>_<kind>_<time>/ of the output (of a unit profiling/unit_<kind>_<index>_<time>/, so the units which compute into one output at once do not share a folder): flamegraph.html (the interactive flame graph), heatmap/ (the source lines with their samples), profile.pstats (for pstats and its viewers), summary.txt (the functions with the most samples) and profile.bin (the samples, against which a later profile is compared, python -m profiling.sampling replay --diff-flamegraph profile.bin). The command exits with its own code, which it records for the profiler (the profiler exits with 0 whatever the command did), 1 if it recorded none; a failure of the profiler or of the conversion of the profile is logged and does not change the code. With an older python --profile is a bad command line (exit code 2) and the command does not run.

sbmlsim.run.profiling.run_profiled(arguments, folder, module=...) profiles any entry point which runs as python -m <module>, e.g. the command line of a framework of its own, which calls record_exit_code(code) before it exits. The profiler runs with one change: python 3.15.2 reads now and then a name of a frame while it changes, which broke the writer of the profile and ended the command; such a sample is left out and their number reported on stderr.

Projects¶

A project, sbmlsim.project.Project, describes a model package once: its model variants, factory, experiments, fits, scans, uncertainty and sensitivity analyses and the settings of its simulator, figures, checks and release. It is an attribute of a module of the package, which the run layer imports, and every spec names it as project: module:attribute. The fields, the references and the project of the HCTZ example are described on the page Projects.

Credibility¶

The kind credibility assesses the contexts of use of a model declared in Project.credibility against its adopted fits into out/credibility/, with the options verification, battery, cross_simulator, lock, uncertainty, variability and source_commit and the inputs cross_validation and record; the declaration, the evidence, the verdicts and the options are described on the page Credibility.

python -m sbmlsim.run credibility credibility.yaml --out results/credibility
python -m sbmlsim.run credibility credibility.yaml --out results/report --input record=results/credibility

Release¶

The kind release builds the release of a project as a repository of its own into out/repository; it commits nothing, and python -m sbmlsim.release publish publishes the build. The build, the release metadata (Project.release), the review and the publishing are described on the page Release.

project: my_model:PROJECT
version: 0.8.0

python -m sbmlsim.run release release.yaml --out results/release

Checks of a project¶

sbmlsim.project_checks has the checks of a model package as functions, which its tests call (check_model, check_registry, initialize_experiments, run_all_experiments, check_battery, start_fits), see Projects.

simrun¶

simrun runs the tasks of a workflow on a cluster. sbmlsim registers its kinds but action as the task types sbmlsim.<kind> of simrun, sbmlsim.fit, sbmlsim.identifiability, sbmlsim.scan, sbmlsim.adopt, sbmlsim.factory, sbmlsim.credibility, sbmlsim.release and so on (verify is a command and no kind, so it has no task type until a workflow needs one); the extra simrun installs simrun, which needs python 3.14:

pip install "sbmlsim[simrun]"

The settings of a task are the spec of its kind. A split kind (fit, identifiability, scan, sensitivity, uncertainty) is split by simrun: its workers compute the units and its merge merges them. The other kinds run as the command python -m sbmlsim.run <kind> {settings} --out {out}/out --summary {out}/summary.json --resume --input={input}, whose --resume lets a retry after an eviction continue the run it finds. A task writes the output of its run into its output port out, which a task downstream reads as the input <task>.out, and its summary into summary.json of the task.

A workflow which fits the HCTZ problem in units, profiles its parameters and writes the report of both:

name: hctz-fit
tasks:
  - id: fit
    type: sbmlsim.fit
    settings:
      project: examples.hctz_fitting.project:PROJECT
      subset: PK
      repeats: 64
  - id: identifiability
    type: sbmlsim.identifiability
    settings:
      project: examples.hctz_fitting.project:PROJECT
      subset: PK
    inputs:
      fit: fit.out
  - id: report
    type: sbmlsim.report
    settings:
      project: examples.hctz_fitting.project:PROJECT
      subset: PK
    inputs:
      fit: identifiability.out

simrun submit hctz-fit.yaml --source sbmlsim=~/git/sbmlsim

The HCTZ project lives in the repository of sbmlsim, which is the only source here. A project of a model package outside of sbmlsim adds the source of the model first and then the one of sbmlsim, as simrun orders them, e.g. simrun submit fit.yaml --source model=~/git/my_model --source sbmlsim=~/git/sbmlsim.

The fit is 64 units of one repeat, the profile likelihood one unit per parameter, and the report reads the fit and its profiles from the output of the identifiability analysis.

• The plan runs in the client. simrun submit plans every task whose type is installed where it runs, before it takes the snapshots, and submits the planned settings: the seed of the fit, the units, the parameters of the profiles. The client therefore needs sbmlsim[simrun] and the model package, so that it imports the project; a task submitted without its plan has no units and fails in the service.
• The project is imported in the pods. The pods install the snapshots editable, which installs the package of a source and not the rest of its repository: a model package is found, while the HCTZ example in examples/ needs the root of the repository on PYTHONPATH.
• Provenance. simrun writes the provenance of the snapshots of a run and names it in SIMRUN_PROVENANCE, which run.json and the records of a fit record, see Provenance.
• Development. The development environment is python 3.14 (.python-version), so uv sync --extra dev installs simrun and uv run ty check checks sbmlsim.simrun against it; the tests of the plugin skip without simrun. The tox environments, which resolve from pyproject.toml, do not install it.