credibility.verification¶

The verification of the predictions a credibility assessment decides on.

A context of use is decided on the predictions of the adopted fits at the data points of its evidence (IPRED). Whether these numbers are the solution of the model, and not of the settings of the integrator or of a broken setup, are the factors discretization error, numerical solver error and use error of V&V 40. One check per factor:

tolerances (TOLERANCES): every adopted fit evaluated again at its reference parameters with the absolute and the relative tolerance of its settings divided by TIGHTENING (tightened).
integrator (INTEGRATOR): the same at the tightened tolerances with the BDF of CVODE of at most order 2 (SECOND_METHOD_ORDER) instead of order 5, a second integration method: another formula with its own local error, stable on a stiff model (the non-stiff Adams-Moulton of CVODE takes too many steps on large stiff models).
setup (SETUP): every variant of the model is valid SBML with consistent units (build_problems) and every decisive prediction is finite.

A numerical check compares the alternative predictions with the reference predictions of the assessment (relative_differences) and passes when the largest relative difference of the decisive points of a context is at most LIMIT and the acceptance of the context does not change, see sbmlsim.credibility.assessment.

The checks are open: a Check is an id with its label and the factor of V&V 40 it assesses, registered by register_check. The checks of the predictions and of the generic battery of the model (BATTERY_CHECKS, see sbmlsim.credibility.battery) are registered here; a project registers the checks of its battery hook (sbmlsim.credibility.battery.BatteryHook.checks), e.g. the mass balance of a PBPK model, when the module of the hook is imported. check_of gets a check by its id, also one of a record whose module is not imported, which is then an unregistered check with the factor of the record.

Check `dataclass` ¶

Check(id, label, factor)

A check of the verification and the factor of V&V 40 it assesses.

The checks of the predictions (PREDICTION_CHECKS) compare the decisive predictions of a context of use; the checks of the battery (sbmlsim.credibility.battery) check the variants of the model, for every context of use alike.

Attributes:

Name	Type	Description
`id`	`str`	the id of the check, e.g. `tolerances`, as records store it.
`label`	`str`	the check in words, e.g. `second integration method`.
`factor`	`str`	the id of the factor of V&V 40 the check assesses, see `sbmlsim.credibility.FACTORS`; empty for an unregistered check of a record without it.

value `property` ¶

value

Get the id of the check, as records store it.

CheckRecord ¶

Bases: TypedDict

The result of a check for a context of use.

A check of the battery (sbmlsim.credibility.battery) is a check of the variants of the model, the same for every context.

Attributes:

Name	Type	Description
`check`	`str`	the id of the check, see `Check`.
`factor`	`str`	the id of the factor of V&V 40 the check assesses.
`n_points`	`int`	the decisive points compared, the cases of a check of the battery.
`max_difference`	`float`	the largest relative difference of a decisive prediction, `NaN` for the setup and without a point; the largest deviation of a check of the battery.
`at`	`str`	the point of the largest difference, `experiment.mapping at x`, empty without one; the case of the largest deviation of a check of the battery.
`acceptance_changed`	`bool \| None`	whether a rule of the acceptance is met with the alternative predictions and not with the reference or the other way, `None` for the setup and the battery.
`passed`	`bool`	whether the check passed.
`problems`	`list[str]`	what failed beyond the numbers, e.g. a failed evaluation or an issue of a variant.

FitSettingsRecord ¶

Bases: TypedDict

The settings of the integrator of an adopted fit and its alternatives.

Attributes:

Name	Type	Description
`fit`	`str`	the subset of the fit.
`reference`	`str`	the settings of the reference predictions, the fit's own.
`tolerances`	`str`	the settings of the check `tolerances`.
`integrator`	`str`	the settings of the check `integrator`.

VerificationSummary `dataclass` ¶

VerificationSummary(
    *,
    tightening,
    limit,
    scale_floor,
    fits,
    problems,
    battery=None,
)

What the verification of an assessment compared, as plain data.

Attributes:

Name	Type	Description
`tightening`	`float`	the factor the tolerances were divided by.
`limit`	`float`	the largest relative difference a check accepts.
`scale_floor`	`float`	the floor of a relative difference, of the largest reference prediction of the mapping.
`fits`	`list[FitSettingsRecord]`	the settings of every adopted fit and of its alternatives.
`problems`	`list[str]`	the issues of the variants of the model, see `build_problems`.
`battery`	`Battery \| None`	the checks of the variants of the model, see `sbmlsim.credibility.battery`, `None` if it did not run.

to_dict ¶

to_dict()

Get the summary as plain data.

from_dict `classmethod` ¶

from_dict(data)

Get the summary from plain data.

Verification `dataclass` ¶

Verification(*, predictions, failures, summary)

The alternative predictions and the problems an assessment is verified with.

Attributes:

Name	Type	Description
`predictions`	`Mapping[Check, Series]`	the predictions of every numerical check (`tolerances`, `integrator`) per row of the table of the fits, `NaN` where the evaluation of a fit failed.
`failures`	`Mapping[Check, Mapping[str, str]]`	the failed evaluations of every numerical check, the message by the subset of the fit.
`summary`	`VerificationSummary`	what was compared, see `VerificationSummary`.

problems `property` ¶

problems

Get the issues of the variants of the model.

register_check ¶

register_check(check)

Register a check by its id, e.g. a check of the battery hook of a project.

Registering the same check again is allowed, so the module of a hook can be imported more than once.

Parameters:

Name	Type	Description	Default
`check`	`Check`	the check.	required

Returns:

Type	Description
`Check`	The registered check.

Raises:

Type	Description
`ValueError`	for an empty id or label, a factor which is no factor of V&V 40 and an id which is registered with another label or factor.

check_of ¶

check_of(check_id, factor=None, label=None)

Get a check by its id.

A check which is not registered, e.g. a check of the battery hook of a record whose module is not imported, is a check with the label and the factor of the record, of a record of pkpdsim 0.19 the ones pkpdsim gave it, else its id in words and no factor.

Parameters:

Name	Type	Description	Default
`check_id`	`str`	the id of the check.	required
`factor`	`str \| None`	the factor of an unregistered check, as its record has it.	`None`
`label`	`str \| None`	the label of an unregistered check, as its record has it.	`None`

Returns:

Type	Description
`Check`	The registered check, or the unregistered one.

check_record_from_dict ¶

check_record_from_dict(data)

Get a check from plain data, None as NaN.

Parameters:

Name	Type	Description	Default
`data`	`Mapping[str, Any]`	the check as `CheckRecord` with `None` for `NaN`.	required

Returns:

Type	Description
`CheckRecord`	The check.

check_record_to_dict ¶

check_record_to_dict(record)

Get a check as plain data, NaN as None.

Parameters:

Name	Type	Description	Default
`record`	`CheckRecord`	the check.	required

Returns:

Type	Description
`dict[str, object]`	The check as plain data.

tightened ¶

tightened(tolerance, factor=TIGHTENING)

Divide an absolute tolerance by a factor, every kind and override.

Parameters:

Name	Type	Description	Default
`tolerance`	`float \| AbsoluteTolerance`	the tolerance, a float is the same for every kind.	required
`factor`	`float`	the factor, greater than 1 tightens.	`TIGHTENING`

Returns:

Type	Description
`AbsoluteTolerance`	The tightened tolerance.

relative_differences ¶

relative_differences(
    table, predictions, scale_floor=SCALE_FLOOR
)

Get the relative differences of alternative predictions to the reference.

The difference of a point is |alt - ref| / max(|ref|, |alt|, floor); the floor is scale_floor of the largest absolute finite reference prediction of the mapping of the point (fit, experiment and key), so a difference below it is none and a point at zero compares with the curve. A point without a finite reference prediction has no difference (NaN, the setup reports it); a finite reference with an alternative which is not finite is an infinite difference.

Parameters:

Name	Type	Description	Default
`table`	`DataFrame`	the data points with `fit`, `experiment`, `mapping` and the reference predictions `IPRED`.	required
`predictions`	`Series`	the alternative predictions per row of the table.	required
`scale_floor`	`float`	the floor relative to the largest reference prediction of the mapping.	`SCALE_FLOOR`

Returns:

Type	Description
`Series`	The relative difference per row of the table.

build_problems ¶

build_problems(paths)

Get the issues of the SBML of the variants of a model.

Every variant is checked by libsbml with the consistency checks and the checks of the units; an issue of severity warning or above is a problem.

Parameters:

Name	Type	Description	Default
`paths`	`Mapping[str, Path]`	the SBML files by the name of their variant, e.g. `renal`.	required

Returns:

Type	Description
`list[str]`	Per variant with issues its file, the number of issues and the first
`list[str]`	`LISTED_ISSUES` with their id and message; empty for valid variants.

verify ¶

verify(
    project,
    fits,
    tightening=TIGHTENING,
    *,
    battery=True,
    cross_simulator=True,
)

Evaluate the adopted fits of a project with the alternatives of the checks.

Parameters:

Name	Type	Description	Default
`project`	`Project`	the project, whose variants are checked.	required
`fits`	`FitTable`	the data points of the adopted fits with their reports, see `sbmlsim.credibility.records.evaluate_fits`.	required
`tightening`	`float`	the factor the tolerances are divided by.	`TIGHTENING`
`battery`	`bool`	run the battery of the checks of the variants, see `sbmlsim.credibility.battery.run_battery`.	`True`
`cross_simulator`	`bool`	run the check `cross_simulator` of the battery, as the settings of the battery say; `False` switches it off.	`True`

Returns:

Type	Description
`Verification`	The alternative predictions, the issues of the variants and the battery.

Raises:

Type	Description
`ValueError`	if the table of the fits has no reports.

credibility.verification¶

Check dataclass ¶

value property ¶

CheckRecord ¶

FitSettingsRecord ¶

VerificationSummary dataclass ¶

to_dict ¶

from_dict classmethod ¶

Verification dataclass ¶

problems property ¶

register_check ¶

check_of ¶

check_record_from_dict ¶

check_record_to_dict ¶

tightened ¶

relative_differences ¶

build_problems ¶

verify ¶

Check `dataclass` ¶

value `property` ¶

VerificationSummary `dataclass` ¶

from_dict `classmethod` ¶

Verification `dataclass` ¶

problems `property` ¶