credibility.extensions¶

The evidence extensions: evidence of the contexts of use which a project adds.

The core of the credibility assesses what every model has: the data points of its adopted fits against the acceptance rules, the coverage of the claimed domain, the verification, the parameter uncertainty and the variability. A domain adds evidence of its own, e.g. the ratios between the arms of a study, the equivalency of the simulated and the reported inputs of the subjects, the chronology of the studies or the provenance of the study data. Such evidence is an EvidenceExtension, which a project registers (sbmlsim.project.CredibilitySettings.extensions) and the core calls at fixed points. Every method has a default which contributes nothing, so an extension overrides what it adds. The contract:

The declaration. A setting of an extension for one context of use, e.g. the acceptance of the ratio pairs, is an entry of ContextOfUse.extensions by the name of the extension; a setting with a method problems (ExtensionSetting) is checked with the declaration. A ratio quantity carries a Pairing, which the extension that knows it claims (claims). The project checks both when it loads the declaration (sbmlsim.project.Project.load_credibility, sbmlsim.credibility.Credibility.extension_problems): a setting by a name of no extension of the project and a pairing which no extension claims are problems, so a typo never falls back to a default silently.

The selection. selection_problems checks the decisive points of every context of use after the selection of the evidence (sbmlsim.credibility.selection.select), e.g. a study of a ratio with two reference arms.

The assessment. prepare reads what the extension needs from the project once per assessment, e.g. the chronology of the studies; assess_context gives its ContextContribution to a context of use:

• acceptance: the results of the rules of the extension. They enter the verdict of the context and the factor output comparison together with the acceptance rules of the context, and they are rows of the acceptance of the record of the context: after the rows of the rules of the context, in the order of the extensions of the project, each with the name of its extension in the field extension (ACCEPTANCE_EXTENSION; a row of a rule of the context has none). acceptance evaluates the same rules on other predictions, those of a draw of the parameter uncertainty or of an alternative of the verification, in the same order.
• quantities: the evaluable points of the quantities the extension derives, e.g. the pairs of a ratio quantity; a quantity with a point is covered.
• factors: a FactorStatement per factor of V&V 40 the extension assesses, combined with the statement of the core by sbmlsim.credibility.factors.combine_factor: the statements before the core (before_core) in the order of the extensions, the statement of the core, the statements after it in the order of the extensions; a factor with a statement of an extension is assessed and has no step. The statement of the core is the one the core gives the factor (an assessed factor, a platform statement or the checks of the verification), none for a factor the core leaves not assessed.
• section: the section of the extension in the record of the context.

assessment_section is its section of the record of the assessment, e.g. the provenance of the study data of every study.

The pooling. pool pools the sections of the contexts of several models into a table whose columns the extension decides; pool_lines words the rows for a command that prints them (by default every row as column=value).

The records. The record of a context of use and of an assessment has the sections of the extensions by their name in its field extensions (EXTENSIONS), which the extension of that name writes and reads; the core keeps a section as it is. A record of pkpdsim 0.19 has the evidence of its extensions as fields of its own, which read_sections reads as their sections (LEGACY_CONTEXT_SECTIONS, LEGACY_ASSESSMENT_SECTIONS) and read_acceptance its rule of the ratio pairs, ratio_acceptance, as a row of the acceptance of the extension ratios (LEGACY_ACCEPTANCE); a record of pkpdsim 0.17 or 0.18 has none.

The report. The rows of the acceptance of an extension are rows of the acceptance table of its context, after the rows of the context, with the name of the rule followed by the label of the extension. context_report_blocks gives the blocks of the extension in the card of a context, each placed after a block of the core (ContextBlock.after, one of CONTEXT_BLOCKS), the blocks after one block of the core in the order of the extensions; report_section a section of the report of its own with an entry in the navigation, after the core sections and before the provenance; report_tables the tables it writes next to the report.

The scorecard. An extension names its columns statically (scorecard_names) and scorecard_columns gives their values for a context, for every name of scorecard_names and no other; they follow the columns of the core (the order of the columns of pkpdsim changes, the scorecards are compared by the names of the columns). The rows of the acceptance of an extension are columns of their rule like the rules of the context, which the core compares as rules; the columns of scorecard_names it never compares (rule_columns): regressions of the extension alone judges them, e.g. a column of pairs which drops from 10 to 4 is no regression of a rule. The core gives regressions the rows as they are: a scorecard of the process has the values scorecard_columns gave, a scorecard read back from its TSV has texts (an empty text for a missing value), and a baseline of an older record lacks the columns; scorecard_number and scorecard_true read a value of either. pool pools the sections of the contexts of several models with the options of the pooling, which the caller passes through as it got them (e.g. {"field": "health"} of the qualification of the ratio pairs by a value of the metadata, an option of a command line), or empty for the defaults of the extension.

observable

problems()

FactorStatement(text, before_core=False)

ContextContribution(
    *,
    section=None,
    acceptance=(),
    quantities=dict(),
    factors=dict(),
)

ContextBlock(html, after='acceptance')

ReportSection(id, title, html)

ScorecardChange(what, baseline, current)

claims(pairing)

selection_problems(cou, decisive, lookup)

prepare(project, credibility, fits)

assess_context(cou, decisive, fits, prepared)

acceptance(cou, decisive, lookup)

assessment_section(prepared)

context_report_blocks(context, section)

report_section(record)

report_tables(record)

scorecard_columns(context)

regressions(baseline, current)

pool(contexts, options)

pool_lines(table, options)

read_sections(data, legacy)

read_acceptance(data)

scorecard_columns(extension, section)

rule_columns(columns, core, extensions)

scorecard_number(value)

scorecard_true(value)