data¶

Module handling data (experiment and simulation).

Data ¶

Data(
    index,
    task=None,
    dataset=None,
    function=None,
    variables=None,
    parameters=None,
    sid=None,
    sel=None,
)

Data of a simulation experiment.

A column of a dataset, a selection of the results of a task or a function of other data. It is a promise which is fulfilled when the experiment runs.

Construct data.

Parameters:

Name	Type	Description	Default
`index`	`str`	what the data is called, i.e., a selection of the results of a task (`"S"` is the amount, `"[S]"` the concentration and `"time"` the time), a column of a dataset or the name of a function.	required
`task`	`str \| None`	id of the task whose results are selected.	`None`
`dataset`	`str \| None`	id of the dataset whose column is selected.	`None`
`function`	`str \| None`	formula of the data, a function of `variables` and `parameters`.	`None`
`variables`	`dict[str, Data] \| None`	the data the function reads, by the identifier in the formula.	`None`
`parameters`	`dict[str, float] \| None`	the numbers the function reads, by the identifier in the formula.	`None`
`sid`	`str \| None`	id of the data, if not given `<task or dataset>__<index>` for an amount and `<task or dataset>__conc__<index>` for a concentration (`"[S]"`), with `__` for a dot of the index (`pk.cmax` is `<task>__pk__cmax`) and `_x<hex>_` for every other character which is no letter, digit or underscore (the rate of change `X'` is `<task>__X_x27_`), so it is a valid SId.	`None`
`sel`	`Mapping[str, Any] \| None`	labels of dimensions to select, `{dim: label}` keeps one point and drops the dimension, `{dim: [labels]}` keeps the dimension; for a dataset the values of columns whose rows are kept, see `get_data`. The sid does not depend on it, give `sid` to tell apart two data of one index with different selections.	`None`

Raises:

Type	Description
`ValueError`	if none of `task`, `dataset` and `function` is given.

sid `property` ¶

sid

Get id.

name `property` ¶

name

Get name.

dtype `property` ¶

dtype

Get data type.

Types ¶

Bases: Enum

Data types.

is_task ¶

is_task()

Check if task.

is_dataset ¶

is_dataset()

Check if dataset.

is_function ¶

is_function()

Check if function.

to_dict ¶

to_dict()

Convert to dictionary.

get_data ¶

get_data(experiment, to_units=None)

Get the values of the data from an experiment which ran.

The values are a labelled array named by the sid of the data, with the unit in attrs["units"] (None for labels), see to_quantity:

a task: the variable or coordinate of its ScanResult with its coordinates, a timecourse over (*dims, time) or (*dims, _point) for a ragged result padded with NaN, a value per simulation over (*dims); time is the time, the plain name of a symbol its timecourse, also when the scan changes it, <dimension>.<target> the values a dimension sets and a coordinate of a dimension are over the dimension, and a dimension id gives its labels; every array of a task names the values of a dimension <dimension>.<target> among its coordinates;
a dataset: the column over the dimension row, whose coordinate is the index of the dataset;
a function: its formula on its variables and parameters, see evaluate_function.

sel selects labels of the dimensions of a task or a function (a dimension of the scan which the data has not is skipped) and the rows of a dataset by the values of its columns. unit is set to the unit of the values.

Parameters:

Name	Type	Description	Default
`experiment`	`SimulationExperiment`	the experiment whose datasets and results are read.	required
`to_units`	`str \| None`	the unit to convert the values to, their own unit without.	`None`

Returns:

Type	Description
`DataArray`	The values.

Raises:

Type	Description
`KeyError`	if the dataset has no column of the index or no unit of it, or the result of the task has no variable or coordinate of the selection, e.g. the timecourse of a changed target which no data of the experiment read.
`ValueError`	if the dataset is no `DataSet`, the result of the task is no `ScanResult` or its selection has no unit, a function has no formula, or `sel` names a dimension, label or column which does not exist.
`DimensionalityError`	if the values cannot be converted to `to_units`.

DataSeries ¶

Bases: Series

DataSet - a pd.Series with additional unit information.

DataSet ¶

Bases: DataFrame

DataSet, a pd.DataFrame with additional unit information.

get_quantity ¶

get_quantity(key)

Return quantity for given key.

Requires using the numpy data instead of the series.

from_df `classmethod` ¶

from_df(df, ureg, udict=None)

Create DataSet from given pandas.DataFrame.

The DataFrame can have various formats which should be handled. Standard formats are 1. units annotations based on '_unit' columns, which apply to the column and its '_sd', '_se', '_min' and '_max' columns 2. units annotations based on 'unit' column which is applied on 'mean', 'value' and 'median', their '_sd', '_se', '_min' and '*_max' columns, and 'sd', 'se', 'min' and 'max' ('cv' is relative)

The unit of a column is the unit of its first row with a value; the unit of a row without a number (no value, sd, se, min or max) is ignored, and a column without any unit has no unit, which is logged as an error if the column or its sd, se, min or max has a number and as a debug message otherwise, e.g. for the empty time of a table which does not report it (#310). Rows with another unit of the same dimension are converted to it, all their numbers, and their unit column is rewritten; the rows of a 'unit' column are converted in one pass, so mean, value and median get one unit and the numbers they share are converted once. A unit with a numeric factor, as PK-DB writes it (ml/min/(1.73*m^2), micromol/(0.5*hr)), is read as a number of the values: the numbers of its rows are multiplied by the factor, and a column whose first unit has a factor gets the unit without it. The data frame of the caller is not changed.

:param df: pandas.DataFrame :param ureg: the unit registry :param udict: optional units of columns

:return: dataset

:raises ValueError: naming the column and its units, if a unit cannot be read (no string, undefined, a factor which is no positive number or a factor of a unit with an offset), cannot be converted by a factor (another dimension or an offset such as degC), or if a column which is converted has values which are no numbers

unit_conversion ¶

unit_conversion(key, factor)

Convert the units of the given key in the dataset via key * factor.

Changes values in place in the DataSet.

The quantity in the dataset is multiplied with the conversion factor. In addition to the key, also the respective error measures are converted with the same factor, i.e. - {key} - {key}_sd - {key}_se - {key}_min - {key}_max

FIXME: in addition base keys should be updated in the table, i.e. if key in [mean, median, min, max, sd, se, cv] then the other keys should be updated; use default set of keys for automatic conversion

:param key: column key in dataset (this column is unit converted) :param factor: multiplicative Quantity factor for conversion :return: None

to_quantity ¶

to_quantity(array, ureg)

Get the values of data as a quantity.

Parameters:

Name	Type	Description	Default
`array`	`DataArray`	the values of a `Data`, see `Data.get_data`.	required
`ureg`	`UnitRegistry`	the registry of the quantity, e.g. the one of the experiment.	required

Returns:

Type	Description
`Quantity`	The values with the unit of `attrs["units"]`.

Raises:

Type	Description
`ValueError`	if the array holds labels or has no unit.

evaluate_function ¶

evaluate_function(formula, variables, ureg)

Evaluate the formula of a Data of type FUNCTION on its data.

The formula is the math of PEtab, see sbmlsim.simulator.formula. The arrays are broadcast by the names of their dimensions, so y / dose over (dose, time) and (dose,) needs no reshaping; two arrays must have the same coordinates of a dimension they share. max and min of a single argument reduce it along its time (time, or _point of a ragged result) or, without one, along the rows of a dataset, ignoring NaN, the padding of a scan; the other dimensions stay, so Y/max(Y) normalizes every simulation of a scan to its own maximum. With two or more arguments they are the elementwise maximum and minimum of PEtab. mean and at need the time points of a simulation, they are reductions of the observables of a scan. An array without a unit is evaluated as plain numbers, a formula without units is dimensionless.

Parameters:

Name	Type	Description	Default
`formula`	`str`	the formula.	required
`variables`	`Mapping[str, DataArray \| float]`	the arrays of the data and the numbers of the parameters, by the identifiers of the formula.	required
`ureg`	`UnitRegistry`	the registry of the units.	required

Returns:

Type	Description
`DataArray`	The value of the formula over the broadcast dimensions, with its unit.

Raises:

Type	Description
`ValueError`	if the formula is not valid math, reads an identifier which is not a variable, uses `mean` or `at`, or combines arrays with different coordinates of one dimension.

check_sel ¶

check_sel(data, sel, labels)

Check that a selection names dimensions and labels of its source.

Parameters:

Name	Type	Description	Default
`data`	`Data`	the data of the selection, named in the error.	required
`sel`	`Mapping[str, Any]`	the selection, see `Data`.	required
`labels`	`Mapping[str, list[Any] \| None]`	the labels of every dimension of the source, `None` for a dimension whose labels are not checked, e.g. the time of a task before it ran.	required

Raises:

Type	Description
`ValueError`	if a dimension is not one of the source, or a label is not one of its dimension; the message names the ones which exist.

load_pkdb_dataframe ¶

load_pkdb_dataframe(
    sid, data_path, sep="\t", comment="#", **kwargs
)

Load TSV data from PKDB figure or table id.

This is a simple helper functions to directly loading the TSV data. It is recommended to use pkdb_analysis methods instead.

This function will be removed.

E.g. for 'Amchin1999_Tab1' the file data_path / 'Amchin1999' / '.Amchin1999.tsv' is loaded.

:param sid: figure or table id :param data_path: base path of data or iterable of data_paths :param sep: separator :param comment: comment characters :param kwargs: additional kwargs for csv parsing :return: pandas DataFrame :raises FileNotFoundError: if the dataset is in none of the data paths

load_pkdb_dataframes_by_substance ¶

load_pkdb_dataframes_by_substance(sid, data_path, **kwargs)

Load dataframes from given PKDB figure/table id split on substance.

The DataFrame is split on the 'substance' key.

This is a simple helper functions to directly loading the TSV data. It is recommended to use pkdb_analysis methods instead.

This function will be removed.

:param sid: :param data_path: :param kwargs: :return: dict[substance, pd.DataFrame]

data¶

Data ¶

sid property ¶

name property ¶

dtype property ¶

Types ¶

is_task ¶

is_dataset ¶

is_function ¶

to_dict ¶

get_data ¶

DataSeries ¶

DataSet ¶

get_quantity ¶

from_df classmethod ¶

unit_conversion ¶

to_quantity ¶

evaluate_function ¶

check_sel ¶

load_pkdb_dataframe ¶

load_pkdb_dataframes_by_substance ¶

sid `property` ¶

name `property` ¶

dtype `property` ¶

from_df `classmethod` ¶