store¶

The run store: run records and the layout of a run on the shared drive.

Layout: <store>/runs/<id>/tasks/<task>, the record in <store>/runs/<id>/run.json.

RunState ¶

Bases: StrEnum

State of a run.

TaskState ¶

Bases: StrEnum

State of a task of a run.

RunRecord ¶

Bases: BaseModel

The record of a run.

Attributes:

Name	Type	Description
`id`	`str`	Run id, `YYYYMMDD-HHMMSS-<6 hex>`.
`submitted_by`	`str`	Submitting user.
`owner`	`str`	Owner of the run.
`agent`	`str \| None`	Agent which submitted the run.
`priority`	`Priority`	Priority in the queue.
`group`	`str \| None`	Group of the run.
`workflow`	`Workflow`	The workflow.
`sources`	`list[SourceProvenance]`	Provenance of the snapshots of the sources.
`image`	`str \| None`	Container image.
`reserve`	`str \| None`	Reservation.
`state`	`RunState`	State of the run.
`tasks`	`dict[str, TaskState]`	State of every task.
`plans`	`dict[str, Any]`	The task plans as dicts.
`backend_name`	`str \| None`	Name of the backend workflow.
`created`	`datetime`	Time of creation.
`built`	`datetime \| None`	Time the image of the run was ready and the submission of its workflow began, None while its environment is resolved and built.
`finished`	`datetime \| None`	Time of the end.
`message`	`str \| None`	A message, e.g. the failure.
`oom_kills`	`dict[str, list[int]]`	The attempts of the pods killed for memory per worker (the worker id, the task id of an unsplit task, `<task>-merge` of a merge).
`merge_pending`	`dict[str, datetime]`	The time the service first saw the coordinator of a split task which merges succeeded without the marker of its merge, by task id; the task waits for the marker until `MERGE_MARKER_GRACE_S` of `simrun.service.controller` after it.
`pending_release`	`str \| None`	A reserved node whose release failed, retried by the service.
`pending_cancel`	`bool`	Whether the workflow must still be cancelled, e.g. one which may have been created by a submission that failed.
`pending_logs`	`bool`	Whether the logs of the pods of an ended run must still be kept in the store (`simrun.service.logs`).
`local_commands`	`Literal['direct', 'docker'] \| None`	How the local backend runs the `command` tasks of the run: `direct` its args, `docker` in its image; None the choice of the backend. The Argo backend ignores it.

RunSummary ¶

Bases: BaseModel

The state of a run without its workflow and plans.

of `classmethod` ¶

of(record)

The summary of a record.

RunStore ¶

RunStore(root)

The run store on the shared drive.

Create a store.

Parameters:

Name	Type	Description	Default
`root`	`Path`	Root directory of the store.	required

new_run_id ¶

new_run_id()

Return a new run id, YYYYMMDD-HHMMSS-<6 hex>.

run_dir ¶

run_dir(run_id)

Return the directory of a run.

task_paths ¶

task_paths(run_id, task_id, inputs, scratch)

Return the locations of a task of a run.

write ¶

write(record)

Write a record atomically (temporary file and rename).

read ¶

read(run_id)

Read a record.

Raises:

Type	Description
`KeyError`	When the run does not exist.

iter_records ¶

iter_records()

Iterate over the records of all runs, ordered by id.