Skip to content

service.index

The index of the runs in SQLite (spec section 4), beside the run store.

The store stays the source of truth: the index is rebuilt from the records at the start of the service and updated after every record the service writes (IndexedRunStore.write), the record first. It holds what the controller and the lists need without reading the records over NFS: the id, the owner, the agent, the group, the name of the workflow, the state, the times of creation and end, the pending release and cancellation, the snapshots and the summary of every run. The controller follows only the live runs (not ended, or with a release or a cancellation pending); GET /runs, a cleanup and the usage filter in the index.

IndexedRun dataclass 

IndexedRun(summary, owner, workflow, pending, snapshots)

A run as the index has it.

Attributes:

Name Type Description
summary RunSummary

Its summary.
owner str

Its owner.
workflow str

The name of its workflow.
pending bool

Whether a release of its node or a cancellation of its workflow is pending.
snapshots tuple[str, ...]

The snapshots of its sources, relative to the store.

RunIndex 

RunIndex(db)

The index of the runs in a SQLite file on a local volume.

A connection is opened per call, so the index is safe to use from several threads of one process.

Create the table if needed.

Parameters:

Name Type Description Default
db Path

Path of the SQLite file.

 required

rebuild 

rebuild(records)

Replace the index by the runs of records.

put 

put(record)

Add or update a run.

remove 

remove(run_id)

Remove a run.

any_unended 

any_unended()

Whether a run has not ended, without reading its row.

live_ids 

live_ids()

The ids of the runs the controller follows, sorted.

Runs which have not ended, and ended runs with a release of their node, a cancellation of their workflow or the logs of their pods pending.

runs 

runs(group=None, owner=None, agent=None, state=None)

The runs with the given group, owner, agent and state, sorted by id.

IndexedRunStore 

IndexedRunStore(root, index)

Bases: RunStore

The run store of the service: every record it writes is indexed too.

A record whose index row could not be written (the index raised after the record was written) and a record which could not be read when the index was rebuilt are kept aside and indexed again at every reindex, which the controller calls at every poll; the ones which still cannot be read are logged then.

Create the store.

Parameters:

Name Type Description Default
root Path

Root directory of the store.

 required
index RunIndex

The index of its runs.

 required

write 

write(record)

Write a record, then index it.

A failure of the index is logged and the run indexed at the next reindex: the record is written, so the caller goes on.

forget 

forget(run_id)

Remove a run from the index, once it was moved out of the runs.

ids 

ids()

The ids of the runs in the store, sorted; hidden directories are no runs.

rebuild 

rebuild()

Rebuild the index from the records.

A record which cannot be read is logged, left out and tried again at every reindex.

reindex 

reindex()

Index the runs whose index row is missing, from their records.

A record which cannot be read is logged at every call, so a run the controller cannot follow is never silent. Only the index row is written, never the record file, so the caller holds the lock of the record writes around the call: a record is read and indexed without a write in between.

Returns:

Type Description
list[str]

The runs whose records were read but whose index rows still cannot be
list[str]

written, which the controller follows besides the live runs.