Skip to content

resources

Resources learned from the reports of workers.

An observation is the memory per worker process and the seconds of a unit in one process of one report of a worker, keyed by the task type, the model and a variant. A request is resolved from the last WINDOW observations of its key: the 95th percentile of the measured memory per worker times MARGIN, and at least RETRY_FACTOR times the memory per worker of a pod killed for memory, until a later run of the key succeeded (its measurement then says what the key needs). The memory measured is anonymous memory (see simrun.worker.MemorySampler), and a worker counts as out of memory only when the backend reports its pod killed for memory.

ResourceKey dataclass 

ResourceKey(task_type, model, variant='')

What observations of a task are grouped by.

Attributes:

Name Type Description
task_type str

Name of the task type.
model str

Name of the first source, empty without.
variant str

A setting the plugin names as cost relevant, empty by default.

text 

text()

Returns the key as stored.

ResolvedResources dataclass 

ResolvedResources(
    memory_base_gb,
    memory_per_worker_gb,
    scratch_gb,
    gpus,
    seconds_per_unit,
    samples,
    source,
)

The resources requested for a task.

Attributes:

Name Type Description
memory_base_gb float

Base memory in GiB.
memory_per_worker_gb float

Memory per worker in GiB.
scratch_gb float

Local scratch in GiB.
gpus int

Number of GPUs.
seconds_per_unit float | None

Median seconds of a unit in one process, None without records.
samples int

Number of memory observations the memory was learned from.
source str

learned from N runs, plugin hint, default or override.

ResourceStore 

ResourceStore(db)

Observations of the workers of past tasks in a SQLite file.

The file must be on a local volume. A connection is opened per call, so the store is safe to use from several threads of one process.

Creates the database if needed.

Parameters:

Name Type Description Default
db Path

Path of the SQLite file.

 required

record 

record(key, report, base_gb, oom=False, run='')

Records the report of a worker.

Parameters:

Name Type Description Default
key ResourceKey

The key of the task.

 required
report WorkerReport

The report.

 required
base_gb float

The base memory of the task in GiB.

 required
oom bool

Whether the worker was killed for memory (see simrun.service.controller.is_oom); the limit is used then.

 False
run str

The id of the run of the report.

 ''

resolve 

resolve(key, hint, override)

Returns the resources to request.

Parameters:

Name Type Description Default
key ResourceKey

The key of the task.

 required
hint ResourceHint

The hint of the plugin.

 required
override ResourceOverride | None

The override of the workflow.

 required

Returns:

Type Description
ResolvedResources

The resolved resources; an override wins over learned values.

resource_key 

resource_key(task_type, settings, model)

The key of the observations of a task, the one place it is derived.

Parameters:

Name Type Description Default
task_type TaskType

The type of the task.

 required
settings Mapping[str, Any]

The settings of the task as submitted.

 required
model str

The name of the model, the first source of the run, empty without.

 required

Returns:

Type Description
ResourceKey

The key; its variant is the value of the cost_setting of the type (the
ResourceKey

default of the settings when the task does not set it), text as is and other
ResourceKey

values as JSON, empty for a type without one.