Skip to content

planner

The planner: the shape of a worker pod and the number of pieces of a task.

PlanError

Bases: ValueError

A task which cannot be planned on the nodes.

Node dataclass 

Node(
    name,
    cores,
    memory_gb,
    gpus=0,
    reserved_by=None,
    allocatable_cores=None,
    allocatable_memory_gb=None,
)

A node of the cluster.

Attributes:

Name Type Description
name str

Name of the node.
cores int

The whole cores a worker pod of a run can have on the node, what the planner plans with.
memory_gb float

The GiB of memory a worker pod of a run can have on the node, what the planner plans with.
gpus int

Number of GPUs.
reserved_by str | None

Name of the reservation holding the node, None if free.
allocatable_cores float

The cores the scheduler gives the pods of the runs (sidecars included), as it counts their requests.
allocatable_memory_gb float

The GiB of memory the scheduler gives the pods of the runs, as it counts their requests.

Create the node; the allocatable cores and memory default to the others.

PodShape dataclass 

PodShape(
    cores,
    memory_gb,
    processes,
    per_worker_gb,
    retry_memory_gb,
    retry_per_worker_gb,
    scratch_gb,
    gpus,
)

The resources of a worker pod.

Attributes:

Name Type Description
cores int

Number of cores.
memory_gb int

Memory in GiB, rounded up.
processes int

Number of worker processes.
per_worker_gb float

Memory per worker process in GiB.
retry_memory_gb int

Memory of a retry in GiB.
retry_per_worker_gb float

Memory per worker process of a retry in GiB.
scratch_gb float

Local scratch in GiB.
gpus int

Number of GPUs.

TaskPlan dataclass 

TaskPlan(
    task_id,
    pods,
    shape,
    base_priority,
    extra_priority,
    units,
    coordinator_merges=False,
)

The plan of a task.

Attributes:

Name Type Description
task_id str

Id of the task.
pods int

Number of worker pods.
shape PodShape

Shape of a pod.
base_priority str

Priority class of the first pods.
extra_priority str

Priority class of the further pods.
units int | None

Number of units, None for an unsplit task.
coordinator_merges bool

Whether the coordinator of a split task merges once its ledger is complete, so the task has no merge pod. False for plans recorded before the merge moved into the coordinator.

shape 

shape(resources, nodes, max_processes=None)

Sizes a pod for the most common node which is not reserved.

When one worker does not fit that node, the largest usable node is taken.

Parameters:

Name Type Description Default
resources ResolvedResources

The resources of the task.

 required
nodes list[Node]

The nodes of the cluster.

 required
max_processes int | None

At most this many processes, as many as fit by default.

 None

Returns:

Type Description
PodShape

The shape of a pod.

Raises:

Type Description
PlanError

One worker fits on no node.

plan_task 

plan_task(
    task,
    task_type,
    resources,
    nodes,
    mode,
    priority,
    units,
    pod_start_s=POD_START_S,
)

Plans the pods of a task.

A pod of a split task has no more processes than the task has units. A split task of the mode fastest gets at most MAX_PIECES pods, as many as fit on the nodes, and no pod without a unit for each of its processes. With the learned seconds of a unit it gets a further pod only while every pod still has at least pod_start_s of work: a pod which takes longer to start than its share of the work would find the units taken by the others and only hold cores.

Parameters:

Name Type Description Default
task TaskSpec

The task.

 required
task_type TaskType

Its type.

 required
resources ResolvedResources

The resolved resources.

 required
nodes list[Node]

The nodes of the cluster.

 required
mode SplitMode

The split mode.

 required
priority Priority

The priority of the run.

 required
units int | None

The number of units, None for an unsplit task.

 required
pod_start_s float

About the seconds a pod takes to start on the cluster.

 POD_START_S

Returns:

Type Description
TaskPlan

The plan; extra pods are not limited by idle capacity, Kueue queues them.

Raises:

Type Description
PlanError

The task fits on no node.