Skip to content

progress.resources

The cores and the memory a run uses, for its progress.

The progress of a long run says what of the machine it uses: the cores it may use (the CPUs of the process bounded by the quota of its cgroup, see sbmlsim.cgroup), the processes it runs on, the CPU it uses, the memory of its processes and the memory of the machine. A run which is held up by the cores and a run which is held up by the memory then look different on the console, in the log and in the events.

The sample is taken from /proc on linux: the stat of the process and of every process below it gives the seconds of the CPU and the resident memory of the tree, meminfo the memory of the machine. The pool of a run is a tree of children, so the sample covers the workers as well, which resource.getrusage does not: it counts a child when it ends. Without /proc (a platform which has none) the CPU is the one of getrusage and the memory is left out, and on a platform without resource (windows) the CPU is left out as well, which the reporters leave out of their text.

The sample costs a scan of /proc, so it is taken at most every second and every reporter of a process takes the same one, see SAMPLER.

Resources dataclass 

Resources(
    cores,
    workers=None,
    cpu_percent=None,
    rss=None,
    memory_used=None,
    memory_total=None,
)

What of the machine a run uses.

Attributes:

Name Type Description
cores int

the cores the run may use.
workers int | None

the processes it runs on, None if it does not say.
cpu_percent float | None

the CPU it used since the sample before, in percent of a core, None for the first sample; 1180 are twelve busy cores.
rss int | None

the resident bytes of its processes, None without /proc.
memory_used int | None

the bytes of the memory of the machine which are used, None without /proc.
memory_total int | None

the bytes of the memory of the machine, None without /proc.

as_dict 

as_dict()

Get the sample as JSON, the field resources of a progress event.

Sampler 

Sampler(now=_now)

The samples of the resources of a process, one per interval.

Attributes:

Name Type Description
sample_ Resources | None

the sample of the interval which runs, None before the first one.

Create the sampler without a sample.

Parameters:

Name Type Description Default
now object

the monotonic clock, for the tests.

 _now

sample 

sample(workers=None, interval=INTERVAL)

Get what of the machine the run uses, at most one sample per interval.

Parameters:

Name Type Description Default
workers int | None

the processes the run runs on, for the text of the sample.

 None
interval float

the seconds a sample is kept before the next one is taken.

 INTERVAL

Returns:

Type Description
Resources

The sample, the one of the interval which runs if it is young
Resources

enough.

format_resources 

format_resources(sample)

Write what of the machine a run uses, what a platform gives of it.

Parameters:

Name Type Description Default
sample Resources

the sample.

 required

Returns:

Type Description
str

The text, e.g. cores 4/12 cpu 380% mem 7.4 GB, machine 18.3/31.0 GB.