snapshot¶

Snapshots of working trees with provenance.

A snapshot is a copy of the files git knows of a working tree (tracked and untracked, not ignored, uncommitted changes included) in the run store. A snapshot is made in two steps: scan lists the files of a working tree with their modes and content hashes, assemble writes them into the store; snapshot does both on the machine of the store, the service does the second for an upload of a client (check_scan, missing, upload). Next to every snapshot <dirname>.files.json maps its paths to the hashes of their contents, so files whose content the latest snapshot of the same name has are hard links to it (copies when their modes differ) and an upload sends only the others. The files of a snapshot are read-only (a-w, part of the hash of its content): a snapshot is named by its content and never changes, and a build backend which writes through a link of a shadow tree into one fails loudly instead. Submodules are out of scope: their content is not copied.

SourceProvenance ¶

Bases: BaseModel

Where the files of a snapshot come from.

Attributes:

Name	Type	Description
`name`	`str`	Name of the source.
`branch`	`str \| None`	Branch of the working tree, `None` for a detached HEAD.
`commit`	`str`	Commit of HEAD.
`dirty`	`bool`	Whether the working tree differs from HEAD.
`changed_files`	`list[str]`	Porcelain status lines, e.g. `M src/a.py`.
`diff_sha256`	`str`	Hash of the snapshot tree (paths, types, modes, contents); the patch stays as provenance.
`snapshot`	`str`	Directory of the snapshot, relative to the store root.
`patch`	`str \| None`	Stored `git diff HEAD --binary`, relative to the store root.
`path`	`str \| None`	The working tree of a source recorded without a snapshot (the local mode), None for a snapshot.

FileEntry ¶

Bases: BaseModel

A file of a working tree as a snapshot holds it.

Attributes:

Name	Type	Description
`path`	`str`	Path below the tree, with `/`.
`mode`	`int`	Mode bits without the write bits (a snapshot is read-only), special bits kept as release 0.1.0 kept them; the ones of the link for a symbolic link.
`sha256`	`str \| None`	Hash of the content of a regular file, None for a link.
`link`	`str \| None`	Target of a symbolic link, None for a regular file.

Scan ¶

Bases: BaseModel

What a snapshot of a working tree is made of, before it is copied.

Attributes:

Name	Type	Description
`branch`	`str \| None`	Branch of the working tree, `None` for a detached HEAD.
`commit`	`str`	Commit of HEAD.
`dirty`	`bool`	Whether the working tree differs from HEAD.
`changed_files`	`list[str]`	Porcelain status lines, e.g. `M src/a.py`.
`patch`	`bytes`	`git diff HEAD --binary`, base64 in JSON.
`files`	`list[FileEntry]`	The files of the snapshot, in the order of git.
`dirs`	`dict[str, int]`	The mode bits of the directories which hold the files, by path; the snapshot keeps them as release 0.1.0 did, with the owner bits added.

SnapshotError ¶

Bases: ValueError

An upload which cannot become a snapshot.

A path outside the tree, a member of the archive which is not a listed file, or a hash which does not match.

MissingContent ¶

Bases: RuntimeError

A file of an upload whose content neither the archive nor the store has.

The latest snapshot changed since the client asked what is missing; the client asks again.

provenance ¶

provenance(repo, name)

The provenance of a working tree without copying it.

The same git calls as snapshot; diff_sha256 is the hash a snapshot of the tree would have, snapshot is empty and there is no patch.

Parameters:

Name	Type	Description	Default
`repo`	`Path`	A git working tree.	required
`name`	`str`	Name of the source.	required

Returns:

Type	Description
`SourceProvenance`	The provenance, with `snapshot=""`, `patch=None` and the absolute `path`
`SourceProvenance`	of the working tree.

scan ¶

scan(repo)

List the files of a working tree with their modes and content hashes.

A file which is gone between the listing of git and its hash is left out.

Parameters:

Name	Type	Description	Default
`repo`	`Path`	A git working tree (a worktree with a `.git` file works too).	required

Returns:

Type	Description
`Scan`	The scan of the tree.

Raises:

Type	Description
`RuntimeError`	When git fails.

manifest_path ¶

manifest_path(snapshot_dir)

The manifest of a snapshot: its paths and the hashes of their contents.

assemble ¶

assemble(store, name, scan, files, move=False)

Write a scanned tree into store/snapshots/<name>.

The files whose content the latest snapshot of the name has are hard links to it (copies when their modes differ), the others come from files: the file to copy or the content. A snapshot is assembled in a temporary directory, its directories get the modes of the scan, it is named by its content and renamed into place; one which exists already is kept.

Parameters:

Name	Type	Description	Default
`store`	`Path`	Root of the run store.	required
`name`	`str`	Name of the source.	required
`scan`	`Scan`	The files and the git state of the tree.	required
`files`	`Callable[[str], Path \| bytes]`	The content of a path of `scan`, a file or bytes; a file which is gone (`FileNotFoundError`) is left out of the snapshot.	required
`move`	`bool`	The files of `files` are given up: renamed into the snapshot instead of copied (on the file system of the store), their content trusted to have the hash of the scan.	`False`

Returns:

Type	Description
`SourceProvenance`	The provenance of the snapshot.

snapshot ¶

snapshot(repo, store, name)

Copy a working tree into store/snapshots/<name>.

Parameters:

Name	Type	Description	Default
`repo`	`Path`	A git working tree (a worktree with a `.git` file works too).	required
`store`	`Path`	Root of the run store.	required
`name`	`str`	Name of the source.	required

Returns:

Type	Description
`SourceProvenance`	The provenance of the snapshot.

Raises:

Type	Description
`RuntimeError`	When git fails.

check_name ¶

check_name(name)

Refuse a source name which is not a directory name of snapshots/.

Raises:

Type	Description
`SnapshotError`	For a name of other characters than letters, digits, `-`, `_` and `.`, or one which does not start with a letter or digit.

check_files ¶

check_files(files)

Refuse a file list which would write outside the tree of a snapshot.

Raises:

Type	Description
`SnapshotError`	For a path which is absolute, not normalized, has a NUL or a component `..` or `.git`, which is listed twice or below another entry; a regular file without a hash or with write or special mode bits; a link whose target is absolute or leaves the tree.

check_scan ¶

check_scan(scan)

Refuse a scan which cannot become a snapshot, see check_files.

Raises:

Type	Description
`SnapshotError`	For a file list `check_files` refuses, a commit which is not a hexadecimal hash, or a directory which is not one of the tree, is also a file, or has other mode bits than permissions and setgid.

missing ¶

missing(store, name, files)

The paths of regular files whose content the latest snapshot lacks.

Parameters:

Name	Type	Description	Default
`store`	`Path`	Root of the run store.	required
`name`	`str`	Name of the source.	required
`files`	`list[FileEntry]`	The files of a scan, checked with `check_files`.	required

upload ¶

upload(store, name, scan, archive)

Write a snapshot from a scan and a tar of the files the store lacks.

Every member of the tar is a regular file of the scan whose content has the hash of the scan; it is extracted, under a name of its own, into a temporary directory beside the snapshots of the name, and the snapshot assembled from it (its files renamed into place) and the latest snapshot.

Parameters:

Name	Type	Description	Default
`store`	`Path`	Root of the run store.	required
`name`	`str`	Name of the source, checked with `check_name`.	required
`scan`	`Scan`	The files and the git state of the tree, checked with `check_scan`.	required
`archive`	`IO[bytes]`	The tar, read once from its start.	required

Returns:

Type	Description
`SourceProvenance`	The provenance of the snapshot.

Raises:

Type	Description
`SnapshotError`	For a member which is not a regular file, not a file of the scan or sent twice, or whose content does not match its hash; or a tar which cannot be read.
`MissingContent`	For a file whose content neither the tar nor the latest snapshot has.