Skip to content

Development

For developers and operators only

This section is for working on PK-DB itself or running a separate server. General users do not need a source checkout, Docker, a database, or administrator access. To use PK-DB, start with Browse and access data or the Python client and API.

Choose a development workflow

Install the Python package from source

For normal use, install pkdb from PyPI. Use a source checkout when changing the library or testing unreleased behavior:

git clone --branch develop https://github.com/matthiaskoenig/pkdb.git
cd pkdb
uv sync --project python --locked --python 3.14
uv run --project python pkdb --help
uv run --project python pytest python/tests -q -n auto

uv sync installs the local package in editable mode and includes its development dependencies. -n auto (pytest-xdist) runs the tests in parallel on every CPU core, as CI does. For an existing virtual environment, use python -m pip install -e /absolute/path/to/pkdb/python. To use this checkout as a standalone CLI, run uv tool install --editable ./python; to add it to another uv project, run uv add --editable /absolute/path/to/pkdb/python from that project. The five speed tests of the client measure CPU time and take about 100 seconds, so they are marked slow and left out by default (addopts in python/pyproject.toml). Run them alone and without -n, so that no other worker shares the cores: uv run --project python pytest python/tests -q -m slow. -m "" runs every test; a speed test named by its node id also needs -m slow.

pkdb curate serves the local curation app from python/src/pkdb/curation/static/. Build it with npm ci and npm run build:curation in frontend/ (Node 24.21.0 and npm 12.1.0); npm run dev:curation serves it with live reload and forwards API requests to the pkdb curate origin in PKDB_CURATION_URL. Building a wheel or source distribution of python/ fails without the built app.

The backend is a separate source package. From the repository root:

uv sync --project backend --locked --python 3.14
uv run --project backend pre-commit install

The backend environment uses the sibling python/ package in editable mode. Python 3.15 is also supported; pass --python 3.15 consistently when syncing and running that environment. See runtime compatibility for its current prerelease dependencies.

Work with the development server

Follow Local setup to start the stack. Its accounts, studies, and API keys are separate from those on the public website. Point scripts at your local endpoint:

export PKDB_ENDPOINT=http://localhost:18083
uv run --project python pkdb vocabulary sync --endpoint "$PKDB_ENDPOINT" --output /tmp/pkdb-vocabulary.json

For authenticated operations, use a key from your local account in PKDB_API_KEY. Frontend edits reload automatically in the Docker development profile; rebuild the backend image after server changes. Native backend reload instructions are in Local setup.

Branches, checks, and review

Start a topic branch from an up-to-date develop, make the change, and run the relevant tests. Pre-commit runs formatting, lint, and type checks on applicable files. Push the topic branch and open a pull request against develop; merge after tests, ruff, ty, and docs pass. Source code, migrations, and documentation belong in the repository; credentials, local databases, attachments, and generated caches do not.

Keep Markdown prose paragraphs on one source line, using editor soft wrapping. See Local setup for frontend/backend checks, migrations, and documentation builds.

Continuous integration

Pull requests and branch pushes test the backend and public Python client on Linux with Python 3.14. Release tag pushes (v*) and manual runs also test the client and its speed tests on macOS and Windows, with Python 3.14; start a manual run of the CI-CD workflow on a branch to test it on every platform before tagging a release. Frontend, container, documentation, lint, and type checks still run for ordinary changes.

The jobs of the CI-CD workflow run in parallel: python (migrations, generated files, backend tests), server (the server wheel, the runtime tests of the backend image and the restore tests), compose (the documented local stack), client (client tests and distributions), client-slow (the speed tests, -m slow, in one process), curation-e2e (the curation app in Chromium on every core of the runner, without the speed tests), curation-speed (the speed tests of the curation app, with one worker so that nothing disturbs the timings), frontend-unit (source check, types, lint, unit tests, build) and frontend-e2e (the frontend browser tests, one job each for Chromium, Firefox and WebKit). server, compose and frontend-e2e each build the backend image with buildx from a layer cache in the GitHub Actions cache, which only server writes, so that no job waits for another. A pull request that changes documentation only (files in docs/ other than docs/study-format.md, in release-notes/, Markdown files at the root, and zensical.toml) skips these jobs, see scripts/ci_changes.py. Every push, so every merge into develop or main, release tag and manual run tests everything, because a docs-only push could otherwise skip the code of an earlier push whose run was cancelled or failed. The required check tests passes when every job passed, or when all of them were skipped for a docs-only pull request. Lint (ruff), types (ty, which checks the client for Linux, Windows and macOS) and the documentation (docs) are workflows of their own.

Continuous integration is kept small: every workflow cancels its running build when a newer commit of the same branch or pull request arrives; uv caches the packages and the interpreters, npm its packages, Playwright each browser by its version, buildx the layers of the backend image, and the client job the LibreOffice packages (a release tag builds its distributions without a cache); a pull request builds the documentation without preparing its deployment, dependabot proposes its updates once a month, and only Python 3.14 is tested. Python 3.15 is tested locally before a pull request is opened: the client with uv run --project python --python 3.15 pytest python/tests -q, the backend with the backend tests and checks run with --python 3.15.

Python package releases

The public pkdb package and pkdb-server use the same version and release tag. Bump versions with the repository's bump-my-version configuration, commit the generated lock updates, and release through a vVERSION tag. Before tagging, start a manual run of the CI-CD workflow on develop and wait until it passes: pull requests test the client only on Linux, and the manual run tests it on Linux, macOS and Windows with Python 3.14; Python 3.15 is tested locally, see continuous integration. The server wheel pins pkdb==VERSION; it is distributed as a GitHub release asset alongside the public wheel and source archive.

After all backend, frontend, container, and Python client checks pass, .github/workflows/ci-cd.yml publishes only the public pkdb wheel and source archive to PyPI. The workflow verifies that both filenames match the release tag. The GitHub release and main synchronization wait for successful PyPI publication. Publishing uses short-lived GitHub OIDC credentials, with no stored PyPI API token.

Maintainers manage the following publishing configuration:

  1. Ensure the maintainer has the appropriate role on the pkdb PyPI project and can configure its trusted publisher.
  2. Create the GitHub repository environment named pypi and restrict deployment to release tags according to the repository's release policy.
  3. Configure a PyPI Trusted Publisher for the project. For a new project, use PyPI's pending publisher registration. Enter PyPI project name pkdb, GitHub owner matthiaskoenig, repository pkdb, workflow filename ci-cd.yml, and environment pypi.

The workflow uses the official PyPA publishing action in a dedicated job with id-token: write. The setup above is external account configuration; adding the workflow does not create the PyPI project or grant publishing access.