Development¶

Contributions are welcome. The repository is matthiaskoenig/pymetadata; development happens against the develop branch via pull requests.

Setup development environment¶

Development needs uv and a checkout of the repository:

git clone https://github.com/matthiaskoenig/pymetadata.git
cd pymetadata

A single sync creates the virtual environment in .venv, installs pymetadata into it in editable mode and adds the complete tooling:

uv sync --extra dev

The dev extra contains everything used below, i.e., pytest, ruff, ty, tox, pre-commit, zensical and bump-my-version, and it pulls in the optional ontology extra, so nothing has to be installed separately. The python version is taken from .python-version (currently 3.14); to work against the oldest supported version instead use uv sync --extra dev --python 3.11, which replaces the environment.

The tools are then run either with uv run <command>, which uses the environment without activating it, or from the activated environment:

source .venv/bin/activate        # Linux and macOS
.venv\Scripts\activate           # Windows

The commands in this document are written without the uv run prefix; prepend it if the environment is not activated.

The last step installs the git hook:

uv run pre-commit install          # install the hook, once per checkout
uv run pre-commit run --all-files  # check the current state of the repository

From now on every commit is checked with ruff (lint and format) and ty, i.e., the same checks that run in continuous integration. On a commit only the changed files are looked at, --all-files checks the whole repository and is what a newly added hook should be tried with.

Testing¶

The tests are written with pytest, tox runs them against every supported python version.

The tox environments are named after the interpreter (py3.11 to py3.14, see envlist in tox.ini), a single one is run with

tox r -e py3.14

tox run-parallel

This needs the interpreters to be available, which uv installs with uv python install 3.11 3.12 3.13 3.14. Continuous integration runs the same environments as uvx --with tox-uv tox -e py3.14.

To run the tests directly against the development environment use

pytest                                            # the full suite
pytest tests/test_omex.py                         # a single module
pytest tests/test_omex.py::test_entry_from_dict   # a single test

Some tests query web services (identifiers.org, OLS, ChEBI, UniChem) and therefore require network access.

Linting and formatting¶

Linting and formatting use ruff:

ruff check     # lint
ruff format    # format

Type checking¶

Type checking is performed with ty:

tox r -e ty

uvx ty check

The configuration lives in [tool.ty] in pyproject.toml. Warnings are treated as errors, so the codebase is kept free of diagnostics. Suppress an unavoidable diagnostic with a rule specific # ty: ignore[rule-name] rather than a blanket comment.

Documentation¶

The documentation is built with Zensical, the static site generator of the Material for MkDocs authors. The sources are markdown files in docs/, the site is configured in zensical.toml in the repository root. Nothing rendered is committed: the site is built by the documentation workflow on every push and published to matthiaskoenig.github.io/pymetadata from the develop branch.

Build the site into site/:

uv run zensical build --clean

For writing, the preview rebuilds on save:

uv run zensical serve

The API reference is rendered from the docstrings by mkdocstrings; a page in docs/api/ only contains the module directive:

# omex

::: pymetadata.omex

Docstrings are therefore the place to document functions and classes, the markdown files provide the narrative around them. Adding a module to the reference means adding such a page and an entry to nav in zensical.toml.

Files for agents¶

Agents and language models read markdown, not rendered html. scripts/llms_txt.py writes the files of the llms.txt convention into the built site, i.e., llms.txt as an annotated index of all pages, llms-full.txt with the complete documentation in a single file, and the markdown of every page next to its html (/omex.md for /omex/). The markdown of the API reference is generated from the docstrings with inspect, since the pages themselves only contain the mkdocstrings directive.

uv run zensical build --clean
uv run python scripts/llms_txt.py

The documentation workflow runs both steps, so the files are regenerated with every push. docs/robots.txt points crawlers at the sitemap and at these files. Zensical will provide agent context files itself at some point, then this script can go.

Regenerating the ontologies¶

pymetadata.ontologies.sbo, kisao and pbpko are generated modules and should not be edited by hand. They are rendered from the ontology releases by pymetadata.ontologies._ontology_builder, which is internal tooling for maintainers rather than part of the public API, and therefore not in the API reference:

python -m pymetadata.ontologies._ontology_builder

This needs the optional ontology dependency (pronto), which is not installed with the package because the generated terms work without it. It is part of the development environment, so uv sync --extra dev covers it.

It downloads the OWL files of the packaged ontologies, stores them gzipped under src/pymetadata/resources/ontologies/ (not part of the repository), and writes one python module per ontology: a class with one attribute per term, documented with the definition of the term so that editors show it, plus the information registered on the class; the behaviour comes from OntologyTerm in pymetadata.ontologies.term. The modules are written with plain python string building, there is no template engine. Adding an ontology means adding an OntologyFile entry and an entry to ontology_patterns with the id pattern of the ontology.

Run ruff format afterwards, since the rendered modules are not formatted.

Release¶

A release is made from develop:

1. update the ontology modules, see Regenerating the ontologies, and commit the changes
2. write the release notes for the version in release-notes/
3. make sure everything passes: tox run-parallel, ruff check, tox r -e ty
4. check the version bump: uvx bump-my-version bump [major|minor|patch] --dry-run -vv
5. bump the version: uvx bump-my-version bump [major|minor|patch], which updates src/pymetadata/__init__.py and CITATION.cff, commits and tags
6. git push --tags, which triggers the release workflow publishing to pypi, followed by git push

7. test the installation from pypi in a fresh environment:

   uv venv --python 3.14
   uv pip install pymetadata
   

8. once Zenodo has archived the release, update the citation information, i.e., date-released in CITATION.cff and the version, date and version DOI of the release in the citation of README.md and docs/index.md. bump-my-version only updates the version, not the date and the DOI, which are only known after the release