Contributing

This guide covers the development setup, conventions, and workflow for contributing to meridian-tools.

Development setup

Clone and install

git clone <repo-url> meridian-tools
cd meridian-tools
python -m pip install -U pip
python -m pip install -c constraints/dev.txt -e ".[dev]"

The constraints file pins the supported development environment, including the Meridian-compatible ArviZ line.

Verify the install

python scripts/verify_release.py

Acceptance gate

Before submitting any change, run the full acceptance sequence from the repository root:

python -m pip install -c constraints/dev.txt -e ".[dev]"
python scripts/verify_release.py

See acceptance.md for the expected results and how to interpret failures.

Code style

Formatting and linting

The project uses Ruff for both linting and formatting:

# Check
ruff check src tests
ruff format --check src tests

# Auto-fix
ruff check --fix src tests
ruff format src tests

Configuration is in pyproject.toml:

[tool.ruff]
line-length = 120
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "C90", "SIM", "RUF"]

Type annotations

All public functions and classes use type annotations. The codebase uses from __future__ import annotations for forward-reference support.

Import conventions

• Standard library imports first, then third-party, then local.
• Heavy dependencies (Meridian, TensorFlow, ArviZ) are imported lazily inside functions, not at module level, in the config/CLI/validation layers.
• Ruff rule I enforces import sorting.

Configuration models

All Pydantic models use ConfigDict(extra="forbid"). New config fields must be added with appropriate types, defaults, and validators.

Testing

Running tests

# Full suite
python scripts/verify_release.py

# Specific file
pytest tests/test_runner.py -v

# Specific test
pytest tests/test_runner.py::test_run_pipeline_writes_manifest -v

Test conventions

• Tests use pytest with tmp_path for temporary directories.
• monkeypatch is used extensively to mock Meridian internals and isolate unit tests from real MCMC sampling.
• Module-scoped fixtures (scope="module") are used for expensive model construction in test_log_likelihood.py and test_model_selection.py.
• Shared test infrastructure is defined inline in individual test modules. There is no top-level conftest.py.

Live Meridian verification

One opt-in command exercises the bounded real Meridian seam:

MERIDIAN_TOOLS_ENABLE_REAL_FIT=1 pytest tests/test_demo_integration.py::test_real_pipeline_refresh_smoke tests/test_log_likelihood.py::test_compute_log_likelihood_dataset_real_posterior_smoke -m real_fit -v

This is not part of the default suite. It proves one reduced real pipeline run over bundled demo data, one stored-run refresh after the original YAML is removed, and the lower-level live log-likelihood seam. Run it after Meridian version upgrades and before release-candidate handoff when you want extra confidence beyond the fast suite.

Writing new tests

• Place tests in the appropriate tests/test_<module>.py file.
• Use monkeypatch to avoid real MCMC sampling in unit tests.
• Test both success paths and error conditions.
• Verify artefact file contents, not just their existence.
• Use tmp_path for all filesystem operations.

Project structure

meridian-tools/
├── src/meridian_tools/       # Package source
│   ├── __init__.py           # Lazy-loading exports
│   ├── artifacts.py          # Manifest helpers
│   ├── cli.py                # CLI entry point
│   ├── config.py             # Pydantic models
│   ├── cv.py                 # Validation splits
│   ├── demo.py               # Demo discovery
│   ├── diagnostics.py        # Diagnostics export
│   ├── exports.py            # Meridian export wrappers
│   ├── launcher.py           # Run execution wrapper
│   ├── lifecycle.py          # Post-run management
│   ├── log_likelihood.py     # Log-likelihood adapter
│   ├── model_selection.py    # LOO/WAIC wrappers
│   ├── terminal.py           # CLI presentation
│   └── version.py            # Static version
├── tests/                    # Test suite
│   ├── _demo_data/           # Bundled demo data (packaged)
├── docs/                     # Documentation
├── runme.py                  # Source-tree demo launcher
└── pyproject.toml            # Build and dependency config

Versioning

The version is defined in src/meridian_tools/version.py:

__version__ = "0.4.0"

Version bumps are manual edits. Update this file when preparing a release.

Documentation

Documentation lives in docs/. When adding new features:

1. Update relevant guide or reference pages.
2. Add API documentation for new public functions or classes.
3. Update the YAML schema reference if config fields changed.
4. Update the output schema if new artefacts are produced.

Common pitfalls

• Do not import Meridian at module level in config, CLI, or validation modules. This breaks CLI responsiveness.
• Do not add extra="allow" to Pydantic models. The extra="forbid" policy prevents silent misconfiguration.
• Do not modify source run directories in lifecycle operations. Always create new sibling directories.
• Do not weaken or delete existing tests without explicit direction.

Acceptance checklist

Use this page as the canonical local acceptance checklist for the current repository state. The acceptance gate is local and command-driven, and it uses the same script as CI.

Acceptance gate

Run the following commands from the repository root:

python -m pip install -c constraints/dev.txt -e ".[dev]"
python scripts/verify_release.py

The canonical acceptance-gate result for the full test step in the current constrained environment is:

485 passed, 2 skipped

That result is the pass or fail line for the default local acceptance gate. The recorded warning profile belongs to the release baseline, not to the acceptance-gate definition itself.

What each command proves

python -m pip check proves that the active constrained environment has no broken package requirements.

python -m compileall src tests scripts runme.py proves that the checked-in Python files parse cleanly. If this step fails, you are dealing with a syntax or import-time parse issue and you should stop there.

python -m meridian_tools.cli --help and python runme.py --help prove that the installed CLI and source-tree launcher surfaces still import cleanly.

ruff check src tests scripts runme.py proves that the repository still satisfies the pinned lint rules. If this step fails, fix the reported lint violations before moving on.

ruff format --check src tests scripts runme.py proves that the checked-in files still match the agreed formatting contract. If this step fails, run the formatter and then rerun the verification sequence.

mypy src proves that the configured static typing baseline still runs cleanly. If this step fails, either fix the reported type issue or update the documented ratchet intentionally.

python docs-site/build_content.py, git diff --exit-code -- docs-site/content, and hugo --source docs-site prove that canonical Markdown, generated Hugo content, and the static site build stay aligned.

pytest --cov=src/meridian_tools --cov-report=term-missing proves the behavioural contract of the repository. This is the broadest local validation step. If it fails, use the failing test names to identify which package contract regressed.

How to interpret failure

If the compile step fails, fix syntax or parse problems first. The later steps will not give you useful signal until that is resolved.

If lint, format, or type checks fail, treat that as a source-tree quality issue, not as an optional clean-up item. Bring the tree back to the pinned Ruff and mypy state before trusting the rest of the loop.

If CLI help fails, assume the published command surface is broken even if the Python modules still import manually.

If the docs drift check fails, update the canonical files under docs/ first, then regenerate docs-site/content.

If the pytest coverage step fails, the acceptance gate is not met. A partial pass is not enough. Fix the failing behavioural contract and rerun the full command sequence.

Optional extra confidence

The repository also carries one opt-in live Meridian verification command for extra technical confidence:

MERIDIAN_TOOLS_ENABLE_REAL_FIT=1 pytest tests/test_demo_integration.py::test_real_pipeline_refresh_smoke tests/test_log_likelihood.py::test_compute_log_likelihood_dataset_real_posterior_smoke -m real_fit -v

This command is not part of the default blocking acceptance gate. It exists to provide one bounded live Meridian route that proves:

• real pipeline execution over bundled demo data
• manifest-backed stored-run refresh after the original YAML is removed
• the lower-level live log-likelihood reconstruction seam

On the reference development environment, the recorded run finished in 135.42 seconds (0:02:15), with command elapsed time of 2:24.03; keep a budget of roughly six minutes or less for this extra-confidence command.

Release baseline

This page records the 0.4.0 release-candidate baseline for the repository. Treat it as a validated project state, not as an automated release system. The baseline uses the canonical constrained verification script and records the observed warning profile, the direct runtime dependency bounds, and the accepted trade-offs that still shape the package.

Release-ready definition in this repository

The repository is release-ready only when the documented local acceptance command set passes, the coverage test step returns the recorded pass/skip count below, the same validated run is recorded with the observed warning profile, the warning categories match the accepted ones below, and the accepted trade-offs remain explicit rather than hidden.

Validated baseline record

The current verified local baseline is:

python -m pip install -c constraints/dev.txt -e ".[dev]"
python scripts/verify_release.py
-> 485 passed, 2 skipped, 83 warnings, 91% coverage
-> pytest runtime 65.27s (0:01:05); command elapsed 1:12.27

The optional extra-confidence live path remains separate:

MERIDIAN_TOOLS_ENABLE_REAL_FIT=1 pytest tests/test_demo_integration.py::test_real_pipeline_refresh_smoke tests/test_log_likelihood.py::test_compute_log_likelihood_dataset_real_posterior_smoke -m real_fit -v
-> 2 passed, 161 warnings in 135.42s (0:02:15); command elapsed 2:24.03

That command remains opt-in local confidence, not the default developer loop or silent CI policy. On the reference development environment, the recorded run finished in 135.42 seconds (0:02:15), with command elapsed time of 2:24.03; keep a budget of roughly six minutes or less for ordinary local execution.

Certified environment

• Python 3.12.11
• meridian-tools==0.4.0
• google-meridian==1.5.3
• arviz==0.19.0
• matplotlib==3.9.4
• protobuf==6.33.6
• ruff==0.14.0
• mypy==1.18.2
• pytest==8.4.2
• Hugo 0.164.0

Runtime dependency boundary

The current runtime boundary recorded from pyproject.toml is:

• requires-python >=3.11
• google-meridian==1.5.3
• arviz>=0.18.0,<0.20.0
• constrained release gate uses arviz==0.19.0 and matplotlib==3.9.4
• protobuf>=5.28.0,<7
• constrained release gate uses protobuf==6.33.6
• pandas>=2.2.0,<3
• pydantic>=2.8.0,<3
• PyYAML>=6.0.0,<7

These are the direct runtime dependency bounds for the milestone baseline. This page does not imply broader environment reproducibility than the constrained gate currently implements.

Accepted warning profile

The recorded warning profile is accepted in the current milestone baseline. The warnings fall into these pinned categories:

• Meridian model / prior warnings
• ArviZ model-selection warnings
• Matplotlib/Pyparsing deprecation warnings from the constrained ArviZ compatibility line
• Meridian schema/protobuf deprecation warnings caused by Meridian 1.5.3 serialising boolean TensorFlow Probability parameters through an int proto path; protobuf 7 rejects that path, so this release candidate constrains protobuf below 7
• Hugo Relearn theme deprecation warnings during static-site build

This baseline does not pretend the repository is warning-free. It records the current observed warning profile honestly and treats those warning categories as accepted for the present milestone.

Accepted trade-offs

The current release baseline also depends on several explicit trade-offs.

The package takes a no-fork Meridian approach. We keep Meridian as the modelling engine and add workflow and compatibility tooling around it rather than modifying Meridian source.

Bayesian model selection remains intentionally limited to fitted Meridian models where holdout_id is None. Validation-fit and authored-holdout runs are not treated as compatible LOO or WAIC candidates.

Lifecycle tooling remains Python-first. The repository does not currently ship a broader lifecycle CLI.

Version bumping remains a manual edit rather than a fully automated release pipeline.

Boundary of this record

This page records one validated milestone state. It does not introduce CI as the source of truth. It does not define publish automation. It does not promise zero warnings. It does not claim a broader release process than the repository actually supports today.

Changelog

All notable changes to meridian-tools are documented in this file.

The format is based on Keep a Changelog.

[Unreleased]

No unreleased changes.

[0.4.0] — 2026-07-29

Added

• YAML media priors — YAML-driven media prior configuration for roi_m, mroi_m, and alpha_m, including scalar priors, per-channel overrides, and support for Normal, LogNormal, TruncatedNormal, and Beta distributions.
• Prior distribution export — prior_distributions.json is written in the model-fit stage so authored and resolved Meridian prior distributions are captured with run artefacts.
• Config templates — Commented starter configs under templates/ cover minimal, standard, media-prior, and blocked-tail validation workflows.
• Geo panel demo artefacts — Stored geo_panel demo run outputs now live under runs/demos/, so reference artefacts cover both bundled pipeline shapes.
• Meridian compatibility inventory — Project documentation now records the Meridian 1.5.3 files and private sampler seams that must be reviewed before changing the dependency pin.
• Validation-spec v2 bindings — Stored validation specs now bind coordinate and data-shape metadata so refresh and rolling-origin execution can fail closed when input data no longer matches the stored contract.
• Ephemeral input provenance — Manifest v4 runs record provenance for the exact temporary snapshot loaded by Meridian.
• Canonical verification gate — scripts/verify_release.py now runs the constrained release checks for dependency consistency, compile/import health, CLI help, Ruff, mypy, Hugo docs, and the coverage suite.

Changed

• Strict run names — Pipeline run names must now match ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$; path separators, whitespace, Unicode, and relative path segments are rejected before run directories are created.
• CSV loader YAML validation — data.coord_to_columns and channel mapping fields now fail earlier with wrapper-owned errors when shapes, keys, or channel ordering are inconsistent with Meridian loader contracts.
• ModelSpec array validation — Known boolean array kwargs forwarded through model_spec.kwargs now reject scalars, ragged lists, strings, and numeric stand-ins before Meridian model construction; their ranks and loaded-data shapes are also checked before model construction.
• Model-selection resilience — Optional LOO/WAIC exports now capture ArviZ warnings and degrade to explicit status artifacts for unexpected ArviZ or artifact-write failures instead of failing the whole run.
• Bounded rolling-origin execution — Each split fits only through its test boundary and records the exact validation window used for execution.
• Manifest v4 completion integrity — Completed runs validate required manifest artefacts as existing, contained regular files before the final manifest is written.
• Release documentation — Architecture, schema, release-baseline, and generated Hugo documentation now describe manifest v4, validation-spec v2, bounded reproducibility, and the canonical constrained gate.

Fixed

• Launcher empty-result handling — The source-checkout launcher now raises a clear runtime error if the pipeline unexpectedly returns no run result.
• Run directory collisions — Same-second runs with the same logical name now use deterministic suffixes such as _001 instead of failing on an existing directory.
• Private destination JSON writes — Newly created destination JSON files now use private file modes.
• Temporary snapshot cleanup — Temporary Meridian input snapshots are removed after handoff while hashed provenance for the loaded bytes is kept.
• Finite numeric configuration — Non-finite numeric, analysis, and nested Meridian configuration values are rejected before they reach Meridian.
• Repo-owned Hugo configuration — The repository-owned Hugo language setting no longer emits a deprecation warning on the supported Hugo line.
• Protobuf compatibility bound — The supported dependency boundary now constrains protobuf<7 because Meridian 1.5.3 serialises boolean TensorFlow Probability parameters through an int proto path that protobuf 7 rejects.

[0.3.0] — 2026-04-24

Changed

• CLI single source of truth — runme.py now delegates directly to meridian_tools.cli, removing duplicate root-level argument parsing.
• Typed runner state — Pipeline orchestration now uses PipelineContext for shared stage state.
• Shared posterior sampling — Runner posterior sampling keyword mapping is centralized in one helper.
• Lifecycle comparison schema — Run comparison rows are generated from declarative comparison field descriptors.
• Meridian compatibility pin — The package pins google-meridian[schema]==1.5.3, and log-likelihood reconstruction refuses unvalidated Meridian versions.
• Static analysis tooling — Development extras now include mypy, and Ruff enables additional complexity, simplification, and Ruff-specific rule families.

Fixed

• Optimized Python safety — Validation helpers now use explicit exceptions instead of assert for runtime invariants.
• Shared confidence validation — Response curve and optimisation configs share one confidence_level validator.
• Export coercion documentation — NetCDF attribute coercion now documents its input-to-output type mapping.

[0.2.0] — 2026-04-07

Added

• Docs site build — Hugo-based website documentation under docs-site/, generated from the repository Markdown set by docs-site/build_content.py.
• Manifest v3 provenance — Explicit input_data_provenance capture for stored runs and lifecycle refresh or compare workflows.
• Typed failure boundaries — ConfigPreflightError, ValidationExecutionContractError, and PipelineRunFailure distinguish wrapper-owned preflight, validation contract misuse, and post-directory runtime failures.
• Bounded live verification — An opt-in Meridian real-fit smoke route gated behind MERIDIAN_TOOLS_ENABLE_REAL_FIT=1.
• Module-path CLI contract — Explicit support and regression coverage for python -m meridian_tools.cli ....

Changed

• Shared launch flow — meridian-tools and the repo-root runme.py launcher now share one launch flow for config loading, preflight checks, progress reporting, and terminal success or failure output.
• Packaged demo assets — Bundled demo configs and datasets are resolved from packaged _demo_data, so demo runs work from installed wheels as well as source checkouts.
• Default demo fit mode — Bundled demos now default to full-sample fits (validation.strategy: none), so loo_summary.json and waic_summary.json are generated by default and 10_validation is recorded as skipped.
• Refresh contract — Stored-run refresh now reloads from the saved resolved config while preserving the original source config copy in run metadata.
• Lifecycle compare semantics — Compare now distinguishes legacy runs without dataset provenance from real dataset changes.
• Documentation layout — Public documentation is reorganised under docs/ into getting-started, guides, reference, concepts, and project sections.

Fixed

• Structured public entrypoint failures — Missing or invalid config paths in public entrypoints now produce structured failure output instead of raw Python tracebacks unless --traceback is used.
• Relative-path refresh — Refreshing a stored run with relative data.path input no longer depends on the original source config location remaining present on disk.
• Partial-run failure reporting — Failed runs that already created an output directory now report the concrete run directory, manifest path, and failing stage through the CLI and runme.py.
• Docs-site theme resolution — Hugo builds resolve the Relearn theme through a pinned module dependency instead of requiring a local theme checkout.

[0.1.0] — 2026-04-02

Added

• Typed YAML configuration — Pydantic-validated config with extra="forbid" strictness for all sections: project, data, model_spec, fit, validation, exports, response_curves, optimisation.
• Staged pipeline runner — Sequential execution through 00_run_metadata, 10_validation, 20_model_fit, 30_model_assessment, 40_decomposition, 60_response_curves, 70_optimisation with manifest persistence after each stage.
• Validation orchestration — blocked_tail and rolling_origin time-series validation strategies with auto-generated holdout masks. Authored holdout passthrough through model_spec.kwargs.holdout_id.
• Diagnostics bundling — diagnostics_bundle.json manifest with optional predictive_accuracy.csv and review_summary.json exports.
• Bayesian model selection — Compatibility-aware LOO and WAIC computation through ArviZ, with automatic log-likelihood reconstruction for fitted Meridian models. Graceful degradation for incompatible runs through structured ModelSelectionError with reason codes.
• Response curves export — Configurable spend multiplier grid with NetCDF and CSV outputs.
• Optimisation export — Fixed-budget and relative-budget optimisation with full artefact set including allocation charts.
• Plot exports — PNG plot artefacts through Altair/vl-convert for model fit, diagnostics, decomposition, response curves, and optimisation stages.
• Lifecycle management — load_run_record, list_run_records, build_refresh_run_config, compare_run_records for post-run analysis and reproducible refresh workflows.
• CLI — meridian-tools run and meridian-tools demo subcommands with lightweight imports for fast startup.
• Bundled demos — timeseries and geo_panel reference workflows with packaged data and configs.
• Manifest versioning — Support for manifest versions 0, 1, and 2 with backward-compatible deserialisation.
• Comprehensive test suite — 218 tests across 15 test files covering configuration, validation, pipeline execution, exports, diagnostics, model selection, lifecycle, and demos.

Meridian Compatibility Inventory

This project currently targets google-meridian[schema]==1.5.3. The local reference checkout used for compatibility review is:

/home/user/Documents/GITHUB/tandpds/meridian

The 0.4.0 release-candidate environment also constrains protobuf>=5.28.0,<7. Meridian 1.5.3 serialises boolean TensorFlow Probability distribution parameters through an int proto path. Protobuf 7 rejects that path, while the constrained protobuf==6.33.6 line preserves the validated Meridian schema save/load smoke route with deprecation warnings.

Reviewed Reference Files

• meridian/version.py
• meridian/data/load.py
• meridian/model/spec.py
• meridian/model/context.py
• meridian/model/model.py
• meridian/model/posterior_sampler.py

Required Symbols And Contracts

• meridian.version.__version__ is 1.5.3.
• protobuf remains below 7 for Meridian schema serialisation.
• meridian.data.load.CoordToColumns defines the CSV coordinate mapping surface.
• meridian.data.load.CsvDataLoader is the CSV input loader used by meridian-tools.
• meridian.model.spec.ModelSpec accepts wrapper-authored kwargs, including holdout_id and calibration/scaling arrays.
• ModelContext.holdout_id validates national holdout masks as (n_times,) and geo holdout masks as (n_geos, n_times).
• Meridian.posterior_sampler_callable returns a posterior sampler with the private reconstruction seams required by the log-likelihood adapter.
• PosteriorMCMCSampler._get_joint_dist_unpinned exists.
• PosteriorMCMCSampler._prepare_latents_for_reconstruction exists.
• PosteriorMCMCSampler._reconstruct_posteriors exists.

Upgrade Checklist

Before changing the pinned Meridian dependency:

1. Compare the reviewed files above against the new Meridian version.
2. Confirm the private posterior sampler seam methods still exist and preserve compatible behavior.
3. Run python scripts/verify_release.py.
4. Run MERIDIAN_TOOLS_ENABLE_REAL_FIT=1 python -m pytest -q -m real_fit.
5. Recheck the protobuf compatibility bound if Meridian schema serialisation changes upstream.
6. Update this inventory and release notes with any compatibility changes.