Why meridian-tools exists

Google Meridian is the modelling engine. meridian-tools is the workflow layer around that engine. It makes a Meridian modelling project easier to review, rerun, compare, hand over, and refresh without forking or modifying Meridian.

The package exists for agency MMM work where a fitted model is not the only deliverable. A team also needs to defend why one specification was selected, show exactly what was run, preserve the artefacts needed for review, and rerun or compare the work after the original notebook state has disappeared.

meridian-tools is therefore not a replacement for Meridian. It is the operating standard around Meridian.

The problem it solves

A bare modelling workflow can fit a Meridian model, but an agency workflow has more obligations:

  • defend a model choice on out-of-sample grounds, not only visual fit;
  • declare validation before fitting, rather than after seeing results;
  • keep the authored configuration and the resolved execution configuration;
  • record which input data, library versions, and artefacts produced the result;
  • hand a stable run directory to another analyst, reviewer, or client team;
  • refresh or compare stored runs without relying on notebook memory.

Those obligations are not theoretical. They are the difference between “this model ran on my machine” and “this model can be reviewed six months later”.

The evaluation gap

In-sample fit metrics such as R² or MAPE are useful diagnostic summaries, but they are not sufficient model-selection criteria. They are computed on data the model has already seen. A more flexible specification can improve in-sample fit while degrading on future weeks.

Expected log predictive density (ELPD) estimates expected performance on data the model has not seen. That is the relevant question when choosing between candidate MMM specifications: which specification is most likely to generalise?

meridian-tools adds a compatibility-aware model-selection layer on top of Meridian and ArviZ:

Need meridian-tools surface
Compute PSIS-LOO compute_loo(...)
Compute WAIC compute_waic(...)
Compare candidate models compare_models(...)
Inspect pointwise reliability loo_pointwise.csv with pareto_k
Record incompatibility model_selection_status.json with a reason code
Record ArviZ warnings model_selection_warnings.json

The important integration point is log-likelihood reconstruction. ArviZ needs a pointwise log_likelihood group before it can compute LOO or WAIC. A fitted Meridian model does not expose that group as a ready-to-use workflow artefact. meridian-tools reconstructs it for supported Meridian versions and passes a temporary InferenceData copy to ArviZ. The original Meridian model is not mutated.

Model comparison is still a statistical judgement. In the comparison table, elpd_diff must be read against dse, the standard error of the difference. If the ELPD difference is small relative to its uncertainty, prefer the simpler or more interpretable specification rather than treating rank as a mechanical decision rule.

A principled Bayesian workflow

meridian-tools keeps ownership explicit. Meridian remains responsible for the model. The wrapper owns the workflow controls around the model.

Workflow stage Where it happens Owner
Prior specification and contract validation YAML config and priors.py meridian-tools
Holdout planning before fitting validation config and cv.py meridian-tools
Posterior sampling Meridian Meridian
Meridian diagnostics Meridian outputs staged by the runner Meridian, packaged by meridian-tools
Out-of-sample information criteria compute_loo(...), compute_waic(...) meridian-tools
Candidate model comparison compare_models(...) meridian-tools
Stored run refresh and comparison lifecycle.py meridian-tools

Declaring validation in configuration is itself a control. It prevents the holdout window from being chosen after results are known. blocked_tail executes one contiguous tail holdout. rolling_origin materialises multiple expanding-window splits through the Python API. In both cases, validation fits are kept separate from the final full-sample production fit.

Reproducibility and governance

Every completed run is a directory that can be inspected without the original notebook. The key files are:

Artefact What it answers
00_run_metadata/config.source.yaml What did the analyst author?
00_run_metadata/config.resolved.yaml What did the system actually run?
00_run_metadata/input_data_provenance.json Which data snapshot was loaded?
run_manifest.json Which stages ran, with which versions and artefacts?
30_model_assessment/* What diagnostics and model-selection evidence were produced?

The provenance record includes the authored path, resolved path, SHA-256 hash, row count, column count, ordered columns, file size, modification time, and provenance schema version. The manifest records meridian-tools and Meridian versions, timestamps, run status, stage status, and a validated inventory of top-level artefacts.

That gives a reviewer three concrete answers:

  1. What was modelled? The archived source config and data provenance.
  2. What did the system execute? The resolved config and manifest.
  3. Can it be rerun or compared later? The run record, lifecycle helpers, seeds, pinned dependency boundary, and stored artefact paths.

New manifest version 4 runs also validate completed artefact paths before the final manifest is written. The validation requires recorded artefact paths to be relative paths under the run directory and to resolve to existing regular files. A completed manifest should not point to missing files or paths outside the run directory.

Why not modify Meridian directly?

Meridian remains an unmodified upstream dependency. The current supported boundary pins google-meridian[schema]==1.5.3 and constrains related runtime dependencies where needed for that version.

A private fork or local patch set would increase upgrade risk, complicate support, and blur the boundary between Google’s modelling library and agency-specific workflow requirements. Keeping the split explicit means a Meridian upgrade is a version change, a compatibility review, and a release gate, not a merge from a private modelling fork.

This is also the right separation of concerns. Meridian should focus on the MMM model. meridian-tools should focus on the repeatable agency workflow around that model.

Capability map

This table describes the repository’s supported Meridian 1.5.3 workflow boundary. It is not a claim about every possible Meridian usage pattern or future Meridian release.

Capability Meridian role meridian-tools role
Core MMM fitting Owns the model and posterior sampling Delegates to Meridian
Model serialisation Provides schema serialisation Stages meridian_model.binpb in the run directory
Project configuration Accepts model inputs through Meridian APIs Provides a typed YAML project surface
Validation planning Accepts holdout masks Builds blocked-tail, rolling-origin, and authored-holdout run specs
Diagnostics Provides diagnostic outputs and plots Packages diagnostics into stable artefacts
LOO and WAIC Supplies fitted model state Reconstructs log likelihood and calls ArviZ
Model comparison Not the workflow owner Provides compare_models(...) and run artefacts
Run provenance Not the workflow owner Writes config archives, data provenance, and manifests
Stored run lifecycle Not the workflow owner Loads, compares, and refreshes stored run records
Client handoff Analyst-owned without a wrapper Provides a predictable staged output directory

When should a team use this?

Use meridian-tools when at least one of these is true:

  • you need to compare candidate Meridian specifications;
  • the model will be refreshed later;
  • another analyst must review or rerun the work;
  • the result will be handed to a client or internal governance process;
  • you need a repeatable CLI or YAML workflow rather than notebook-only state.

Do not add the wrapper for its own sake. A single exploratory Meridian fit with no comparison, no refresh cadence, and no handoff requirement may not need this layer. The value appears when the model becomes part of an operating process. For a shorter decision note, see the adoption brief.

What this does not do

The boundary is deliberately narrow.

  • It does not replace Meridian’s model or sampler.
  • It does not compute convergence diagnostics such as R-hat, ESS, or divergences itself. It stages Meridian diagnostic outputs and adds workflow artefacts around them.
  • It does not provide a full prior-predictive checking policy. It can request Meridian prior sampling through fit.sample_prior_draws, validates configured prior contracts, and records resolved prior distributions, but it does not add a wrapper-owned pass/fail rule for prior predictive checks.
  • It does not make LOO or WAIC available for holdout-fitted models. Those runs record model_selection_status.json because comparing holdout-fit ELPD to full-fit ELPD would be statistically ambiguous.
  • It does not make a run perfectly reproducible across all hardware, dependency versions, random seeds, or future Meridian releases. It records the conditions needed for a bounded, reviewable rerun.

These limits are intentional. A wrapper that hides Meridian’s responsibilities would be harder to trust. meridian-tools is valuable because it makes the boundary visible.

The short version

Meridian gives the agency a modelling engine. meridian-tools gives the agency a repeatable scientific workflow around that engine: validation plans, model-selection evidence, provenance, manifests, stable artefacts, and lifecycle operations.