GUIDE

Provenance, reports & metrics

Everything Ringmaster does is on the record. Every artifact records how it was made. You can view that record, export it in a couple of formats, or let the app roll the numbers across runs into metrics you can act on.

Provenance

Provenance is the complete lineage behind a single artifact. For any artifact, Show Provenance gathers:

the workflow version, run, stage, and slot that produced it;
the runtime and model that ran;
the prompt file used, and any role overlay applied;
the inputs it consumed (the upstream artifacts);
the validation outcome;
your review decision;
git context and timestamps.

It's assembled from what Ringmaster already recorded during the run, not stored twice, so it always reflects what actually happened. You can copy a provenance record to share it, or export it as a report.

Reports

Reports turn a run into something you can hand to someone else: a reviewer, an auditor, a teammate who wasn't there. There are three kinds, each available as Markdown (for reading) or JSON (for tooling):

Execution: one attempt's prompt, output, runtime/model, validation, and timing.
Comparison: the side-by-side for a multi-agent attempt, including the consensus analysis when present.
Provenance: the full lineage record described above.

Export from the app, or from the command line, which is handy in CI:

ringmaster export-report .ring/runs/<run-id> --stage spec --attempt 2 \
  --kind execution --format markdown --out report.md

The app and the CLI produce identical output, so a report you generate by hand and one your pipeline generates are the same document.

Metrics

Ringmaster aggregates the numbers across your runs so you can answer practical questions: which model actually gets picked most often, what each one costs, how long they take, how often they pass validation.

For each runtime-and-model it tracks things like execution count, success rate, validation pass rate, timeout and cancel rates, how often a human selected it when it was one of several candidates, average duration, and total and average cost (when the runtime reports it). Settings → Runtimes rolls these up across your workspaces, so over time you build an evidence base for which models are worth their cost on your kind of work, instead of going on vendor claims.

Advanced metrics and report export are licensed features; see Installation & licensing.

Trends

Metrics tell you where a workflow stands; trends tell you which way it's moving. Ringmaster buckets a workflow's runs by day or by week and charts how its cost, duration, and validation pass rate change over time. A model that's quietly getting more expensive, or a workflow whose pass rate is slipping, shows up here before it surprises you. Trends fill in once a workflow has a run or two behind it.

Cost budgets

You can set a monthly cost budget for a workspace. Ringmaster tracks your month-to-date spend against it, projects where the month is heading at the current pace, and tells you when that projection runs over. It's a guardrail rather than a brake: nothing gets blocked, you just get an honest read on the trajectory so there are no surprises at the end of the month.

Trust score

A trust score is a single, explainable read on how much a workflow has earned your confidence. You'll find it in the workflow inspector. It's built from three signals taken straight from the run history: how often runs complete, how often executions pass validation, and how often you approve what a human reviewed. Each signal is shown with the evidence behind it (for example, "9 of 10 runs completed"), and the score is only ever the average of the signals that actually have evidence, so a workflow you haven't reviewed yet isn't marked down for it. A brand-new workflow with no history doesn't get a fake zero; it has no score until there's something to base one on.

Trends, budgets, and the trust score are licensed features, alongside the metrics they build on.

Timeline

Every run also keeps an append-only event log. The Timeline view replays it (each attempt start, state change, and completion in order) and lets you filter by stage, agent, or event type to follow one thread through a busy multi-agent run. It's the low-level companion to provenance: provenance tells you about an artifact, the timeline tells you about the run as it unfolded.