The ringmaster CLI
ringmaster is the companion command-line tool. It runs the same workflows as the app, against the same workspaces and run history, with no UI, which makes it the way to put Ringmaster in CI, batch jobs, and scripts. Its exit codes are stable and documented, so a pipeline can branch on them.
The CLI is distributed as a signed, notarized standalone binary. Put it on your PATH and you're set.
Commands at a glance
ringmaster version
ringmaster providers list
ringmaster validate <workflow>
ringmaster run <workflow>
ringmaster replay <run>
ringmaster compare <attempt-dir>
ringmaster export-report <run>
ringmaster dataset build
ringmaster eval <dataset>
ringmaster package export <workflow>
ringmaster package import <package>
Every command takes --help. Add --json to the ones that support it for machine-readable output; the plain-text output is meant for humans, so use --json whenever a script needs to parse the result.
version
ringmaster version
Prints the version and build number.
providers list
ringmaster providers list
ringmaster providers list --json
Lists the runtimes, whether each is available in this environment, and the models it offers. With --json you get an array of objects (id, displayName, runsLocally, available, availabilityHint, models, …), useful for asserting a runner has what a workflow needs before you run it.
validate
ringmaster validate path/to/workflow.yaml
Checks a workflow without running it: schema, identifiers, stage references, runtime IDs, and validator config. Exits 0 if valid, 2 if not. Add --json for a structured diagnostic report. This is the natural first step in any pipeline.
run
ringmaster run path/to/workflow.yaml [--auto-approve] [--acknowledge-policy] [--allow-custom-scripts] [--output <dir>] [--json]
Runs a workflow headlessly, advancing as far as the automation rules allow and stopping at the points that need a human. It stops at a review gate, a candidate that needs selecting, a blocking validation failure, a policy that requires approval, a licensing refusal, or a pre-flight error.
| Flag | Effect |
|---|---|
--auto-approve | Plain-approve a succeeded single-agent or race attempt that has no blocking validation failures. It never selects among multiple candidates and never overrides validation. |
--acknowledge-policy | Proceed past a stage whose policy requires approval. Recorded in the run's events. Mirrors the app's "Approve and Run". |
--allow-custom-scripts | Permit custom_script validators, the headless stand-in for the app's consent prompt. Without it, a stage with a custom script is refused at pre-flight. Pass it only when you've reviewed the script. |
--output <dir> | Write runs somewhere other than the workspace's .ring/runs/, handy for throwaway CI artifacts. |
--json | Emit one JSON object per lifecycle event, then a final summary object, on stdout. |
The JSON summary includes the run ID and directory, why it stopped, the exit code, and per-stage the canonical artifact paths the approved attempt produced.
replay
ringmaster replay .ring/runs/<run-id> --stage <id> --attempt <n> \
[--slot <id>] [--runtime <id>] [--model <id>] [--local-only] [--json]
Re-runs a recorded attempt, feeding it the original prompt and inputs. --stage and --attempt are required. Use --runtime/--model to retarget, --slot to scope to one slot of a multi-agent attempt, and --local-only to refuse any escalation to a paid provider. See Replay.
compare
ringmaster compare .ring/runs/<run-id>/stages/<stage>/attempts/<n>
Prints the comparison table and consensus block for a multi-agent attempt. --json emits the raw comparison.json.
export-report
ringmaster export-report .ring/runs/<run-id> --stage <id> --attempt <n> \
[--kind execution|comparison|provenance] [--format markdown|json] [--out <file>]
Exports a report for an attempt. --kind defaults to execution, --format to markdown. Without --out it writes to stdout. The output is byte-for-byte the same as the app's export.
dataset
ringmaster dataset build [--workspace <dir>] [--workflow <id>] [--stage <id>] \
[--status <status>] [--name <name>] [--out <file>]
Builds an eval dataset by querying your run index: it gathers matching attempts into a portable list you can replay later. With no filters it takes every approved attempt; narrow it with --workflow, --stage, or --status. Writes JSON to --out, or to stdout. A Professional feature.
eval
ringmaster eval <dataset> [--workspace <dir>] [--runtime <id>] [--model <id>] \
[--local-only] [--format markdown|json] [--out <file>]
Replays a dataset against a candidate model and emits a regression report: for each entry it compares the candidate's validation result, cost, duration, and output against the recorded baseline, then summarizes what improved, what regressed, and what held steady. Name the candidate with --runtime/--model (omit them to re-run on the original), and use --local-only to refuse any escalation to a paid provider. --format defaults to markdown; --out writes to a file instead of stdout. A Professional feature.
It exits 0 when nothing regressed, 2 when at least one entry's validation regressed (a baseline that passed now fails), and 7 when the dataset produced nothing to compare, so a pipeline can tell a real regression apart from an empty evaluation.
package
ringmaster package export <workflow.yaml> [--out <dir>] [--package-version <version>]
ringmaster package import <package-dir> [--into <dir>]
Bundles a workflow and its files into a shareable, checksummed package, or installs one. export writes a package folder (a Professional feature); import is free and verifies every file against its checksum before installing, refusing anything incomplete or modified. See Workflow packages for the full story.
Exit codes
The codes are a frozen contract, safe to branch on in CI.
| Code | Meaning | Commands |
|---|---|---|
0 | Success | all |
1 | Usage or internal error (bad arguments, missing file, write error) | all |
2 | Validation failure: an invalid workflow, a blocking validation failure at run time, a package that fails its integrity check, or a validation regression in eval | validate, run, replay, eval, package |
3 | Stopped at a review gate: the hand-off worked; open it in Ringmaster to review | run |
4 | Policy refusal (a stage needs approval; pass --acknowledge-policy) | run, replay |
5 | Licensing refusal (a gated mode or feature without the entitlement) | run, eval, dataset, package |
6 | Pre-flight failure (a provider is unavailable, a credential is missing, or a custom script wasn't allowed) | run |
7 | Nothing to evaluate (the dataset produced no comparison) | eval |
Credentials in CI
A headless run never prompts the Keychain; it reads credentials from the environment. For each hosted API, Ringmaster checks its own RINGMASTER_* variable first, then the provider's conventional one:
- Gemini API:
RINGMASTER_GEMINI_API_KEY, thenGEMINI_API_KEY, thenGOOGLE_API_KEY - OpenRouter:
RINGMASTER_OPENROUTER_API_KEY, thenOPENROUTER_API_KEY
CLI-agent subprocesses inherit the environment unchanged, so their usual variables (ANTHROPIC_API_KEY, OPENAI_API_KEY, and so on) flow straight through. Local servers need no credentials. A missing credential fails pre-flight (exit 6) with a message naming the variable it wanted.
A CI example
# Fail the build if the workflow is malformed.
ringmaster validate workflow.yaml || exit 1
# Run it; treat "stopped at a review gate" as a soft stop, anything else as failure.
ringmaster run workflow.yaml --auto-approve --output "$RUNNER_TEMP/runs" --json
case $? in
0) echo "completed" ;;
3) echo "stopped at review gate: open in Ringmaster"; exit 1 ;;
*) echo "run failed"; exit 1 ;;
esac
Sharing a workspace with the app
The CLI and the app can work the same workspace safely. They can't touch the same run at the same time; Ringmaster marks a run with a writer lock while it's active, and the other process will refuse it rather than corrupt it. Licensing behaves exactly as in the app: single-agent runs and replay always work; gated modes need an active license and refuse at pre-flight with the same message you'd see in the UI.