Run verify in GitHub Actions

Run ctrlrun verify against your policy on every push with the CTRLRun action: the workflow, the report it produces, the N/A line, the exit codes.

Your policy lives in the repository with the agent. Every push should prove the guarantees it declares still hold against it, in a scratch store, reaching nothing outside the runner, and fail the build if one does not. That is one workflow step.

The policy

runnable

schema: ctrlrun.policy/v2

actions:
  stripe.refund:
    effect: "refund:{payment_id}"
    rules:
      - when: { amount_gte: 0, amount_lte: 50000 }
        decision: allow
      - decision: approve
  k8s.delete_namespace:
    effect: "namespace:{cluster}:{name}"
    decision: approve

The code

Locally, and in the recipe’s directory, the check is one command:

runnable file=run.sh

ctrlrun verify
ctrlrun verify --json > verify-report.json
python -c "import json; s = json.load(open('verify-report.json'))['summary']; print('applicable', s['applicable'], 'passed', s['passed'], 'not applicable', s['not_applicable'])"
rm -f verify-report.json

In CI, the workflow:

name: CTRLRun verify

on: [push, pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: CTRLRun/ctrlrun@v0.6.1
        with:
          policy: ctrlrun.yaml

The ref pins the action’s steps and not the package they install: install defaults to ctrlrun, unpinned, so the job takes whatever PyPI serves that day. Add install: ctrlrun==0.6.1 to pin the tool too, and pin CTRLRun/ctrlrun by commit rather than by tag where you want a ref nobody can move.

What the agent sees

The agent sees nothing; this is the operator’s check. The build sees:

CTRLRun verify — ctrlrun 0.9.0, catalogue ctrlrun.guarantees/v5
policy     /private/var/folders/gp/jrq5dccs6xldv1jw839n7dsw0000gn/T/tmpmax85d9e/ctrlrun.yaml (ctrlrun.policy/v2, mode: enforce)
authority  none
store      sqlite, scratch (created and destroyed for this run)

G1   mutated approval refused         PASS  k8s.delete_namespace
G2   replayed approval refused        PASS  k8s.delete_namespace
G3   duplicate effect refused         PASS  k8s.delete_namespace
G4   one winner under concurrency     PASS  k8s.delete_namespace (8 processes)
G5   ambiguous blocks a blind retry   PASS  k8s.delete_namespace
G6   unknown action refused           PASS
G7   no principal refused             PASS  k8s.delete_namespace
G8   expired authority refused        N/A   no authority section
G9   delegation cannot escalate       N/A   no authority section
G10  unknown exception is ambiguous   PASS  k8s.delete_namespace
G11  an altered receipt is detected   PASS  k8s.delete_namespace
G12  a byte written is ambiguous      PASS  k8s.delete_namespace
G13  clock divergence is named        N/A   the store verify was given reads only the application's clock, so there is no second clock to diverge from; pass --store-url postgresql://… to grade this
G14  token changes across a renewal   PASS  k8s.delete_namespace (attempt 1 and its renewal carry different tokens)
G15  renewal past the ceiling refused N/A   no action verify can drive to allow or approve declares both `effect:` and `max_attempts`
G16  a moved fingerprint is refused   PASS  k8s.delete_namespace
                                            (verify supplies its own precondition provider; whether
                                            your @protect declares one is in your code, which verify
                                            does not read. The gateway and the ACS hook cannot name a
                                            provider at all, and refuse an approval that carries a
                                            fingerprint)
G17  an unentitled approver refused   N/A   no cited control names an approver role
G18  the requester cannot approve     PASS  k8s.delete_namespace
G19  one principal counts once        N/A   no action requires more than one approval
G20  revoked before its exp: no       PASS  k8s.delete_namespace
                                            (G20 is graded against a revocation feed verify supplies:
                                            whether this deployment configures one is a fact about its
                                            own code, which verify cannot read)
G21  unapproved policy decides no     PASS  stripe.refund
                                            (G21 is graded with require_approved_policy set by verify:
                                            whether this deployment sets it is a fact about its own
                                            code, which verify cannot read)
G22  held budget refuses next reserve N/A   no authority section
G23  a failing scope provider refuses N/A   no action this configuration admits carries a resource
G24  grant refused off its task       N/A   no authority section
                                            (a token is unique only as far as your effect keys are:
                                            two stores sharing a provider account must not produce the
                                            same effect-key string for different effects, and nothing
                                            here can check that)

15/15 declared guarantees pass. 9 not applicable: G8, G9, G13, G15, G17, G19, G22, G23, G24.

The first line is on stderr, from G7’s own scenario driving an action with no principal — the guarantee passing, not a problem. Every row names k8s.delete_namespace because verify takes the first action that fits each scenario in alphabetical order; which one appears says nothing about it. The policy line is the resolved absolute path, so yours will differ.

Four guarantees are not applicable: no authority: section (G8, G9), no second clock in a scratch store (G13), no max_attempts to refuse past (G15). Each is listed with its reason and excluded from the denominator. Green means nothing that could be checked was wrong.

The receipt

The report is the receipt: --json writes a ctrlrun.verify/v1 document and --junit a JUnit file, and the action uploads both with the badge JSON as one artifact. Exit 0 means every applicable guarantee passed; 1 a failure; 2 a refused or unusable configuration, including mode: observe and a policy in which nothing can be exercised; 3 an internal error.

When an AMBIGUOUS appears

Verify’s G5 and G10 make an ambiguous effect on purpose, in the scratch store, and assert that a blind retry is refused. Your store is never opened, so nothing here can leave a real effect ambiguous. An AMBIGUOUS in your own store is the agent’s, and the resolve recipe is for it.

Run it

The policy and the code above are also a directory in the repository, extracted from this page and run in CI against a fake remote:

git clone https://github.com/CTRLRun/ctrlrun && cd ctrlrun/examples/cookbook/verify-in-github-actions
pip install ctrlrun && bash run.sh

Next

• Verify in CI: inputs, outputs and publishing the badge.
• Exit codes · Get started · Why.