Run an OpenAPI diff in GitHub Actions

Q: Should every detected change fail CI?

Fail confirmed breaking changes and report additive changes without blocking unless your policy says otherwise.

Q: Should the workflow compare generated or source specs?

Compare the artifact that consumers actually receive, while also validating its source generation path.

A useful pull-request check compares the consumer-visible contract from the base revision with the candidate contract, then fails only when policy says the change is unsafe.

Use an immutable baseline

Check out the pull request code and read the base spec directly from the base commit. Do not compare against a mutable production URL unless monitoring production drift is the explicit goal. A remote file can change while the workflow runs and makes the result difficult to reproduce.

git show "$BASE_SHA:path/to/openapi.yaml" > /tmp/openapi-base.yaml
cp path/to/openapi.yaml /tmp/openapi-head.yaml
npm run openapi:diff -- /tmp/openapi-base.yaml /tmp/openapi-head.yaml

The final command is intentionally a project script. Pin the chosen diff engine and its arguments in your repository so local and CI behavior stay identical.

Validate before comparing

Run structural validation on both documents first. A diff from one invalid document is not a trustworthy compatibility decision. Validation failures should identify which revision is invalid and stop the comparison with a clear message.

Separate severity from workflow policy

Have the diff step classify breaking, potentially breaking, and non-breaking changes. Let a small policy layer decide the exit code. Confirmed removals or narrowed request schemas can fail the job. Additive operations can be reported in the summary without blocking. Potential changes should request review when client behavior is unknown.

Publish a durable report

Console logs are hard to scan and disappear into long workflows. Write a Markdown or JSON report, add a short job summary, and upload the full artifact. Include operation identifiers and schema paths, but do not include secrets, example credentials, or private server URLs from untrusted specs.

Protect the workflow

Pull request content is untrusted. Avoid evaluating arbitrary code from a changed spec, do not expose write tokens to forked workflows, and keep permissions read-only unless a separate trusted job needs to comment. Pin dependencies and action versions according to your repository security policy.

Test the gate itself

Keep small fixture pairs for one breaking, one additive, and one invalid change. Run them in the same CI job so a dependency update cannot silently turn a failing rule into a passing one. A green diff workflow is meaningful only when its negative case is regularly exercised.

Frequently Asked Questions

Which file should be the baseline?

Use the exact file from the pull request base revision, not a separately downloaded mutable URL.

Should every detected change fail CI?

No. Fail confirmed breaking changes and report additions unless your policy says otherwise.

Should the workflow compare generated or source specs?

Compare the artifact consumers receive, while also validating its source generation path.