Wiring wavsummary into a CI pipeline

A single wavelens analyze run answers “did this one file change?” wavsummary answers the question a CI pipeline actually needs: “did anything in this batch change?” — and it answers it with an exit code, not a report you have to open by hand.

What it actually does

wavsummary scans a directory of wavelens reports (the JSON output, written with --json alongside each report’s HTML) and rolls every one of them up into a single pass/fail summary. It doesn’t re-run any checks — it’s purely a rollup of results wavelens already produced.

$ wavsummary ./reports --json

142 reports · 3 failed
exit 1

Exit code 0 means every report in the directory passed. Anything else means at least one didn’t, and the summary output tells you which ones.

The minimal CI step

The whole point is that this fits into a build system as one command with a meaningful exit code:

$ wavelens analyze --ref reference.wav --dut build/output.wav --out reports/output.html --json
$ wavsummary ./reports --json || exit 1

In a real pipeline you’re usually generating many reports first — one per test file, one per platform build, one per content type — and then running wavsummary once at the end as a single gate:

$ for f in build/*.wav; do
    wavelens analyze --ref "refs/$(basename $f)" --dut "$f" --out "reports/$(basename $f .wav).html" --json
  done
$ wavsummary ./reports --json

A failed unit fails the build. Nobody has to remember to open a report by hand for it to count.

What to gate on, and what not to

It’s worth deciding upfront which failures should actually block a merge versus which should just be visible. A codec round-trip that reliably fails Bit-Exactness but passes everything else isn’t a regression — it’s expected behavior for a lossy pipeline (see Bit-Exact vs. “close enough”). If that’s your situation, run wavelens analyze without --bitexact so Bit-Exactness stays non-gating, rather than filtering wavsummary’s output after the fact — the exit code stays meaningful, and nobody has to remember why a “failure” is actually fine.

Archiving the evidence

wavsummary’s exit code is what gates the build, but the individual HTML reports are what someone actually looks at when a build fails. Most teams archive the reports/ directory as a build artifact alongside the wavsummary output — that way a red build comes with the exact report that explains why, not just a number.