Add ci/selftest.sh: prove the checks can still fail
Three rounds of fixes to this kit, and each round found the previous round's blind spot: a check that examined nothing, then a check that examined everything with a pattern that could not match, then a flag that made one rule skip binary files without a word. None of them were visible from a green run, and each was found by a sibling plugin rather than by anything here. So the battery stops being a shell loop somebody retypes and becomes a script CI runs. 26 assertions: 13 credential shapes that must be flagged, 6 lookalikes that must stay quiet, the direct-access count, the four text-domain verdicts, the annotated path being a real file, and a plugin.conf with its floors stripped failing to load. 🔴 It asserts on the thing that has CONSEQUENCES — the exit status of the script CI runs, and the specific failure message. Never a filename, a count, or a log line that merely sits near the signal. The previous ad-hoc battery asserted "the fixture's filename appears in the output", which passes on grep's own `binary file matches` diagnostic, because that diagnostic prints the filename in exactly the place a real failure would. A sibling plugin recorded two binary cases as PASS on that basis. The probe had the same defect as the check it was testing. The self-test was then checked to go red: making the AKIA pattern inert fails exactly one assertion, naming the shape. Its first run failed an assertion, and the fixture was at fault rather than the product. This script sources ci/lib.sh, so the floors are in the environment and a plain subshell inherits them — ci_load_conf then passed over a plugin.conf that does not define them. Fixed with an explicit unset, and the reason is inline so nobody weakens the assertion instead. A failing probe is evidence about the probe until you have looked at it. Wired into the static job in BOTH workflow roots, byte-identical: .gitea hides .github completely rather than merging it, so a step added to one root only would not run on Gitea and would look exactly like a runner that had not picked the job up. ci/workflows-in-sync.sh passes. Excluded from the release zip, and it plants nothing in the real tree — the fixtures go into a throwaway copy under mktemp, several being credential-shaped by definition. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
3f85642235
commit
0a58661864
@@ -30,6 +30,14 @@ jobs:
|
||||
- name: Lint, hygiene, versions and compatibility headers
|
||||
run: ci/static.sh
|
||||
|
||||
# Runs static.sh again against planted fixtures and asserts it goes RED.
|
||||
# Every check in ci/ has been silently hollow at least once — reporting
|
||||
# success having examined nothing, then examining everything with a
|
||||
# pattern that could not match. Neither is visible from a green run, so
|
||||
# the only defence is a thing that must be found.
|
||||
- name: The checks can actually fail
|
||||
run: ci/selftest.sh
|
||||
|
||||
- name: The release zip is buildable and complete
|
||||
run: ci/build-zip.sh
|
||||
|
||||
|
||||
Reference in New Issue
Block a user