Table of Contents
- The reliability gap most teams underestimate
- What deterministic guardrails are actually for
- Lessons from this rollout
- Why prek is a strong fit
- What we implemented in this repository
- What changed in practice for this repo
- What to standardise across projects
- What not to do
- The practical trade-off
- Closing position
AI coding assistants are now good enough to produce useful code quickly.
They are also good enough to produce subtle rubbish quickly.
I have seen the same pattern in project after project: velocity improves immediately, but so does low-grade risk unless quality boundaries are explicit and enforced.
In this repository, that became very concrete. We saw how easy it was for:
- formatting quirks to slip through generated docs;
- workflow assumptions to drift (for example docs publishing mode);
- policy intent to be documented but not enforced.
The key insight is simple:
- prompts are guidance;
- deterministic checks are policy.
If a constraint can be checked in code, it should not rely on model behaviour.
The reliability gap most teams underestimate
Many teams already lint and test their code, but still treat AI-specific failure modes as “review will catch it”.
In practice, review is exactly where this falls over:
- odd Unicode/control characters are visually hard to spot;
- schema-valid output can still be semantically wrong;
- low-grade anti-patterns accumulate because each one looks minor in isolation;
- language/tooling drift spreads across repositories unless standards are centralised.
This is why deterministic guardrails matter. They shift correctness from “best effort” to “enforced”.
What deterministic guardrails are actually for
A useful framing is not “block everything suspicious”.
It is “enforce what must always be true”.
For AI-assisted repositories, that usually means five policy layers:
-
Text integrity
Ensure files are valid UTF-8 and free from dangerous invisible/control characters. -
Repository hygiene
Catch merge markers, malformed config files, private keys, and basic formatting defects. -
Code quality
Keep lint and test gates deterministic and non-optional. -
Project-specific anti-patterns
Add focused checks for known failure classes (for example embedded SQL literals in Python where you do not want them). -
Release/documentation integrity
Preserve packaging, docs, and link-check quality as first-class policy.
This is less about paranoia and more about designing a reliable socio-technical system around probabilistic tools.
Lessons from this rollout
Three practical lessons stood out while hardening this project:
-
Policy without enforcement is mostly aspiration
We had clear intent in docs, but reliability only improved when checks were embedded directly into local hooks and CI. -
Docs and release quality are operational concerns, not “nice to have”
Publishing workflows, link checks, and content integrity checks belong in the same quality conversation as tests and linting. -
Small deterministic checks compound
A UTF-8/invisible-character check, a language-mix heuristic, and a few hygiene hooks are each modest. Together, they materially reduce noise and review fatigue.
Why prek is a strong fit
The practical value of prek in this context is not novelty; it is portability and speed.
You get a single executable path for local hooks and CI usage, with pre-commit-compatible config surfaces and a straightforward route to standardisation across projects.
That consistency matters. Deterministic guardrails only work organisationally when they are easy to install, easy to run, and hard to bypass accidentally.
What we implemented in this repository
To turn principles into enforceable policy, this repo now has:
- deterministic hygiene hooks (merge-conflict, key detection, JSON/YAML/TOML checks);
- repository-local safety hooks for:
- text-safety (UTF-8 + invisible/control character checks),
- likely SQL-in-Python literal detection;
- an explicit allowance mechanism for intentional exceptions;
- CI enforcement of the same
prekhook suite to prevent local bypass becoming merge-time drift.
This is deliberate: local checks optimise feedback loops, CI checks preserve policy integrity.
What changed in practice for this repo
After wiring deterministic guardrails into both local hooks and CI:
- policy checks run the same way for everyone;
- suspicious text artefacts are caught before merge;
- likely odd language mixing gets surfaced with explicit waiver mechanics;
- the “did we remember to run X?” question largely disappears.
That is the real value: less ambiguity, less manual policing, and cleaner review conversations.
What to standardise across projects
If you want this to scale, standardise a “guardrail pack”, not just ad-hoc hooks.
The reproducible pattern is:
-
Shared hook implementation
A central repo or package containing safety scripts. -
Shared policy defaults
A baselineprekprofile every project inherits or synchronises. -
Local extension points
Project-specific rules added on top, not in place of the baseline. -
Explicit waiver model
Lightweight inline markers for intentional exceptions, with review accountability. -
CI parity
The same gates in CI as local, so policy is deterministic at merge boundaries.
This gives teams the right blend of consistency and autonomy.
What not to do
If you are introducing this pattern, avoid these traps:
-
Do not start with a giant blocking policy.
Begin with high-signal checks, measure false positives, then tighten. -
Do not treat waivers as failures.
Keep a clear waiver path for intentional exceptions, but require explicit rationale. -
Do not let local and CI policies diverge.
If CI enforces a different rule set, trust erodes quickly. -
Do not confuse schema-valid with correct.
Structural validity is important, but semantic correctness still needs domain checks.
The practical trade-off
Deterministic guardrails are not free:
- they can produce false positives;
- they require tuning;
- they add policy maintenance overhead.
But the alternative cost is hidden and usually worse: silent quality erosion and operational uncertainty that only surfaces during incidents.
The goal is not maximum strictness.
The goal is reliable, explainable constraints that keep AI speed without AI chaos.
Closing position
The pattern I trust most is:
- LLMs for generation and exploration;
- deterministic guardrails for enforcement;
- humans for judgement and exceptions.
That division of labour is what keeps modern AI-assisted delivery fast, accountable, and boring in the ways that production systems should be.