Posted on :: 948 Words :: Tags: , , , , ,

AI coding assistants are now good enough to produce useful code quickly.
They are also good enough to produce subtle rubbish quickly. I have seen the same pattern in project after project: velocity improves immediately, but so does low-grade risk unless quality boundaries are explicit and enforced.

In this repository, that became very concrete. We saw how easy it was for:

  • formatting quirks to slip through generated docs;
  • workflow assumptions to drift (for example docs publishing mode);
  • policy intent to be documented but not enforced.

The key insight is simple:

  • prompts are guidance;
  • deterministic checks are policy.

If a constraint can be checked in code, it should not rely on model behaviour.

The reliability gap most teams underestimate

Many teams already lint and test their code, but still treat AI-specific failure modes as “review will catch it”.

In practice, review is exactly where this falls over:

  • odd Unicode/control characters are visually hard to spot;
  • schema-valid output can still be semantically wrong;
  • low-grade anti-patterns accumulate because each one looks minor in isolation;
  • language/tooling drift spreads across repositories unless standards are centralised.

This is why deterministic guardrails matter. They shift correctness from “best effort” to “enforced”.

What deterministic guardrails are actually for

A useful framing is not “block everything suspicious”.
It is “enforce what must always be true”.

For AI-assisted repositories, that usually means five policy layers:

  1. Text integrity
    Ensure files are valid UTF-8 and free from dangerous invisible/control characters.

  2. Repository hygiene
    Catch merge markers, malformed config files, private keys, and basic formatting defects.

  3. Code quality
    Keep lint and test gates deterministic and non-optional.

  4. Project-specific anti-patterns
    Add focused checks for known failure classes (for example embedded SQL literals in Python where you do not want them).

  5. Release/documentation integrity
    Preserve packaging, docs, and link-check quality as first-class policy.

This is less about paranoia and more about designing a reliable socio-technical system around probabilistic tools.

Lessons from this rollout

Three practical lessons stood out while hardening this project:

  1. Policy without enforcement is mostly aspiration
    We had clear intent in docs, but reliability only improved when checks were embedded directly into local hooks and CI.

  2. Docs and release quality are operational concerns, not “nice to have”
    Publishing workflows, link checks, and content integrity checks belong in the same quality conversation as tests and linting.

  3. Small deterministic checks compound
    A UTF-8/invisible-character check, a language-mix heuristic, and a few hygiene hooks are each modest. Together, they materially reduce noise and review fatigue.

Why prek is a strong fit

The practical value of prek in this context is not novelty; it is portability and speed.

You get a single executable path for local hooks and CI usage, with pre-commit-compatible config surfaces and a straightforward route to standardisation across projects.

That consistency matters. Deterministic guardrails only work organisationally when they are easy to install, easy to run, and hard to bypass accidentally.

What we implemented in this repository

To turn principles into enforceable policy, this repo now has:

  • deterministic hygiene hooks (merge-conflict, key detection, JSON/YAML/TOML checks);
  • repository-local safety hooks for:
    • text-safety (UTF-8 + invisible/control character checks),
    • likely SQL-in-Python literal detection;
  • an explicit allowance mechanism for intentional exceptions;
  • CI enforcement of the same prek hook suite to prevent local bypass becoming merge-time drift.

This is deliberate: local checks optimise feedback loops, CI checks preserve policy integrity.

What changed in practice for this repo

After wiring deterministic guardrails into both local hooks and CI:

  • policy checks run the same way for everyone;
  • suspicious text artefacts are caught before merge;
  • likely odd language mixing gets surfaced with explicit waiver mechanics;
  • the “did we remember to run X?” question largely disappears.

That is the real value: less ambiguity, less manual policing, and cleaner review conversations.

What to standardise across projects

If you want this to scale, standardise a “guardrail pack”, not just ad-hoc hooks.

The reproducible pattern is:

  1. Shared hook implementation
    A central repo or package containing safety scripts.

  2. Shared policy defaults
    A baseline prek profile every project inherits or synchronises.

  3. Local extension points
    Project-specific rules added on top, not in place of the baseline.

  4. Explicit waiver model
    Lightweight inline markers for intentional exceptions, with review accountability.

  5. CI parity
    The same gates in CI as local, so policy is deterministic at merge boundaries.

This gives teams the right blend of consistency and autonomy.

What not to do

If you are introducing this pattern, avoid these traps:

  • Do not start with a giant blocking policy.
    Begin with high-signal checks, measure false positives, then tighten.

  • Do not treat waivers as failures.
    Keep a clear waiver path for intentional exceptions, but require explicit rationale.

  • Do not let local and CI policies diverge.
    If CI enforces a different rule set, trust erodes quickly.

  • Do not confuse schema-valid with correct.
    Structural validity is important, but semantic correctness still needs domain checks.

The practical trade-off

Deterministic guardrails are not free:

  • they can produce false positives;
  • they require tuning;
  • they add policy maintenance overhead.

But the alternative cost is hidden and usually worse: silent quality erosion and operational uncertainty that only surfaces during incidents.

The goal is not maximum strictness.
The goal is reliable, explainable constraints that keep AI speed without AI chaos.

Closing position

The pattern I trust most is:

  • LLMs for generation and exploration;
  • deterministic guardrails for enforcement;
  • humans for judgement and exceptions.

That division of labour is what keeps modern AI-assisted delivery fast, accountable, and boring in the ways that production systems should be.