Insights

AI & Strategy

Your AI Coding Tools Are Lying to You (And Your Engineers Don't Know It Yet)

Pull request sizes are up 33%. Change failure rates are climbing nearly 30%. The culprit isn't laziness — it's confidence without context.

February 21, 2026 · 4 min read · Asim Mohammad · Originally on Substack

Across the portfolios I diligence, the same pattern shows up in the metrics before anyone names it. Pull request size is up. Cycle time looks "faster." Escaped defects and rollback rates climb in the same quarter. Leadership celebrates velocity. Production pays the invoice.

The tools are not malicious. They are confident. They emit plausible code against incomplete context — missing invariants, stale domain rules, tests that assert the wrong thing. Engineers who treat that output as finished work ship risk they cannot see.

Context engineering is the CTO-level discipline here. Who owns the system of record for requirements? Which specs are machine-readable? What is the definition of done when an agent wrote half the diff? If those answers are fuzzy, AI coding tools amplify whatever chaos already existed.

I ask three questions in diagnostics. First: can a new engineer reconstruct why a change was allowed without pinging three people? Second: do your automated checks catch the failure modes that actually hurt customers — or only the ones that are easy to unit test? Third: when an AI suggestion conflicts with an architectural decision, who wins, and how is that recorded?

Most teams fail the second question. They have coverage theater: high line counts, weak behavioral assertions, and no path from product risk to test priority. AI makes that worse because it fills the gaps with fluent nonsense.

The fix is not a ban on assistants. It is making verification the scarce resource. Specs before generation. Contracts that fail loudly. QA capacity that scales with merge rate — human or autonomous — instead of hoping review culture keeps up.

If your change failure rate rose while AI adoption rose, do not treat that as a people problem first. Treat it as a verification problem. The code got cheaper. Correctness did not.

That is the 2026 CTO job: decide how much unverified code the business is willing to ship, then build the system that enforces the answer.

Related