1,901 type errors to zero on a live AI codebase

Type checking had been switched off for a long time. How a shrinking baseline, a safety net of replay tests and import contracts got a fast-moving Python agent service to strict mypy, and the 15 real bugs it found.

Tariq Amarneh18 October 20263 min read

Python
mypy
Architecture
Code quality

Our AI engine is a fast-moving Python service: LangGraph agents, FastAPI, async database access, many contributors. Type checking had been tried before and switched off, and the Makefile still said "temporarily disabled". When the codebase started to feel hard to change, I listed what was actually wrong: a disabled checker, god modules, and packages importing each other in circles. The language wasn't the problem; discipline was.

Type errors: 1,901 → 0; across 271 modules
Package cycles: 115 → 0; enforced by import-linter in CI
Latent bugs found: 15; each fixed with a test

Step 0: a safety net before touching anything

A typing sweep across hundreds of files is a large refactor, and refactoring agents without tests is how you ship a bot that confidently says the wrong thing. Before fixing a single error I added:

Deterministic graph replay tests: 25 scripted cases replayed deterministically through the real graph, so any behaviour change shows up as a diff.
Snapshots of the graph topology and the OpenAPI schema, so a refactor can't quietly add a node or change an API.
import-linter, so the layering rules could be checked in CI later.

Step 1: a baseline that only shrinks

Turning on mypy across the whole codebase at once would have meant a frozen branch for weeks. Instead, mypy went into CI immediately, with an override listing the 274 modules that didn't pass yet:

toml

# Fix a module, then delete its line. Never add one.
[[tool.mypy.overrides]]
module = [
    "app.services.example_service",
    "app.graph.example_node",
    # … 272 more
]
ignore_errors = true

New modules were strict from day one; old ones left the list as they were fixed.

Then the list was worked down to nothing in one large, mechanical PR: 283 files, every ORM model moved to SQLAlchemy's Mapped[...] (with identical generated DDL to prove nothing changed), stub packages added where they existed, and libraries without stubs collected into one explicit override. A week later strict = true and warn_unreachable were switched on. Session state and API boundaries got TypedDicts, with @final on tool results so an in check narrows the type.

One LangGraph gotcha: a node factory typed as Callable[[State], ...] doesn't satisfy add_node's protocol. Typing it as Callable[..., Awaitable[X]] does.

The bugs mypy found

The most convincing argument for strict typing was the list of real bugs the baseline had been hiding:

Dead code that looked alive: RTF text extraction always failed; an empty-reply fallback could never run; loading documents from URLs always returned an empty list.
Async cancellation: asyncio.CancelledError isn't an Exception, so two except Exception handlers let it escape in ways nobody intended.
None where nobody expected it: a missing message in conversation history, upload verification with no metadata returning a 500, deleting a file with no file ID.
Wrong shapes: .values() called on a list; an endpoint returning {} where a list was required, causing a 422.
Caching a failure: an empty speech-to-text result was cached as if it were a real transcript.
An unbound variable on one branch of an event handler.

Step 2: layers you can't cross

Types fix what a function promises. They don't stop an agent node from reaching straight into the database. For that, the layering is written down as import-linter contracts and checked on every PR:

text

routers → bootstrap → consumers → execution → graph
        → services → llm | integrations → infra → repositories → models

forbidden: graph ↛ repositories
acyclic:   no package import cycles anywhere in app/

Getting there meant moving every query into repositories, deleting a database "god class", splitting a 618-line config module, and replacing 356 in-function imports, the usual workaround for import cycles. Layer violations went from 55 to 0, and package cycles from 115 to 0.

Keeping it that way

The rules that stuck are short. No ignore_errors. No bare # type: ignore: each one names its error code and a reason. cast only for invariants that hold by construction. And most importantly: when a gate fails, fix the code; never widen the gate. Every allowlist only shrinks.

Build the safety net first. Replay tests and snapshots made a 283-file PR reviewable.
Use a baseline that can only shrink, so the checker protects new code from day one.
Count the bugs you find. They're what convinces a team that strict typing pays for itself.
Types and architecture rules are different tools. You want both in CI.

Step 0: a safety net before touching anything

A typing sweep across hundreds of files is a large refactor, and refactoring agents without tests is how you ship a bot that confidently says the wrong thing. Before fixing a single error I added:

Deterministic graph replay tests: 25 scripted cases replayed deterministically through the real graph, so any behaviour change shows up as a diff.

Snapshots of the graph topology and the OpenAPI schema, so a refactor can't quietly add a node or change an API.

import-linter, so the layering rules could be checked in CI later.

Step 1: a baseline that only shrinks

Turning on mypy across the whole codebase at once would have meant a frozen branch for weeks. Instead, mypy went into CI immediately, with an override listing the 274 modules that didn't pass yet:

toml

# Fix a module, then delete its line. Never add one.
[[tool.mypy.overrides]]
module = [
    "app.services.example_service",
    "app.graph.example_node",
    # … 272 more
]
ignore_errors = true

New modules were strict from day one; old ones left the list as they were fixed.

The bugs mypy found

The most convincing argument for strict typing was the list of real bugs the baseline had been hiding:

Dead code that looked alive: RTF text extraction always failed; an empty-reply fallback could never run; loading documents from URLs always returned an empty list.

Async cancellation: asyncio.CancelledError isn't an Exception, so two except Exception handlers let it escape in ways nobody intended.

None where nobody expected it: a missing message in conversation history, upload verification with no metadata returning a 500, deleting a file with no file ID.

Wrong shapes: .values() called on a list; an endpoint returning {} where a list was required, causing a 422.

Caching a failure: an empty speech-to-text result was cached as if it were a real transcript.

An unbound variable on one branch of an event handler.

Step 2: layers you can't cross

Types fix what a function promises. They don't stop an agent node from reaching straight into the database. For that, the layering is written down as import-linter contracts and checked on every PR:

text

routers → bootstrap → consumers → execution → graph
        → services → llm | integrations → infra → repositories → models

forbidden: graph ↛ repositories
acyclic:   no package import cycles anywhere in app/

Keeping it that way

Build the safety net first. Replay tests and snapshots made a 283-file PR reviewable.

Use a baseline that can only shrink, so the checker protects new code from day one.

Count the bugs you find. They're what convinces a team that strict typing pays for itself.

Types and architecture rules are different tools. You want both in CI.