An insurance broker reads policy documents with AI — with an audit trail an auditor can love

Representative examplesEvery case study in this library is an illustrative composite of the kind of engagement we deliver — written to show our method and standards, not to name clients.

Where they started

Brokers at this commercial-insurance brokerage place coverage for businesses of every shape, and every placement means reading policy documents: covers, exclusions, warranties, renewal dates, buried in forty pages of carrier-specific language. The work is regulated, the documents are client-confidential, and the firm's compliance posture was written by a broker who has seen vendors promise security and deliver a paragraph. Nothing leaves their cloud tenancy, and every client-facing summary carries a broker's signature — those facts shaped everything that followed.

What it was costing

Brokers reviewed every client's policy PDFs manually to extract covers, exclusions, and renewal dates for comparison. Forty-page documents, four hours each, and the extraction quality depended on which broker was tired that day.

What they could see

• Preparing a client's policy comparison meant hours with PDFs open, extracting covers and exclusions into a spreadsheet by eye.
• Extraction quality depended on who was tired; two brokers reviewing the same policy produced different summaries.
• Renewal dates and exclusion clauses were the fields that slipped, and both are the ones that cost money when missed.
• Senior brokers spent their expertise on transcription instead of on the placement decisions only they could make.

The constraints we worked inside

• Regulated industry: extraction errors create client harm — every field needs provenance.
• Client documents are confidential — the pipeline had to run in the brokerage's own cloud tenancy.
• Brokers sign off every summary — the AI extracts, the broker verifies, always.

What had been tried before

Copy-paste templates in Word were standardized across the brokerage for policy summaries.

The template sped up formatting, not reading — the hours stayed, the errors stayed, and provenance for any extracted clause was whatever the broker remembered.

A carrier-specific parsing tool was bought for the two largest carriers.

Carriers outnumber templates and change layouts at renewal; the tool covered a fraction of documents and went quiet when formats shifted.

What we proposed

We proposed an extraction pipeline that runs entirely inside the brokerage's own cloud tenancy, maps every policy document to a structured schema with per-field confidence and page references, and marks broker verification as a mandatory gate before anything reaches a client. Provenance was the design's spine: an auditor must be able to walk from any extracted clause back to its source page, and from any release back to the eval set that certified it. Previously hand-extracted files became the benchmark, so every release is measured against what the brokers themselves produced.

Just as important is what we ruled out, and why:

• Sending documents to a third-party extraction service — Client confidentiality is contractual; the pipeline had to run inside the brokerage's own cloud tenancy with no external retention, which ruled the SaaS route out.
• Fine-tuning a private model per carrier — Carrier formats shift every renewal cycle; maintaining fine-tunes would have become a permanent engineering habit the firm cannot staff.
• Screen-scraping brokers' desktops with RPA — It replicates the manual pace with brittleness added, and produces no per-field provenance an auditor could accept.

How the work ran

Delivered by the systems pod — engineer + automation specialist over 9 weeks, with working increments reviewed with the client every week.

The stack, and the reasoning

Cloud-hosted LLM API

Deployed inside the brokerage's own cloud account with retention disabled — their compliance checklist reviewed the configuration before a single document was processed.

Document extraction pipeline

Built to read page structure before language, because exclusion clauses live in layout as much as in wording and the schema needs the page reference either way.

Structured output schema

Every field carries its confidence and source page, so a broker verifies against the document instead of against a paragraph of prose.

Eval suite

Previously hand-extracted files became the benchmark; every release runs against what the brokers themselves produced, which is the standard that matters.

Audit logging

Regulated work needs a trail that survives staff turnover — every extraction, correction, and approval is recorded with who and when.

What went wrong

How we worked together

Cadence

A fortnightly review with the compliance officer and senior brokers, with a written extraction-quality report in between each.

Client side

The compliance officer owned the tenancy and approval gates; two senior brokers supplied the human baseline and verified outputs during the pilot.

Decisions

Confidence thresholds, retention settings, and anything audit-facing required the compliance officer's written sign-off before we touched production.

They provided

Forty previously hand-extracted client files for the baseline, sandbox tenancy access, and an afternoon with their compliance checklist.

What changed

The headline: average policy-comparison preparation per client file, verified over 40 files — 4 hrs → 25 min, read from Broker time logs plus eval suite. A second check: field-level extraction accuracy against the human baseline at 96%.

The brokers stopped describing policy review as the worst part of the month. Comparisons now start from a structured draft with page references, and the verification pass — reading the AI's field against the source — is faster and calmer than transcription ever was. Junior brokers handle document sets they would once have escalated, because the schema tells them where to look, and the compliance officer's audit requests are answered by exporting a log instead of reconstructing a memory.

The result was read from Broker time logs plus eval suite against the pre-engagement baseline over the stated window, with a guardrail check on field-level extraction accuracy against the human baseline. Where platform-reported numbers and business outcomes differ, this record says which layer it is quoting.

What they own now

• The extraction pipeline in the brokerage's own cloud account, with credentials.
• The structured output schema and its per-field provenance specification.
• The eval suite with the human-baseline files and a scheduled release run.
• The audit-log export runbook the compliance officer practiced twice.
• A broker training pack covering verification, confidence flags, and escalation.