NEXSUM_LABS
  1. Home
  2. Work
  3. A factory's new hires stopped hunting through a 400-page handbook
Book a call

[ Case study ]

ManufacturingLLM assistant platformRAG over revisioned handbookTablet UIQuarterly eval checklist

A factory's new hires stopped hunting through a 400-page handbook

Line workers needed machine settings, safety procedures, and spec tolerances from a 400-page PDF handbook that lived on a shared drive. New hires asked supervisors, supervisors stopped what they were doing, and the answers were sometimes out-of-date revisions.

CLIENT a precision-components manufacturer — FOCUS Ingest by revision, cite by revision

AI Chatbots & AssistantsAI & AutomationAI Chatbots & AssistantsManufacturingRepresentative example
Client
a precision-components manufacturer
Industry
Manufacturing
Engagement
6 weeks — systems pod — automation specialist
Service
AI & Automation / AI Chatbots & Assistants
Headline outcome
Average time to find a procedure or spec, observed across 50 shift interactions: 6.5 → 1.5 min, read from Supervisor time study

Representative examplesEvery case study in this library is an illustrative composite of the kind of engagement we deliver — written to show our method and standards, not to name clients.

Where they started

Precision components leave this factory only when the machine settings, tolerances, and safety procedures behind them were exactly right, and all of that lives in a 400-page handbook maintained by the engineering office. The handbook is authoritative, current, and almost unreadable at the point of need — a shared drive PDF consulted between machine cycles, on shared tablets, wearing gloves. Supervisors are the de facto interface: new hires ask them, and the best of them carry the common answers in their heads.

What it was costing

Line workers needed machine settings, safety procedures, and spec tolerances from a 400-page PDF handbook that lived on a shared drive. New hires asked supervisors, supervisors stopped what they were doing, and the answers were sometimes out-of-date revisions.

What they could see

  • New hires interrupted supervisors constantly for settings and procedures the handbook already answered.
  • Answers from memory were sometimes from an older revision, and nobody on the floor could tell the difference.
  • Finding a procedure in the PDF on a shared tablet took longer than walking to the supervisor did.
  • Rarely-needed procedures were effectively unknown — when they came up, the shift stopped until the engineering office answered.

The constraints we worked inside

  • Safety-critical content: the answer must cite the exact current revision — a stale answer is worse than no answer.
  • Shop-floor devices are shared tablets with gloves-on use — the interface had to be one-tap simple.
  • The handbook updates quarterly; maintenance had to be a checklist, not a project.

What had been tried before

Key procedures were printed and taped to the machines at supervisors' initiative.
Paper freezes one revision in place; within a quarter the taped sheets contradicted the handbook and nobody knew which to trust.
A training module walking new hires through the handbook was added to induction.
Induction covers the structure, not the moment of need; three weeks later the same questions returned to the same supervisors.

What we proposed

We proposed an assistant grounded in the handbook itself, indexed per revision so every answer cites which revision it read from — a superseded answer is visibly stale, which matters more than fluency in a factory. The interface is one search box, large text, and read-aloud for the noisy bays, designed on the shop floor with gloves on. The quarterly update becomes a checklist: ingest the new PDF, run the 30-question eval set, review the drift report, go live. Nothing replaced the handbook; it finally became consultable at the moment of need.

Just as important is what we ruled out, and why:

  • A document management system rolloutThe shop floor needs answers in seconds between machine cycles, not a library to browse; structure was already handled by the handbook's own revisions.
  • Re-printing controlled binders per revisionQuarterly updates guarantee stale paper somewhere on the floor, and a stale safety answer is worse than no answer.
  • An off-the-shelf FAQ bot over the raw PDFWithout revision awareness it would happily quote the superseded section — the one failure mode this factory cannot afford.

How the work ran

01Ingest by revision, cite by revision

The handbook is indexed per revision; every answer states which revision it read from, so a superseded answer is visibly stale.

02Build for gloves and noise

One search box, large-text answers, read-aloud for noisy areas — designed on the shop floor, not in a meeting room.

03Make the quarterly update boring

A revision checklist ingests the new PDF, runs the eval set (30 known questions with expected answers), and flags drift before go-live.

Delivered by the systems pod — automation specialist over 6 weeks, with working increments reviewed with the client every week.

The stack, and the reasoning

LLM assistant platform
Selected for citation discipline — the platform can surface source references per answer, which is the difference between an answer and a liability.
RAG over revisioned handbook
The handbook indexes per revision and the index is checked on every quarterly update, so an answer from a superseded revision announces itself instead of posing as current.
Tablet UI
One search box, large text, read-aloud for the noisy bays — designed on the shop floor with gloves on, not in a meeting room.
Quarterly eval checklist
Each revision runs 30 known questions with expected answers before go-live, so an update that breaks answers is caught by the checklist, not by a shift.

What went wrong

Obstacle

Revision headers were prose, not structured fields — 'revised March, supersedes earlier' — so the first ingestion indexed answers without revision numbers and the pilot quietly quoted superseded settings.

Handled: We re-ingested with structured header parsing, added a header-presence check to the quarterly checklist, and re-ran the eval set to confirm every answer cites a revision.

Obstacle

The shared tablets' browser cached aggressively, and the first week saw two tablets serving the previous quarter's interface after the update.

Handled: The quarterly checklist gained a cache-clearing step with a visible build stamp, so any device serving stale content identifies itself immediately.

How we worked together

Cadence
Weekly sessions on the factory floor itself — supervisors and the training lead testing with real questions between machine cycles — plus a summary after each.
Client side
The training lead owned the pilot cohort of new hires; the engineering office owned revisions; two supervisors tested the interface during live shifts, gloves and all.
Decisions
Safety-adjacent wording required the engineering office's sign-off; everything else was settled on the floor the same day it came up.
They provided
The handbook with its revision history, supervised tablet time during shifts, and the 30 known questions the engineering office wrote for the eval.

What changed

The headline: average time to find a procedure or spec, observed across 50 shift interactions6.5 → 1.5 min, read from Supervisor time study. A second check: answers served from superseded revisions since launch at 0.

Supervisors got their mornings back — the constant interruptions thinned out, and the questions that do reach them are the ones genuinely needing judgment. New hires stopped apologizing for asking, because the tablet never makes them feel like a burden. The engineering office trusts the answers because every one carries its revision, and the quarterly update has become a checklist afternoon instead of a re-launch. Two supervisors now flag handbook sections they think need rewriting, and the afternoon-long stoppage a rare procedure used to cause hasn't happened since launch — the engineering office mentions it before we do.

The result was read from Supervisor time study against the pre-engagement baseline over the stated window, with a guardrail check on answers served from superseded revisions since launch. Where platform-reported numbers and business outcomes differ, this record says which layer it is quoting.

What they own now

  • The assistant indexed against the engineering handbook, with the revisioned index as delivered.
  • The quarterly update checklist, from PDF ingest through eval to go-live, including the workspace credentials.
  • The 30-question eval set with expected answers, owned by the engineering office.
  • The tablet UI configuration, including the cache-clearing and build-stamp steps.
  • A supervisor briefing pack for introducing the tool to each new cohort.

What we would do differently

We would have digitized the revision headers properly at ingest — the first week's answers cited revision dates in prose because the headers weren't structured.

AI & AutomationAI Chatbots & AssistantsManufacturingLLM assistant platform

Next case study

A wholesale distributor cut invoice processing from 45 minutes to 4 per batch