[ Case study ]
Support answered English properly and three other languages badly — machine-translated replies from staff, inconsistent tone, and twice the handling time. The knowledge base was English-only, so even good translations drifted from policy.
CLIENT a boutique travel platform — FOCUS Keep English the source of truth
Representative examplesEvery case study in this library is an illustrative composite of the kind of engagement we deliver — written to show our method and standards, not to name clients.
Travel booking is emotional, international, and occasionally sarcastic — the platform sells curated trips to customers who write casually in four languages and expect replies that sound native, not translated. Support is a small team of generalists; the knowledge base documenting booking changes, refunds, and cancellation windows exists in English only, because English is where the policy wording is maintained. Anything a customer reads in another language is, one way or another, derived from that English source.
Support answered English properly and three other languages badly — machine-translated replies from staff, inconsistent tone, and twice the handling time. The knowledge base was English-only, so even good translations drifted from policy.
We proposed keeping English as the single source of truth: answers compose from the English knowledge base and translation runs as an output step, so policy wording never forks per language. Before composing, every message is classified — booking change, refund, information, complaint — because intent decides the reply's shape, and sarcasm or anger routes to a human rather than meeting a cheerful template. A bilingual eval set of policy questions in all four languages runs against every pipeline change, so a model update cannot silently degrade French while everything looks fine in the demo.
Just as important is what we ruled out, and why:
Answers generate from the English knowledge base with translation as an output step, so policy wording never forks per language.
Each message is classified (booking change, refund, information, complaint) against policy rules before any reply is composed — sarcasm and edge cases route to humans.
A bilingual eval set (policy questions in all four languages) runs against every pipeline change, so a model update can't silently degrade French.
Delivered by the systems pod — engineer over 7 weeks, with working increments reviewed with the client every week.
Obstacle
The first intent classifier handled the four planned classes until a sarcastic refund request arrived phrased as praise, and the reply cheerfully recited policy to an already furious customer.
Handled: We added a complaint-intent tier above the others, mined the ticket archive for its phrasings, and wrote those exact cases into the eval set.
Obstacle
Language quality varied more than expected across the four — the model handled two languages comfortably and needed different prompting conventions for the others.
Handled: Each language got its own prompt profile tuned against the bilingual eval set, with per-language thresholds gating anything that auto-sends.
The headline: average support handling time across non-english tickets, first full month — −41%, read from Support desk metrics. A second check: policy-consistency spot-checks passing across all four languages at 98%.
Non-English customers stopped feeling like second-class users: replies arrive in register, on time, and consistent with what an English customer would have been told. The support team's relationship with the other languages changed from anxiety to confidence — they answer in English and trust the pipeline with the rest. Complaints that once escalated over tone now resolve on the first reply often enough that the manager checks the classifier before checking the staff. The English knowledge base, suddenly load-bearing for four languages, got the maintenance it had always deserved.
The result was read from Support desk metrics against the pre-engagement baseline over the stated window, with a guardrail check on policy-consistency spot-checks passing across all four languages. Where platform-reported numbers and business outcomes differ, this record says which layer it is quoting.
What we would do differently
We would have added the complaint-intent class in v1 — two sarcastic refund requests confused the first classifier and taught us the taxonomy was missing a tier.
[ Related service ]
[ Related builds ]
45 min 4 minAverage handling time per invoice batch (human review only), measured over the first full month
44% 71%Tier-1 tickets resolved without human touch, month two post-launch versus month before
[ Next step ]
Next case study