Architecture
The pipeline matters more than the model.
Most of the return comes from pipeline design, not from the largest model. We combine document processing, classifiers, rules, retrieval, generation and bounded tool-calling, so the expensive step runs only when cheaper ones cannot finish the job.
The reference path, from document to system of record.
Any stage can be skipped when the one before it already has the answer.
Document or event. An email, PDF, photo, form or system event arrives.
The diagram as text:
- Document or event. An email, PDF, photo, form or system event arrives.
- OCR / IDP. Reads the page into typed fields: names, dates, amounts, policy and loan numbers.
- Classifier + rules. Routes by document type and completeness. Known cases are settled here without a language model.
- Retrieval (RAG). Pulls the relevant policy clause or procedure, with a citation back to the source.
- LLM. Handles what is left: messy narratives, summaries and draft notes.
- Bounded tools. Changes records only through an approved list of actions, each one logged.
- Human gate. Low-confidence or high-risk items wait for a person before anything is written.
- System of record. The approved result is written to the system you already use.
Items move left to right. Most leave after the classifier and rules stage because a rule or a small model has already settled them. A thin stream reaches the language model. Anything the system is unsure about stops at the human gate until a person approves it, and only then is it written back.
What each stage is for.
Cheap, predictable layers first. Generation only where the text is genuinely open-ended.
OCR / IDP
Structure what is already on the page.
Intelligent document processing turns PDFs, email bodies and images into typed fields such as names, dates, amounts and IDs before any generative call. Validation and schema mapping happen here, so later stages work on data rather than pixels.
Classifier + rules
Route the work. Don’t invent the answer.
Document type, product line, urgency and completeness decide the queue. When the categories are known, a closed taxonomy, SLA rules or a small classifier usually beat a general language model. Generation is kept for open-ended text.
Retrieval (RAG)
Fetch the source, then generate.
Retrieval-augmented generation indexes your handbooks, contracts, tickets and procedures, then pulls the relevant passages into the prompt with citations. Compared with pasting long documents into the context window, it uses fewer tokens, invents less and gives answers you can check against the file.
LLM
Judgment, drafts and exception notes.
The model summarizes messy packets, drafts readable notes and explains why a file fails a completeness check. It is sized to the step. It is never the system of record for numbers or IDs, which stay in extracted fields and databases.
Bounded tools
Allowlisted tools, a clear stop and an audit trail.
An agent plans a short sequence, such as opening a record, attaching a file and setting a status, only through allowlisted tools with defined exits and logging. It is not a free-roaming agent that tries to do everything.
Cost follows the shape of the pipeline.
Classifiers and retrieval cut the volume before generation. Tiered models and evaluations keep spend and quality under control.
- Classify before you generate
- Document-type and intent classifiers, or plain rules, route the work and skip model calls on closed categories.
- Retrieve before you lengthen the context
- Pull ranked passages with citations instead of pasting the whole corpus into the prompt just in case.
- Tier the models
- Small, cheap models handle routine classification and extraction. Large models are kept for hard judgment.
- Cache and batch
- Repeat lookups are cached and overnight backlogs run in batches, instead of fresh generation on every request.
- Route by confidence
- High-confidence paths clear automatically. Low-score or high-risk items wait for a person before write-back.
- Gate releases on evaluations
- Golden sets, citation checks and prompt regression tests run in CI, so quality is a release gate rather than a hope.
Failure modes we design against.
| Failure mode | What we do instead |
|---|---|
| Treating the model as a database | Facts and numbers come from extracted fields and systems of record, never from model memory. |
| Retrieval without citations | Every retrieved passage must be attributable. Uncited retrieval is a quieter form of hallucination. |
| Agents without a tool allowlist | Every tool is named, scoped and logged. Nothing moves across systems unsupervised. |
| Personal data in prompts, no evaluations | Outbound fields are minimized and masked, payloads are logged, and every release ships with evaluations, because prompt drift is a production risk. |