Case Study — Document Indexing
Index AI
A multi-agent document-indexing system for banking. Documents arrive in combined-scan format; Index AI runs them through OCR, then uses an ensemble of agents to index each page — classifying and segmenting it — with per-page majority voting deciding the final index.
- Role
- Lead AI Engineer
- Duration
- Feb 2025 – Mar 2026
- Status
- In Production
- Domain
- Banking Documents (BFSI)
Project Overview
In banking, large volumes of documents arrive in combined-scan format — many distinct documents fused into a single scanned file. Index AI takes those documents and indexes them into a separate, organized set of files.
Indexing is not the same as classification. Classification only assigns a whole document to a class. Indexing goes further: it classifies and divides the document into the multiple segments that belong to each class. That segmentation is what makes the problem harder — and what Index AI is built to do reliably.
Why it's difficult
The Engineering Problem
The input is a combined scan: several documents concatenated into one file, first legible to a machine only after OCR. The task is not merely “what class is this document?” but “where does each class-relevant segment begin and end, page by page?” A single model pass over noisy OCR output produces an indexing that is plausible but inconsistent — and there is no second opinion to catch where it went wrong.
The engineering problem was to produce an indexing that is reliable at the page level across noisy, combined-scan input — accurate enough to trust as the organized output the rest of the business works from.
My Role
What I owned on this project:
- Architected the indexing pipeline end to end — from OCR ingestion of combined-scan documents through to the final indexed files.
- Designed the multi-agent ensemble: N agents that index each document in parallel, and the per-page majority-voting scheme that resolves their outputs.
- Framed indexing as classification plus segmentation — dividing each document into the segments belonging to each class, not just labeling it.
- Built the evaluation benchmark — a synthetic 'silver' dataset — to measure indexing accuracy.
Organized by engineering problem
Technical Contributions
OCR ingestion of combined scans
Problem
Documents arrive as combined scans — multiple documents fused into one file — and are not machine-readable as received.
Approach
Ran incoming documents through OCR as the first stage, making their text available to the downstream indexing agents.
Outcome
A machine-readable representation of combined-scan input that indexing can operate on.
Ensemble indexing agents
Problem
A single indexing pass over noisy OCR is inconsistent, and indexing requires segmenting each document into class-aligned parts — not just classifying it.
Approach
Ran N agents in parallel, each independently indexing the document by classifying and dividing it into the segments that belong to each class.
Outcome
Multiple independent indexings of the same document, ready to be reconciled into one.
Per-page majority voting
Problem
The ensemble produces several candidate indexings that must be reduced to a single, trustworthy result.
Approach
Applied majority voting per page across the agents' outputs to decide the final indexing.
Outcome
98% accuracy on the evaluation dataset — a synthetic 'silver' dataset created for this purpose.
Reasoning, not just the result
Engineering Decisions
Why indexing rather than classification?
Classification only assigns a document to a class; the business needs the document divided into the segments that belong to each class. Indexing does both — classifying and segmenting — which is why a plain classifier was never sufficient.
Why an ensemble of parallel agents instead of one indexer?
A single pass over noisy combined-scan OCR is inconsistent. Running N agents in parallel produces independent indexings that can be reconciled, so no single agent's error decides the outcome.
Why majority voting per page?
Resolving agreement at the page level is the granularity that matters for indexing. Per-page majority voting turns several imperfect indexings into one consensus result that is more reliable than any single agent.
Why a synthetic 'silver' dataset for evaluation?
Measuring indexing accuracy needs labeled data at scale. A created synthetic silver dataset made systematic evaluation possible, giving a defensible accuracy number to improve against.
High-level view
Architecture
Combined-Scan Documents
Multiple documents fused into one file
OCR
Makes scanned input machine-readable
Parallel Indexing Agents (N)
Each classifies & segments independently
Per-Page Majority Voting
Reconciles agents into one indexing
Indexed Files
Organized output for the business
Challenges & Trade-offs
Challenge
Reliability vs. compute.
Trade-off
Running N agents in parallel and voting per page costs far more compute than a single indexing pass — a deliberate trade of cost for the consensus reliability the ensemble provides.
Challenge
Evaluating without large labeled real data.
Trade-off
A synthetic 'silver' dataset unblocked systematic evaluation, but silver labels approximate ground truth rather than replace it — so the 98% figure is read against that benchmark, not a fully hand-labeled corpus.
How it was measured
Evaluation
- Accuracy measured on a synthetic 'silver' evaluation dataset created for the task.
- Reached 98% indexing accuracy on that dataset.
- Evaluation targeted the page level, matching the per-page majority-voting resolution.
Impact
- Turns combined-scan banking documents into an organized, indexed set of files.
- Delivers reliable indexing — classification plus segmentation — rather than document-level classification alone.
- Reached 98% indexing accuracy on the evaluation benchmark.
Technologies
Pipeline
Agents
Consensus & Evaluation
Languages
Lessons Learned
- Indexing is more than classification: it requires segmenting a document into class-aligned parts, not just labeling the whole.
- An ensemble with per-page majority voting is more reliable than a single indexing pass over noisy OCR.
- A synthetic 'silver' dataset can unblock evaluation when large labeled real data isn't available.
- Indexing quality is bounded upstream by OCR, since the inputs arrive as combined scans.