When I started building the AI pipeline for Iranian stock exchange document analysis, one of the first challenges was a deceptively simple question: how do you make an LLM reliably answer questions about a 200-page Persian financial report without hallucinating?
The answer, in short, is RAG — but the details matter enormously.
Why RAG over full-document prompting
Feeding an entire financial report into a single LLM context window is expensive, slow, and often unreliable. For Persian-language documents, it was also practically difficult: token costs scale fast, and most LLMs have weaker instruction-following in Persian at long context lengths.
RAG solves this by splitting the retrieval problem from the generation problem. You index the document, retrieve the relevant chunks for each question, and only then pass a focused, small context to the LLM.
What made this harder in Persian
- OCR quality variability: Scanned PDFs produced inconsistent text, especially for tables and numbers
- Right-to-left layout parsing: Standard chunking strategies break on RTL text structures
- Terminology gaps: Financial and ESG terminology in Persian is not well-represented in most embedding models
What worked
The most reliable approach was a hybrid retrieval strategy: dense semantic search for conceptual questions + keyword BM25 for specific numerical fields. Cross-encoder reranking before passing context to the LLM added another layer of accuracy.
The Pydantic validation layer was the single most important addition to the pipeline — it caught malformed outputs before they entered the dataset, keeping extraction quality high without manual review of every record.
Key takeaway
For domain-specific document extraction, the retrieval architecture matters more than the LLM choice. A well-designed chunking and reranking pipeline with a mid-tier LLM will outperform a poorly-structured pipeline using the best available model.