There’s a gap between how LLMs are marketed to business analysts and how they’re actually useful in practice. Having used them extensively — in my own analysis work, in building the financial document pipeline, and in guiding 40+ student teams through data projects — I’ve developed a fairly clear view of where they genuinely add value and where they don’t.
Where LLMs Are Genuinely Useful
Unstructured data extraction. This is where LLMs shine without qualification. If you have PDFs, emails, call transcripts, survey responses, or any text that needs to be converted into structured data, LLMs are dramatically faster than any previous approach. The key is a strict output schema (Pydantic, JSON Schema) and a validation layer — never trust the output without verification.
Initial data profiling. Uploading a dataset and asking “what’s unusual here?” or “what relationships should I investigate?” produces genuinely useful starting points. Not conclusions — starting points. The analyst still needs to verify and interpret.
Code generation for repetitive transformations. Writing Pandas boilerplate, SQL window functions, or Power Query M code is faster with LLM assistance. The risk here is subtle: LLMs generate plausible-looking code that can have logic errors, especially with complex groupby operations. Always test on a subset with a known expected output.
Writing and communication. Turning analysis into a stakeholder-ready narrative is real work. LLMs can draft first versions of executive summaries, interpretation paragraphs, and presentation scripts — which you then edit for accuracy and tone.
Where They’re Overrated
Causal reasoning. LLMs are very good at describing correlations in familiar language. They are not good at causal inference, and they’ll confidently generate plausible-sounding causal stories that aren’t supported by the data. For anything involving “why”, you need to do the analytical work yourself.
Domain-specific accuracy without grounding. In specialist domains — Iranian financial regulations, specific industry accounting standards, niche technical terminology — LLMs will fill knowledge gaps with confident-sounding errors. RAG (Retrieval-Augmented Generation) with verified sources partially solves this, but it’s not automatic.
Replacing the framing conversation. The most valuable part of a business analysis engagement is the conversation that clarifies what question is actually being asked. LLMs can’t have that conversation with a stakeholder. That skill remains fully human.
A Practical Framework
When I introduce LLMs to student teams or consulting clients, I use a simple filter: use LLMs to do things faster, not to skip things entirely. Data cleaning faster, code drafting faster, report writing faster — but the verification, the interpretation, and the framing are still yours.
The analysts who benefit most from LLMs are the ones who already understand the craft well enough to know when the output is wrong. If you’re relying on the LLM to tell you what’s correct, you’ve outsourced your judgement, and that’s where things go wrong.