Does PageIndex earn its cost on financial filings?
Accuracy against cost per question
Each dot is one method with one model setting. Up is more accurate, left is cheaper (log scale). Hover a dot for details.
What this shows
- Agentic search is the cheap, accurate cluster. With gpt-6-luna at high effort it ties PageIndex for the best accuracy (83%) at $0.0008 per question versus $0.0128, about 16× cheaper, with no index to build.
- Vector RAG is the cheapest and refuses the most. About $0.0005 per question, but 13–20% of answers are "cannot find": the 10 closest chunks often miss the table the question needs.
- PageIndex is not cheap per question. With Gemini it cost $0.12–0.31 per question, more than putting the whole filing in the prompt (~$0.10). Each turn of its tree search re-sends the conversation so far, and some searches run long (one question took 64 model calls in the pilot). The averages include those outliers.
- Thinking helps cheap models cheaply. gpt-6-luna gains 7–17 points from no thinking to high effort, for a fraction of a cent.
- Read the gaps with care. 30 questions, so one question moves accuracy by 3.3 points; differences under ~10 points are within noise. One judge model, strict grading against FinanceBench's gold answers, and each question asked against its own filing (finding the right filing among many is not tested).
All questions: how each method did
| # | Company | Question | Gold answer | Whole filling | PageIndex | Vector RAG | Agentic search |
|---|
Methods
- : the whole filing's text goes into the prompt. The brute-force baseline.
- : the PageIndex SDK (local mode, Flash tree index). The model walks a table-of-contents tree of the filing and reads the pages it picks.
- : the filing is split into ~500-token chunks and embedded once; the 10 chunks closest to the question are given to the model.
- : no index. The model gets two tools, a regex search over the filing and "read pages", and explores like a coding agent.
Table
| Method | Setting | Accuracy | Refused | Errors | $ / question | Median s |
|---|