Use hierarchical chunking with table‑aware tokenizers and a hybrid retrieval pipeline that combines dense vector search with structural metadata filters.
Step‑by‑step solution
1. Extract raw layout – Run pdfplumber (v0.10+) or pymupdf to get bounding boxes for cells, rows, and column headers. Save as JSON: {page, table_id, row, col, bbox, text}.
2. Create semantic sub‑chunks – For each row, concatenate cell texts preserving header context, then split with tiktoken using a max‑tokens=256 limit. Prefix each chunk with a synthetic header string "Table {table_id} Row {row}" to retain semantics.
3. Generate embeddings – Use openai embeddings (text-embedding-3-large) or a local Mistral‑Embed (mistral-embed-v0.2) with normalize=True. Store vectors in a Qdrant collection with payload containing table_id, row, col_range.
4. Hybrid index – Enable Qdrant’s filter on payload.table_id and payload.col_range while performing a search with ef=128 and top_k=10. Combine scores: final_score = 0.7vector_score + 0.3metadata_match.
5. RAG prompt assembly – Retrieve top‑k chunks, then reconstruct the original row by joining chunks that share the same table_id and adjacent row numbers. Feed the assembled snippet to the LLM with a system prompt that defines the table schema.
Quick comparison
| Method | Token limit | Structure awareness | Retrieval latency (ms) |
|--------|-------------|--------------------|------------------------|
| Flat chunking | 4 k | ❌ | 45 |
| Hierarchical (this) | 256 per chunk | ✅ | 62 |
| Hybrid vector+SQL | 4 k | ✅ (via filter) | 78 |
Gotcha: Qdrant’s payload filters are not applied to vectors generated with normalize=False; always enable normalization or pre‑normalize embeddings, otherwise the hybrid score will be skewed.