I'm building a RAG pipeline where the source documents (PDFs) contain both text and images (diagrams/figures), and I need a chunking strategy that handles both properly.
What I've done so far:
Extracting images from PDFs separately from the text (so my text splitter never touches image data directly)
Considering Google's multimodal embedding model to embed images and text into the same vector space
Using semantic chunking for the text content
Where I'm stuck:
Text splitters (e.g., LangChain's chunking utilities) only operate on plain text — they have no way to recognize or preserve the position of an image within a document. Since text and images are processed through separate pipelines, I'm unsure of the best way to:
Keep an image linked to its relevant surrounding text (so retrieval doesn't return an image with no context, or vice versa)
Decide whether images should be their own standalone chunks, or bundled with nearby text as a single retrievable unit
Store/retrieve both types together in the same vector database without losing the association between them
Question: What's a reliable pattern for chunking mixed text+image documents so that related content stays linked, while still allowing each modality to be embedded and retrieved on its own terms? Are there established approaches (e.g., placeholder markers, metadata linking, hierarchical chunking) that people use in production RAG systems for this?