Talk to Your Tables and Images with Multimodal RAG
Price
950 ₺
In today's era of applied AI, organizations struggle to access and extract meaning from comprehensive information enriched not only with text but also with visual and tabular data. While traditional RAG architectures are text-centric, real-world documents (manuals, reports, technical drawings) contain complex visual and tabular structures. This fast-paced 3-hour "Multimodal Retrieval-Augmented Generation (RAG) with LangChain Workshop" offers participants both the theory and practice of implementing a multimodal RAG architecture from scratch.
The workshop covers the core concepts of Multimodal RAG, how to orchestrate different modalities (text, image, table) with LangChain, and aims to build a live, working chatbot capable of understanding complex documents by the end of the day. Participants will experience the end-to-end chain of multimodal data extraction, diverse embedding models, unified vector search, and prompt engineering, directly observing the power of accessing enriched information that goes beyond plain text.
Who Should Attend
Who Should Attend
Data scientists, MLOps engineers, software developers – those looking to build LLM-based applications capable of extracting meaning from complex, multimodal documents.
Technical leaders aiming to bring the practical benefits of Multimodal RAG architecture to their organization.
Professionals seeking tangible outputs in a short time through a learn-build-demo cycle.
Delivering intensive learning within a focused timeframe, this workshop will rapidly advance participants on the RAG journey from "multimodal documents to actionable intelligence" and ensure you leave with a live-demoed project built on real data.
Curriculum
Introduction to Multimodal RAG
Why Multimodal RAG?
Core RAG recap
Uncovering insights from documents (manuals, reports, pages with diagrams)
Workshop goal: Building a chatbot that understands text, images, and tables
Deconstructing the PDF: Beyond Plain Text
Understanding PDF structure: text streams, image objects, vector graphics, and embedded table data
PDF parsing tools
Data extraction challenges for each modality
Multimodal RAG Workflow
Distinct processing pipelines for text, images, and tables
Modality-specific embedding models vs. unified multimodal embeddings
Storing and retrieving multimodal information
Presenting combined context to the LLM
Extracting and Embedding Images and Tables
Image Extraction and Embedding
Extracting images from a sample PDF
Introduction to visual embedding models (CLIP, OpenAI models, etc.)
Hands-on: Generating embeddings for extracted images
Storing image embeddings and associated metadata (source page, image ID)
Table Extraction & Representation for RAG
Extracting tables from documents
Strategies for representing tables:
Embedding as structured text (Markdown, CSV string)
Embedding as an image for complex/graphical tables
Brief discussion on using an LLM to summarize tables concisely
Hands-on: Converting an extracted table to Markdown and generating embeddings
Text Extraction & Embedding
Brief recap of text chunking and embedding
Ensuring metadata consistency across modalities (such as page numbers)
Multimodal Vector Store & Retrieval Strategy
Vector Store Design
Option 1: A single vector store with metadata distinguishing modalities
Option 2: Separate vector stores for each modality
Pros and cons (simplicity vs. specialized search)
Hands-on: Setting up a Vector Database storing text, image embeddings, and table representations with metadata
Multimodal Retrieval: Finding the Right Pieces
Cross-modality querying:
Simple approach: Querying all relevant vector stores/indexes and merging results
LangChain's MultiVectorRetriever concept
Retrieving relevant text chunks, image references/descriptions, and table snippets
Merging and ranking results: weighting scores from different sources or simply combining top-k from each modality
Hands-on: Implementing a retrieval function that fetches top-k text, top-k image references, and top-k table snippets for a user query
Prompting for Multimodal RAG
Representing Multimodal Context in Prompts
How to "show" images to an LLM when not using a vision-capable LLM: image descriptions, captions, or placeholders (e.g., [Image: chart_on_page_5.png ...])
Integrating tabular data (Markdown) directly into the prompt
Prompt structures that guide the LLM to synthesize all context types
Effective Prompts for Multimodal Q&A
"Based on the text and the provided [Image: diagram_1], explain how X works."
"Summarize the key findings from [Table: financial_summary] and the surrounding text."
"What does the chart on page 7 show about user growth?" (Mapping page → image)
Iterative prompt refinement for desired outputs
Building the UI with Gradio
Ingesting Data into the System:
User uploads a PDF
Data Extraction (text, image, table)
Embedding generation
Storing in the vector store
Retrieval & Generation:
Receive user query
Perform multimodal retrieval
Construct the prompt with retrieved context
Call the LLM
Display the response
Live Demo & Testing
Testing the application with a sample PDF containing text, charts (images), and tables
Asking questions that require synthesizing information across modalities
Debugging
Wrap-Up
Summary of what was accomplished
Q&A
Other Courses in This Category
-
Hands-on fundamentals for building an end-to-end RAG system over enterprise documents: ingestion, chunking, embeddings, vector retrieval, grounded prompting with citations and baseline evaluation.
- Live Course
- 6 modules
-
Deep dive into dense, sparse (SPLADE/BM25), Matryoshka and late-interaction (ColBERT) embeddings, Turkish/multilingual retrieval quality, domain adaptation and semantic search engineering.
- Live Course
- 6 modules
-
Engineering deep dive into ANN indexing (HNSW, IVF, DiskANN, PQ) and vector databases (pgvector, Milvus, Weaviate, FAISS, Pinecone), covering filtered search, multi-tenancy and capacity planning.
- Live Course
- 6 modules
-
Parsing complex PDFs, tables and scanned enterprise documents with Docling, Unstructured and VLMs, and applying structural, semantic, parent-child and contextual chunking for high-recall RAG.
- Live Course
- 6 modules
-
Engineering RAG systems for strict grounding, verifiable inline citations, conflict resolution across sources, calibrated abstention when evidence is missing and automated faithfulness verification.
- Live Course
- 6 modules
-
Advanced RAG
Enterprise RAG architectures covering BM25-dense hybrid retrieval, ColBERT, neural reranking, self-correcting Agentic RAG, GraphRAG, Text-to-SQL, and GPU-accelerated serving.
- Live Course
- 10 modules
Get in touch
