Model Online / On-Site
Duration
Modules 6 modules

Course Objective & Scope

At millions of document chunks and high query concurrency, search recall, tail latency and infrastructure cost depend directly on the underlying vector index and database architecture. This course provides a rigorous comparison of Approximate Nearest Neighbour (ANN) algorithms (Flat, IVF, HNSW, DiskANN, Product/Scalar Quantization) and leading engines (PostgreSQL `pgvector`/`pgvectorscale`, Milvus, Weaviate, FAISS and Pinecone).

Participants learn when staying on PostgreSQL with `pgvector` is best versus adopting a dedicated vector database, and work hands-on with filtered ANN search, index mutations, multi-tenancy and RAM/NVMe capacity sizing.

Who Should Attend?

  • AI, data and platform engineers building large-scale RAG and semantic search infrastructure
  • Solution and data architects selecting vector storage for the enterprise data stack
  • Database administrators and SREs responsible for PostgreSQL and search performance
  • Prerequisite: database fundamentals, SQL/Python skills and basic familiarity with embeddings

Key Takeaways

  • Tune HNSW, IVF-PQ and disk-based (DiskANN) indexes across recall, QPS and RAM/disk trade-offs
  • Choose between `pgvector`, Milvus, Weaviate, FAISS and Pinecone based on scale and ops model
  • Solve low-recall pitfalls in filtered vector search using iterative scans and hybrid index layouts
  • Design multi-tenant vector storage using namespaces, partitions and tenant promotion strategies
  • Run benchmarks measuring index build time, p99 latency and Recall@k under concurrent load

Curriculum

ANN Indexing Algorithms: IVF, HNSW, PQ and DiskANN
  • From exact k-NN to approximate search; tuning HNSW (`M`, `efConstruction`, `efSearch`) and IVF (`nlist`, `nprobe`)
  • Product Quantization (PQ), Scalar Quantization and SSD-backed DiskANN/StreamingDiskANN
Comparing Libraries, Extensions and Dedicated Vector Databases
  • FAISS (embedded library), PostgreSQL `pgvector`/`pgvectorscale` (transactional SQL) and Pinecone (serverless)
  • Distributed architectures in Milvus and Weaviate: decoupling storage and compute
Production Architectures with PostgreSQL pgvector
  • HNSW and IVFFlat/DiskANN index builds, `halfvec`/`bit` types and maintenance memory tuning
  • Handling relational JOINs and selective WHERE clauses with `iterative_scan`
Distributed Vector Databases: Milvus, Weaviate and Pinecone
  • Collection schemas, sparse+dense hybrid fields, sharding, replication and consistency levels
  • Multi-tenancy patterns: partition keys, namespaces and hot/cold tiering
Index Lifecycle, Capacity Planning and Operations
  • Managing high write/delete churn, compaction and zero-downtime re-indexing
  • Sizing RAM, NVMe IOPS and cloud cost from vector count, dimensions and quantization
Lab: Vector Database Benchmark and Selection Matrix
  • Benchmarking QPS, Recall@k and p99 latency across `pgvector`, Milvus/Weaviate and FAISS
  • Producing a justified vector database decision report for the organisation

Other Courses in This Category

  • Data Preparation, Document Parsing and Chunking for RAG

    Parsing complex PDFs, tables and scanned enterprise documents with Docling, Unstructured and VLMs, and applying structural, semantic, parent-child and contextual chunking for high-recall RAG.

    RAG & Enterprise Knowledge Systems Corporate
    Live Course
    6 modules
    View
  • Citations, Grounding and Hallucination Control in RAG Systems

    Engineering RAG systems for strict grounding, verifiable inline citations, conflict resolution across sources, calibrated abstention when evidence is missing and automated faithfulness verification.

    RAG & Enterprise Knowledge Systems Corporate
    Live Course
    6 modules
    View
  • Advanced RAG Advanced RAG

    Enterprise RAG architectures covering BM25-dense hybrid retrieval, ColBERT, neural reranking, self-correcting Agentic RAG, GraphRAG, Text-to-SQL, and GPU-accelerated serving.

    RAG & Enterprise Knowledge Systems Corporate, Individual
    Live Course
    10 modules
    View
  • Hybrid Search and Reranking for High-Precision RAG

    Boosting RAG retrieval precision and recall by combining BM25/SPLADE lexical search with dense vectors via RRF, query rewriting (multi-query, HyDE) and cross-encoder or ColBERT reranking.

    RAG & Enterprise Knowledge Systems Corporate
    Live Course
    6 modules
    View
  • Metadata Architecture and Permission-Aware Retrieval in RAG

    Designing metadata schemas, dynamic filter extraction and RBAC/ABAC/ReBAC security-trimmed retrieval so enterprise RAG assistants never leak unauthorized documents.

    RAG & Enterprise Knowledge Systems Corporate
    Live Course
    6 modules
    View
  • Agentic RAG: Multi-Step, Corrective and Adaptive Retrieval Architectures

    Building Agentic RAG systems with LangGraph that plan sub-queries, route across heterogeneous data sources, grade retrieved evidence, self-correct queries (CRAG/Self-RAG) and bound latency.

    RAG & Enterprise Knowledge Systems Corporate
    Live Course
    6 modules
    View

Get in touch