Weaviate, Milvus, and Knowledge-Graph RAG

Beyond Pinecone: Why Alternatives Matter

Pinecone is a leading managed vector databaseVector DatabaseA specialized database optimized for storing and querying high-dimensional vectors (embeddings). Enables fast similarity search across millions of documents for RAG and recommendation systems.See glossary, but it is not the only option -- and for some banking institutions, it may not be the right one. Banks with strict data residency requirements, existing on-premises infrastructure mandates, or specific technical needs may find better alignment elsewhere.

This unit covers two open-source vector databases and one retrieval approach: Weaviate (mature hybrid search and flexible deployment), Milvus (open-source scale), and knowledge-graph retrieval, often called GraphRAG. A third open-source database, Qdrant, now commonly appears on bank shortlists alongside Weaviate and Milvus; its "Hybrid Cloud" option runs a vendor-managed service on the bank's own Kubernetes, on-premises or in any cloud.

Weaviate: Mature Hybrid Search and Flexible Deployment

What Makes Weaviate Different

Weaviate has long been known for its native hybrid search -- the ability to combine semantic searchSemantic SearchSearch that understands meaning rather than just matching keywords. Uses embeddings to find conceptually similar documents even when they use different terminology.See glossary (finding conceptually similar documents) with keyword search (finding documents containing specific terms) in a single query. This is not two separate searches stitched together; it is a unified search that blends both signals to produce better results.

Hybrid search is no longer unique, however. Pinecone, Milvus, Qdrant, Elastic and OpenSearch, MongoDB and PostgreSQL all offer it today. Weaviate's strengths are a mature implementation, flexible deployment, and its Query Agent, an agentic search layer that plans searches on the user's behalf (agentic retrieval is covered in the Advanced RAG Patterns unit).

Why Hybrid Search Matters for Banking

Consider a compliance officer searching for guidance on "BSA/AML customer due diligence for politically exposed persons." A pure semantic search might find documents about customer screening, risk-based approaches, and enhanced due diligence -- conceptually related, but potentially missing documents that specifically mention "PEP" or "politically exposed persons." A pure keyword search would find documents with those exact terms but miss relevant guidance that uses different terminology like "senior foreign political figure."

Hybrid search delivers both: documents that are semantically related to the concept and documents that contain the specific regulatory terms. For banking, where precise regulatory terminology coexists with varied business language, this combination produces meaningfully better retrieval.

Deployment Flexibility

Weaviate can run as:

• A managed cloud service (Weaviate Cloud) for teams wanting operational simplicity, with shared and dedicated options
• A self-hosted deployment using Docker or Kubernetes for institutions with on-premises requirements
• An embedded database within your application for lightweight use cases

This flexibility suits banks with mixed data classifications: internal policy documents might be indexed in a self-hosted Weaviate instance inside the bank's data center, while publicly available regulatory guidance could be indexed in the cloud-hosted version.

Milvus: Open-Source Scale

What Makes Milvus Different

Milvus is a fully open-source vector database designed for massive scale. It is architected to handle billions of vectors across distributed clusters, and it is the vector database you evaluate when your data volume is enterprise-grade and your infrastructure team has the expertise to operate distributed systems.

Milvus continues to move quickly. Version 2.6 added BM25 full-text search, so it now supports hybrid search too, and went generally available on Zilliz Cloud in January 2026. Milvus 3.0, announced in July 2026, is "lake-native": it can search vectors stored in data-lake formats and supports richer ranking and multi-vector retrieval.

Key Capabilities

Distributed architecture. Milvus separates storage and compute, allowing independent scaling of each. When query volume increases, you add compute nodes. When data volume grows, you add storage. This architectural flexibility matters at banking scale.

Multiple index types. Milvus supports a wide range of indexing algorithms (IVF, HNSW, SCANN, DiskANN), each optimized for different trade-offs between search accuracy, speed, and memory usage. Your engineering team can tune the index type to your specific performance and accuracy requirements.

On-premises deployment. As a fully open-source project, Milvus can run entirely within your data center with no data leaving your network. For banks with data classification policies that prohibit cloud-hosted storage for certain document types, this is a requirement, not a preference.

Managed options. Zilliz Cloud offers managed Milvus, including a bring-your-own-cloud option that runs inside the bank's cloud account -- a way to get Milvus without operating it yourself.

Cost at scale. At very large embeddingEmbeddingsNumerical representations (vectors) of text that capture semantic meaning. Similar concepts produce vectors that are close together, enabling machines to understand relationships between words, sentences, or documents.See glossary volumes, self-hosted Milvus can be significantly less expensive than managed services. The trade-off is the operational expertise required to manage a distributed database.

Banking Considerations

Milvus is the right choice when:

• Your vector data volume is in the hundreds of millions or billions
• You have infrastructure teams experienced with distributed systems (Kubernetes, distributed storage), or you use the managed Zilliz option
• Data residency requirements mandate on-premises deployment with no exceptions
• Cost optimization at large scale is a priority

It is not the right choice when:

• You are in the early stages and need to move quickly (self-hosting carries significant operational overhead)
• Your team lacks distributed systems expertise and a managed option is not acceptable
• Your data volume is in the low millions (simpler solutions will serve you well)

Knowledge-Graph RAG (GraphRAG)

A Different Approach to Retrieval

Knowledge-graph retrieval takes a fundamentally different approach. Instead of treating documents as isolated chunks with embedding vectors, it builds a knowledge graph -- a structured map of entities (customers, accounts, exposures), the relationships between them, and the facts extracted from your documents -- and combines that graph with vector search. This pattern is often called GraphRAG. Common ways to build it include a graph database with vector search, such as Neo4j, or Microsoft's open-source GraphRAG project.

Why Knowledge Graphs Matter

Traditional RAG retrieves document chunks that are semantically similar to a question. But some questions require understanding relationships between entities, not just similarity between text passages.

Consider: "Which of our commercial loan customers also have deposit relationships, and what is their total combined exposure?" This question requires understanding entity relationships (customer to loan, customer to deposit, customer to exposure amount) that are poorly served by vector similarity search alone.

Knowledge graphs excel here because they explicitly model entities and relationships; combined with vector search, they enable both similarity-based and relationship-based retrieval.

Banking Relevance -- and the Costs

Knowledge graphs align well with banking's inherently relational data: customers have accounts, accounts have products, products have terms, loans have collateral, entities have beneficial owners, and so on. A RAG system enhanced with a knowledge graph can answer questions that span these relationships -- something pure vector search struggles with.

Two caveats. Building the graph typically takes several times more LLM processing than plain vector indexing, because the model has to read every document to extract entities and relationships. And automated entity extraction is imperfect -- it will merge two customers with similar names or miss a guarantor -- so the graph needs the same data-quality controls you would apply to any customer master data.

Comparison at a Glance