Embeddings and Vector Search — The "Memory" Layer

Beyond Keyword Search: Finding Information by Meaning

Every bank sits on a vast repository of documents -- regulatory filings, credit memos, policy manuals, customer correspondence, audit findings, and board presentations. Today, finding specific information in this repository typically means keyword search: type "Basel III capital requirements" and hope the exact phrase appears in the document you need.

But what if you could search by meaning instead of exact words? What if your system could find all documents discussing capital adequacy -- even those that never use the phrase "Basel III"? That is exactly what embeddingsEmbeddingsNumerical representations (vectors) of text that capture semantic meaning. Similar concepts produce vectors that are close together, enabling machines to understand relationships between words, sentences, or documents.See glossary and vector search make possible.

This technology is not theoretical. It is the foundation of the most powerful enterprise AI applications being deployed in banking today, and understanding it will help you evaluate AI vendors, assess architecture proposals, and make informed investment decisions.

What Are Embeddings?

At its core, an embedding is a way to convert text into numbers that capture meaning. When you run a sentence, paragraph, or entire document through an embedding model, you get back a list of numbers -- typically 768 to 3,072 numbers -- called a vector. These numbers represent the semantic content of that text in a mathematical space.

The critical insight is this: texts with similar meanings produce vectors that are close together in this mathematical space. A document about "mortgage delinquency trends" and one about "home loan default patterns" would produce vectors that are nearly identical -- even though they share few keywords.

How Vector Search Works

Once you have converted your documents into embeddings, you can search them by meaning using semantic searchSemantic SearchSearch that understands meaning rather than just matching keywords. Uses embeddings to find conceptually similar documents even when they use different terminology.See glossary (also called vector search or similarity search). The process works in three steps:

1. Embed the query: When a user asks a question -- say, "What is our policy on concentrated commercial real estate exposure?" -- the system converts this question into an embedding vector using the same model
2. Find nearest neighbors: The system compares the query vector against all stored document vectors and finds the closest matches using mathematical distance calculations (typically cosine similarity)
3. Return ranked results: The most semantically similar documents are returned, ranked by relevance -- regardless of whether they contain the exact keywords from the query

This approach solves a fundamental problem in banking: institutional knowledge is spread across thousands of documents using inconsistent terminology. Different departments, different eras, and different authors describe the same concepts in different ways. Keyword search fails when terminology varies. Semantic search succeeds because it understands meaning.

Why This Matters for Banking

Consider a compliance officer searching for all internal policies related to "customer due diligence." With keyword search, they might miss documents titled "KYC Procedures," "Client Onboarding Standards," or "Enhanced Verification Protocols" -- all of which address the same regulatory requirement. With embedding-based search, all of these documents surface because their meanings are similar, regardless of the specific words used.

Vector Databases: Storing and Searching at Scale

A vector databaseVector DatabaseA specialized database optimized for storing and querying high-dimensional vectors (embeddings). Enables fast similarity search across millions of documents for RAG and recommendation systems.See glossary is a specialized database designed to store embedding vectors and perform fast similarity searches across millions or billions of entries. Traditional relational databases (the kind powering your core banking systems) are optimized for exact matches -- "find the account with ID 12345." Vector databases are optimized for approximate nearest-neighbor searches -- "find the 10 documents most similar to this query."

Leading vector database options include:

• Pinecone: Fully managed cloud service, simple to deploy, popular for proof-of-concept projects
• Weaviate: Open-source with enterprise features, supports hybrid search (combining keyword and vector approaches)
• pgvector: An extension for PostgreSQL that adds vector search capabilities to your existing database infrastructure
• Qdrant: High-performance open-source option with strong filtering capabilities

For banking institutions, the choice of vector database involves familiar enterprise considerations: data residency, security, scalability, and integration with existing infrastructure. Many banks start with pgvector because it adds vector capabilities to PostgreSQL databases they already operate and secure.

The Embedding Pipeline

Deploying embeddings in a banking context follows a structured pipeline:

1. Document ingestion: Collect and preprocess documents from internal repositories, regulatory feeds, and knowledge bases
2. Chunking: Split large documents into smaller, overlapping segments. A 200-page regulatory filing might be split into 500-word chunks with 50-word overlaps to preserve context at boundaries
3. Embedding generation: Run each chunk through an embedding model to produce its vector representation
4. Storage: Store the vectors alongside metadata (source document, date, department, classification level) in a vector database
5. Query processing: When a user searches, embed their query and find the nearest document chunks

Real Banking Applications

Embeddings power several high-value banking use cases:

• Intelligent policy search: Relationship managers find relevant policies instantly, even when they do not know the exact document title or regulatory reference
• Regulatory change detection: Automatically match new regulatory guidance to affected internal policies by comparing embeddings of new regulations against your policy library
• Customer communication analysis: Cluster customer complaints by semantic theme to identify emerging issues before they become systemic
• Due diligence research: Search across thousands of news articles, filings, and reports to surface relevant information about counterparties or market segments