What Is a Large Language Model (LLM)?

The Technology Reshaping Financial Services

Every decade or so, a technology arrives that fundamentally changes how banks operate. In the 1990s, it was online banking. In the 2010s, it was mobile-first platforms. Today, Large Language ModelsLarge Language Model (LLM)A neural network trained on vast amounts of text data that can understand and generate human language. LLMs power chatbots, document analysis, code generation, and many enterprise AI applications.See glossary represent the next inflection point -- and understanding them is no longer optional for banking leaders.

But here is the good news: you do not need a computer science degree to grasp what LLMs are and why they matter. You need the same analytical framework you already use when evaluating any major technology investment.

How LLMs Actually Work

At the most fundamental level, an LLM is a prediction engine. Given a sequence of words, it predicts what comes next. That sounds simple, but when you train this prediction engine on trillions of words and give it billions of internal parameters to adjust, something remarkable happens: the model develops an apparent understanding of language, logic, and even domain-specific knowledge.

Here is a simplified view of the process:

1. Training: The model reads enormous amounts of text -- regulatory filings, news articles, scientific papers, books, code repositories -- and learns statistical patterns about how language works
2. Parameters: During training, the model adjusts billions of internal weights (think of these as dials that get fine-tuned). GPT-4 is estimated to have over a trillion parameters
3. InferenceInferenceThe process of running a trained model to generate predictions or outputs from new input data. Inference cost, latency, and throughput are key factors in enterprise AI deployment.See glossary: When you ask the model a question, it uses those learned patterns to generate a response, one word (technically, one "tokenTokensThe basic units of text that LLMs process. A token is roughly 3/4 of an English word. Token counts determine cost, speed, and context window limits for every API call.See glossary") at a time
4. Context windowContext WindowThe maximum amount of text (measured in tokens) a model can process in a single request. Larger context windows allow more information but increase cost and latency.See glossary: The model can consider a fixed amount of text at once -- modern models can process 100,000+ tokens, equivalent to a 300-page document

What Makes LLMs Different from Traditional AI

Traditional AI in banking -- the kind powering your credit scoring models and fraud detection systems -- is narrow and task-specific. A fraud detection model does one thing exceptionally well. An LLM, by contrast, is a generalist. The same model that can summarize a 100-page regulatory filing can also draft a customer communication, explain a complex derivatives structure, or generate Python code for a risk calculation.

This generality is both the opportunity and the challenge. LLMs are remarkably versatile, but that versatility means they require careful governance -- a topic we will explore in depth in the Governance and Risk module.

To see where LLMs fit in the broader technology picture, explore the interactive AI technology stack below. Click any layer to learn what it does, which tools power it, and how banks use it.

Why Banking Executives Should Care

The banking industry generates and consumes more text than almost any other sector. Consider the daily volume of:

• Regulatory documents: Basel frameworks, OCC bulletins, Fed guidance, state-level regulations
• Customer communications: Emails, chat transcripts, complaint letters, advisory notes
• Internal reports: Credit memos, risk assessments, audit findings, board presentations
• Legal documents: Loan agreements, compliance certifications, litigation filings
• Market research: Analyst reports, earnings calls, economic forecasts

Every one of these text-heavy workflows is a potential LLM use case. Banks that deploy LLMs strategically can process information faster, reduce manual effort on routine tasks, and free their most expensive resource -- experienced professionals -- to focus on judgment-intensive work.

Key Capabilities for Financial Services

LLMs bring several capabilities that map directly to banking needs:

Document Summarization and Analysis

An LLM can read a 200-page regulatory proposal and produce a structured summary highlighting the provisions most relevant to your institution. What currently takes a compliance analyst two days can be reduced to minutes -- with the analyst then reviewing and refining the output rather than starting from scratch.

Intelligent Search and Retrieval

When combined with your internal document repositories (a technique called Retrieval-Augmented Generation, or RAG), LLMs can answer natural-language questions about your own policies, procedures, and historical decisions. Instead of searching through SharePoint folders, a relationship manager could ask: "What is our current policy on commercial real estate concentration limits?"

Draft Generation

From customer correspondence to board reports to regulatory responses, LLMs can generate first drafts that human experts then refine. This shifts the workflow from creation to curation -- a significantly more efficient model.

Code and Data Analysis

LLMs can write SQL queries, Python scripts, and data analysis code from plain-English descriptions. This democratizes data access for business users who understand the questions but lack programming skills.

Limitations and Risks

No discussion of LLMs is complete without an honest assessment of their limitations -- particularly in a regulated industry like banking.

HallucinationsHallucinationWhen an AI model generates plausible-sounding but factually incorrect information. A critical risk in banking where inaccurate outputs could lead to regulatory violations or financial losses.See glossary. LLMs can generate plausible-sounding but factually incorrect information. In banking, where accuracy is non-negotiable, every LLM output touching customers, regulators, or financial decisions must be verified by qualified humans.

Data privacy. Sending sensitive customer data or proprietary trading strategies to a cloud-hosted LLM creates data leakage risk. Many banks deploy LLMs within their own infrastructure or use enterprise agreements with strict data handling provisions.

Bias and fairness. LLMs inherit biases present in their training data. If used in any capacity that influences lending, pricing, or customer treatment decisions, banks must validate for fair lending compliance -- the same way they validate traditional models under SR 11-7.

Regulatory uncertainty. Banking regulators are still developing frameworks for AI governance. The OCC, Fed, and FDIC have issued joint guidance, but the regulatory landscape is evolving rapidly. Deploying LLMs without a clear governance framework is a material risk.

What Comes Next

Understanding what an LLM is represents the first step. In the units that follow, you will learn how different foundation models compare, how to connect LLMs to your own data through vector databases and RAG, and how to build the governance framework that makes responsible deployment possible.

The executives who understand these technologies -- not at the level of building them, but at the level of making informed strategic decisions about them -- will be the ones who guide their institutions successfully through this transformation.