No single solution universally wins between Large Language Models (LLMs, ≥30B parameters, often via APIs) and Small Language Models (SLMs, ~1–15B, typically open-weights or proprietary specialist models). For banks, insurers, and asset managers in 2025, your selection should be governed by regulatory risk, data sensitivity, latency and cost requirements, and the complexity of the use case.
Financial services are subject to mature model governance standards. In the US, Federal Reserve/OCC/FDIC SR 11-7 covers any model used for business decisioning, including LLMs and SLMs. This means required validation, monitoring, and documentation—irrespective of model size. The NIST AI Risk Management Framework (AI RMF 1.0) is the gold standard for AI risk controls, now widely adopted by financial institutions for both traditional and generative AI risks.
In the EU, the AI Act is in force, with staged compliance dates (Aug 2025 for general purpose models, Aug 2026 for high-risk systems such as credit scoring per Annex III). High-risk means pre-market conformity, risk management, documentation, logging, and human oversight. Institutions targeting the EU must align remediation timelines accordingly.
Core sectoral data rules apply:
Supervisors (FSB/BIS/ECB) and standard setters highlight systemic risk from concentration, vendor lock-in, and model risk—neutral to model size.
Key point: High-risk uses (credit, underwriting) require tight controls regardless of parameters. Both SLMs and LLMs demand traceable validation, privacy assurance, and sector compliance.
SLMs (3–15B) now deliver strong accuracy on domain workloads, especially after fine-tuning and with retrieval augmentation. Recent SLMs (e.g., Phi-3, FinBERT, COiN) excel at targeted extraction, classification, and workflow augmentation, cut latency (<50ms), and allow self-hosting for strict data residency—and are feasible for edge deployment.
LLMs unlock cross-document synthesis, heterogeneous data reasoning, and long-context operations (>100K tokens). Domain-specialized LLMs (e.g., BloombergGPT, 50B) outperform general models on financial benchmarks and multi-step reasoning tasks.
Compute economics: Transformer self-attention scales quadratically with sequence length. FlashAttention/SlimAttention optimizations reduce compute costs, but don’t defeat the quadratic lower bound; long-context LLMs can be exponentially costlier at inference than short-context SLMs.
Key point: Short, structured, latency-sensitive tasks (contact center, claims, KYC extraction, knowledge search) fit SLMs. If you need 100K+ token contexts or deep synthesis, budget for LLMs and mitigate cost via caching and selective “escalation.”
Common risks: Both model types are exposed to prompt injection, insecure output handling, data leakage, and supply chain risks.
Three proven modes in finance:
Regardless, always implement content filters, PII redaction, least-privilege connectors, output verification, red-teaming, and continuous monitoring under NIST AI RMF and OWASP guidance.
JPMorgan Chase deployed a specialized Small Language Model (SLM), called COiN, to automate the review of commercial loan agreements—a process traditionally handled manually by legal staff. By training COiN on thousands of legal documents and regulatory filings, the bank slashed contract review times from several weeks to mere hours, achieving high accuracy and compliance traceability while drastically reducing operational cost. This targeted SLM solution enabled JPMorgan to redeploy legal resources toward complex, judgment-driven tasks and ensured consistent adherence to evolving legal standards
FinBERT is a transformer-based language model meticulously trained on diverse financial data sources, such as earnings call transcripts, financial news articles, and market reports. This domain-specific training enables FinBERT to accurately detect sentiment within financial documents—identifying nuanced tones like positive, negative, or neutral that often drive investor and market behavior. Financial institutions and analysts leverage FinBERT to gauge prevailing sentiment around companies, earnings, and market events, using its outputs to support market forecasting, portfolio management, and proactive decision-making. Its advanced focus on financial terminology and contextual subtleties makes FinBERT far more precise than generic models for financial sentiment analysis, providing practitioners with authentic, actionable insights into market trends and predictive dynamics
References:
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.