From Keywords to Meaning: A Practical Tour of Enterprise Search Engines

consulting@stevesale.com (Steve Sale) — Tue, 15 Sep 2026 15:25:51 GMT

If you've worked on an enterprise search project in the last few years, you've probably noticed the vocabulary shifting under your feet. "Relevance tuning" used to mean adjusting field boosts and stemming rules. Now it means choosing embedding models, tuning ANN indexes, and deciding how to fuse rankings from two completely different retrieval paradigms.

This post is a tour of that landscape; where enterprise search has been, where it's going, and why the most effective systems today rarely pick just one approach. We'll move through three stages:

Statistical (lexical) search The workhorse of search for many, many years
Vector (semantic) search Retrieval based on meaning, not matching words
Hybrid search with semantic reranking Combining both, the way most production systems actually work today

Part 1: Statistical Search: Search Built on Word Statistics

Statistical, or "lexical," search engines work by matching the literal terms in a query against the literal terms in documents, then ranking results using statistics about how those terms are distributed across the corpus. No understanding of meaning is involved; it's pattern matching, made smart through mathematics.

1.1 The Inverted Index

Almost every statistical search engine (Such as Elasticsearch, OpenSearch, Solr, classic Lucene) is built on an inverted index: a mapping from each term to the list of documents (and positions) in which it appears. This is what makes lexical search fast, instead of scanning every document for a query term, the engine does a direct lookup.

1.2 TF-IDF

Term Frequency–Inverse Document Frequency is the foundational scoring idea. It weighs a term's importance by two competing signals:

Term Frequency (TF): how often the term appears in this document (more mentions, more relevant)
Inverse Document Frequency (IDF): how rare the term is across the whole corpus (common words like "the" or "system" get discounted; rare, distinctive words get boosted)

A document scores highly when it contains query terms frequently, and those terms are otherwise rare in the collection. TF-IDF is simple, explainable, and still conceptually underlies most lexical scoring today.

1.3 BM25 — The Modern Default

BM25 (Best Matching 25) is a refinement of TF-IDF and is the default scoring algorithm in Elasticsearch, OpenSearch, and Solr. It improves on raw TF-IDF in two important ways:

Term frequency saturation — the 10th occurrence of a word matters much less than the 2nd. BM25 diminishes returns on repeated terms instead of scaling linearly.
Document length normalisation — a long document naturally contains more term matches by chance. BM25 penalises this so a short, focused document isn't unfairly outranked by a long one that happens to mention the term more.

BM25 is fast, requires no training, is highly explainable (you can show exactly why a document scored the way it did), and remains extremely competitive — especially for precise, keyword-heavy queries: product codes, legal citations, exact names, error messages.

1.4 Other Statistical Techniques Worth Mentioning

Boolean retrieval: The pre-statistical ancestor (AND/OR/NOT matching), still underlies filtering logic in modern engines
Field boosting and query-time weighting: Manually telling the engine that a title match matters more than a body match
Synonym expansion, stemming, and lemmatisation: Techniques to bridge small vocabulary gaps ("run" / "running" / "ran") without leaving the lexical paradigm
Language models for Information Retrieval (e.g. query likelihood models) A more probabilistic cousin of TF-IDF/BM25, less common in enterprise tooling but conceptually important.

1.5 Where Statistical Search Falls Short

The core weakness of every technique above is that they only match words, not meaning. A query for "employee exit process" will not match a document titled "offboarding procedure" unless someone has manually added a synonym rule. This is the vocabulary mismatch problem, and it's the single biggest driver behind the move to vector search.

Part 2: Vector Search — Search Built on Meaning

2.1 The Core Idea: Embeddings

Vector search represents text (or images, audio, etc.) as embeddings; dense numerical vectors produced by a neural network, positioned in a high-dimensional space such that semantically similar content ends up close together, regardless of the specific words used.

"Offboarding procedure" and "employee exit process" will land near each other in embedding space, even though they share no words — because the model has learned that they mean similar things.

2.2 How It Works, End to End

Chunking: The document is first split into a series of passages small enough to be individually retrieved and fed to the model, but large enough to retain context. Chunking strategy is one of the most underrated variables in vector search quality: too small and you lose context, too large and you dilute relevance and blow through token budgets.
Embedding generation: An embedding model converts each document chunk into a vector. These typically have anything from 300 – 3500 dimensions.
Indexing: Vectors are stored in a vector index, usually using an Approximate Nearest Neighbour (ANN) algorithm such as HNSW (Hierarchical Navigable Small World graphs), which trades a small amount of accuracy for dramatic speed gains over exact nearest-neighbour search.
Query time: The user's query is embedded using the same model, and the engine finds the k nearest vectors by a distance metric, usually cosine similarity or dot product.

2.3 Why It Matters for Enterprise Search

Handles paraphrase and vocabulary mismatch: The exact problem lexical search struggles with.
Cross-lingual retrieval: Some embedding models place equivalent content from different languages near each other.
Works well on conversational, natural-language queries: Increasingly the norm as users bring chatbot-style habits to internal search.

2.4 Where Vector Search Falls Short

Precision on exact terms: Vector search can be worse than lexical search at matching exact product SKUs, error codes, or names, because embeddings smooth over specifics in favour of general meaning.
Explainability: It's much harder to tell a stakeholder why a document scored highly; there's no simple term-overlap story.
Domain drift: General-purpose embedding models may not capture the nuance of a specific enterprise's jargon unless fine-tuned.
Cost and infrastructure: Embedding generation and ANN indexes add real compute and storage overhead compared to an inverted index

This complementary failure pattern, lexical is precise but rigid, vector is flexible but fuzzy, is exactly why the two are increasingly combined rather than treated as competitors.

Part 3: Hybrid Search and Semantic Reranking

3.1 The Case for Hybrid

Neither paradigm alone is sufficient for most enterprise use cases. A legal team searching for a specific clause number needs BM25's precision. The same team searching "what happens if a supplier breaches confidentiality" needs vector search's grasp of meaning. Hybrid search runs both retrieval methods in parallel and merges the results.

3.2 Fusion Techniques

The central challenge in hybrid search is that BM25 scores and cosine-similarity scores live on completely different scales and aren't directly comparable. Two common approaches:

Reciprocal Rank Fusion (RRF): Instead of combining raw scores, RRF combines rankings. Each document gets a score based on its rank position in each result list (1/(k + rank)), and these are summed. This sidesteps the scale problem entirely and is simple, robust, and requires no tuning.
Weighted score combination: Normalising both score sets (e.g. min-max scaling) and combining them with a tunable weight (e.g. 0.3 × BM25 + 0.7 × vector). More flexible but requires tuning per use case and corpus.

3.3 Semantic Reranking: The Second Stage

Hybrid retrieval typically returns a good candidate set (say, the top 50–100 documents), but the ranking within that set can still be improved. This is where semantic rerankers come in, usually cross-encoder models.

The key architectural distinction:

Bi-encoders (used in vector search) embed the query and document separately, then compare vectors. Fast, but the model never actually looks at the query and document together.
Cross-encoders (used in reranking) feed the query and document into the model together, letting it directly attend to how they relate. This produces much more accurate relevance judgments but is far too slow to run over an entire corpus, which is why it's reserved for reranking a small candidate set rather than initial retrieval.

A typical modern pipeline looks like this:

Retrieve: BM25 and vector search each return their top N candidates
Fuse: RRF or weighted fusion merges the two lists into one candidate set
Rerank: A cross-encoder (e.g. Cohere Rerank, BGE-reranker, or a fine-tuned in-house model) rescoring that candidate set for final relevance
Return: The top-k reranked results are served to the user or passed to a downstream LLM (in a RAG setup)

3.4 Why This Architecture Wins in Practice

Recall from two different signal types: You catch documents that either method alone would miss.
Precision from reranking: The expensive, accurate model is only run on a small candidate set, keeping latency manageable.
Graceful degradation: If the vector index is stale or the embedding model is poorly suited to a niche domain, lexical search still provides a solid baseline.
Tunability at each stage: You can adjust fusion weights, reranker choice, and candidate set size independently as you learn more about your users' query patterns.

Blog