AI embeddings vs rule-based matching: what changes
How embedding-based record matching works: cosine similarity, threshold tuning, pair counts, what to embed, failure modes, and how to test it on your data.
Two company lists. One row says Cedar Foods Group Inc, the matching row on the other side says CFG. Elsewhere, Cedar Foods Group Inc sits next to Cedar Farms Group Inc: one word apart, same city, same state, similar revenue, and a different company.
A string matcher cannot connect the first pair and has little to separate the second. In our synthetic benchmark of seven methods on 1,000 × 1,150 company records, none of them found any of the 30 initials-only acronym matches. Cases like these are where embeddings are often considered, in the hope that a model that has “read” a lot of text will connect CFG to its full name and keep Foods and Farms apart.
Whether it does on your data is a measurement question, and we ran it on the same benchmark. On this synthetic data, embeddings landed in the same overall range as the best string methods (F1 0.892), found none of the 30 acronyms, and an LLM confirmation step on top of them lowered the score. The measured results are near the end, with the caveats. The rest of the post covers what changes in a pipeline when you add embeddings, the decisions involved, how embeddings fail, and how to test them. For the string methods alone, see deterministic vs probabilistic matching.
What a rule compares, and what an embedding is
A rule compares the characters of two values: exact match asks whether they are identical; Jaro-Winkler, token overlap and trigram TF-IDF score shared characters, words or character triples. All run locally, are deterministic, and can explain a score by pointing at the characters involved.
An embedding model maps a piece of text to a fixed-length list of numbers, a vector. Texts the model treats as similar get vectors that point in similar directions. You never compare the strings directly any more: you compare their vectors.
The vector size is a property of the model, not of embeddings in general. Google’s gemini-embedding-001 returns 3,072 numbers by default, with the output size configurable from 128 to 3,072 (the documentation recommends 768, 1,536 or 3,072). It is text-only and accepts up to 2,048 input tokens. The newer Gemini Embedding 2 (gemini-embedding-2) is multimodal and accepts up to 8,192 input tokens (Gemini embeddings docs, as of October 2026).
Cosine similarity, on toy vectors
The score you threshold is usually cosine similarity: the cosine of the angle between two vectors. It is 1 when they point the same way and falls as they diverge. Only direction counts, not length.
The snippet below shows the mechanics on toy vectors: four hand-picked numbers each, not output from any model. The values were chosen to illustrate a point, not to predict how a real model behaves.
"""Cosine similarity mechanics on TOY vectors: four hand-picked numbers each.
These are not embeddings. A real model produces the vectors (gemini-embedding-001
returns 3072 numbers by default); the arithmetic below is the same."""
import numpy as np
def cosine(a, b):
return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))
a = np.array([0.9, 0.1, 0.4, 0.0]) # toy vector for record A
B = {
"B1 toy: same entity, reworded": np.array([0.7, 0.2, 0.6, 0.1]),
"B2 toy: sibling, one detail differs": np.array([0.9, 0.1, 0.3, 0.2]),
"B3 toy: unrelated": np.array([0.1, 0.9, 0.0, 0.6]),
}
for name, v in sorted(B.items(), key=lambda kv: -cosine(a, kv[1])):
print(f"{cosine(a, v):.3f} {name}")
# Length does not matter, only direction: scaling B3 by 10 leaves its score unchanged.
print(f"{cosine(a, 10 * B['B3 toy: unrelated']):.3f} B3 scaled x10")
# After scaling every vector to length 1, cosine is a plain dot product,
# so one matrix multiply scores every A x B pair at once.
M = np.array(list(B.values()))
M = M / np.linalg.norm(M, axis=1, keepdims=True)
print(np.round((a / np.linalg.norm(a)) @ M.T, 3))
Output:
0.974 B2 toy: sibling, one detail differs
0.948 B1 toy: same entity, reworded
0.167 B3 toy: unrelated
0.167 B3 scaled x10
[0.948 0.974 0.167]
Three takeaways. Nothing in the arithmetic knows which detail matters: if a vector moves only slightly when a model number or business word changes, a sibling can outrank the true match, as it does here by construction. Unit-length vectors turn cosine into a dot product, which is how you score a whole A × B block at once. And the score never says “same entity” by itself; a threshold does, and it has to come from somewhere.
The threshold is yours to tune, not to borrow
A cutoff from a blog post, a vendor default or another project tells you nothing about your data. It depends on the model, the output size, the field, the preprocessing and the mix of hard non-matches. The string benchmark shows how far apart tuned cutoffs land, each chosen on one random seed and evaluated on another:
| Method | Tuned threshold | Precision | Recall |
|---|---|---|---|
| Jaro-Winkler on normalized names | 0.943 | 0.971 | 0.821 |
| Character 3-gram TF-IDF cosine | 0.828 | 0.902 | 0.853 |
| Token overlap (Jaccard) | 0.667 | 1.000 | 0.784 |
| Fellegi-Sunter style weights | 10.810 | 1.000 | 0.806 |
Source: our offline benchmark run. The scales are not comparable between methods, which is the point: a number that works for one scorer means nothing for another.
Fixed cutoffs also behave badly on small real files. On the 15-record sample company files in our repository, difflib’s similarity picked the right registry record for all 15 CRM names, but the correct matches scored anywhere from 0.50 (Johnson & Johnson Medical vs J&J Medical Devices LLC) to 1.00. A cutoff of 0.7 would have dropped two correct matches (the J&J pair and Pacific Northwest Tech vs Pacific NW Technology Inc, which scores just under 0.70); 0.9 would have dropped four. That example is tiny and every CRM record has a match, so it shows the spread of scores, not accuracy.
The procedure is the same for embeddings: label a few hundred pairs, including look-alike non-matches, sweep the cutoff, pick it on one sample and confirm it on another. If you change the embedding model or its output size, the vectors change, so re-embed both sides and re-tune. Understanding similarity thresholds and measuring precision and recall cover the mechanics.
The pair count is the real cost driver
Comparing every record on one side with every record on the other produces N × M pairs. For 1,000 × 1,150 records that is 1,150,000 pairs. Multiply both sides by 10 and you get 115 million.
How many of those pairs a method actually scores depends on candidate generation, not the similarity function. In the benchmark:
Embeddings change the cost structure. You embed each record once, so model calls grow with N + M (2,150 texts here), not N × M. Comparing the vectors is local arithmetic. LLM confirmation is the opposite: every pair you ask about adds its own prompt and response tokens.
Here is what that means in money, as arithmetic on assumptions, not a measurement. gemini-2.5-flash-lite lists $0.10 per 1M input tokens and $0.40 per 1M output tokens (standard tier, pricing page, as of October 2026; output includes thinking tokens; the Batch API is 50% cheaper). Assume 150 input tokens and 10 output tokens per pair:
| Pairs sent to the LLM | Standard | Batch (50% off) |
|---|---|---|
| 1,150,000 (every pair) | $21.85 | $10.93 |
| 154,435 (token-index candidates) | $2.93 | $1.47 |
| 1,000 (one top candidate per A record) | $0.019 | $0.0095 |
Per pair: 150 × $0.10/1M + 10 × $0.40/1M = $0.000019. Longer prompts, multi-field records or thinking tokens change this linearly. Not a typical cost; compute yours from your own token counts.
The shape matters more than the totals: the bill scales with how many pairs survive your filters. For embeddings, the price of gemini-embedding-001 is not on Google’s current pricing page; Gemini Embedding 2 lists $0.20 per 1M text input tokens ($0.10 batch). For indexing and blocking strategies, see the survey by Papadakis et al., “Blocking and Filtering Techniques for Entity Resolution: A Survey” (ACM Computing Surveys 53(2), 2020, doi:10.1145/3377455), and our guide to blocking keys.
What to embed
The input text is a design decision with real consequences.
One field or several. Concatenating name, address and category into one string gives a single score that mixes all of them. You cannot tell which field drove it, and a long address can dominate a short name. Embedding each field separately gives you per-field similarities that you can threshold or weight individually, the same way field agreements are combined in Fellegi-Sunter style scoring.
Raw or normalized. In the string benchmark, normalization alone (lowercase, strip punctuation and legal suffixes, expand common abbreviations) raised exact-match recall from 0.414 to 0.717 without a single wrong link. Whether normalizing before embedding helps or hurts has not been measured here. The Gemini benchmark run embedded normalized names; run both variants on labelled pairs from your own data before choosing.
Length and type. gemini-embedding-001 accepts up to 2,048 input tokens and only text. Long free-text fields need to be shortened or summarized before embedding. For images or PDFs, either describe them first with an extraction step and embed the description, or use a multimodal model such as Gemini Embedding 2.
Exact identifiers. IDs, SKUs, phone numbers and tax numbers are better compared with rules. An embedding of X100-16 and X100-32 has no built-in notion that the last digits are the whole point.
Where embeddings fail
These are general, well-known limitations. The benchmark near the end reports overall wrong links for one model on one synthetic dataset, but it does not break them down by cause, so how often each of these occurs is not measured here.
- Small differences that change the entity. Model numbers, sizes, quantities, version numbers and single swapped words (
FoodsvsFarms) can move a vector only slightly, so look-alike non-matches can score close to, or above, true matches. The benchmark includes 150 such “sibling” records, which share city and state with an A record and have a similar revenue, precisely to test this. - Negation and qualifiers.
gluten-free pastaandpasta, contains glutenshare most of their content. A similarity score is not a logical comparison. - Knowledge the model cannot have. The benchmark’s acronyms are initials of invented names, so a model is unlikely to have seen them. Recovering
CFGrequires recognizing an initials pattern or using another field, not recall of a known company. - Opacity. A cosine of, say, 0.86 does not say which tokens mattered. Debugging means looking at the nearest neighbors, not at a rule.
- Version drift. Vectors from different models, or from the same model at different output sizes, are not interchangeable. Upgrading means re-embedding everything and re-tuning.
Rules have different failures, and those are measured: typos were almost invisible to equality rules (recall 0.01) while Jaro-Winkler found 73%; word-order swaps hurt Jaro-Winkler (0.49) but not token overlap (0.81); dropped words were found only by trigram TF-IDF (0.85).
A hybrid pipeline, as a design
The usual way to combine them is a funnel. This is a design pattern, not a measured result.
- Deterministic filtering and candidate generation. Exact keys you trust, normalized-name indexes, numeric tolerances. This is cheap and explainable. A filter that is wrong removes true matches permanently, so filter only on fields you trust, and measure the recall of this step on its own.
- Embedding similarity on the survivors. Embed only records that survived step 1, score candidate pairs by cosine, and apply a tuned threshold.
- Optional LLM confirmation on a score band. Send pairs within a band around the threshold to an LLM with both records in full. Because cost grows per pair, the size of that band is a budget decision.
We tested a narrow version of steps 2 and 3 on the benchmark (top-1 candidate per record, then an LLM yes/no on a band that extends below the threshold). In that test the LLM step lowered F1; the details are below. That is one prompt on one dataset, not a verdict on the pattern.
Two decisions sit between the stages: what each stage hands to the next (top-1 candidate per record, top-k, or everything above a floor) and which stage gets the final say. The history of matching methods explains where each layer came from.
Evaluation checklist
- Label pairs from your own data, including hard non-matches (siblings, branches, product variants), not just obvious ones.
- Tune on one sample, report on another.
- Report precision and recall separately, and recall by error type (typos, abbreviations, word order, acronyms, missing words).
- Measure the recall of the candidate-generation step on its own; nothing downstream can recover a pair it dropped.
- Record pair counts and token counts per stage, then compute cost from the current price list.
- Re-run the whole evaluation when the model, output size or input text changes.
Measured on the benchmark: embeddings and LLM confirmation
We ran embeddings and LLM confirmation on the same synthetic data as the string methods (1,000 A records, 1,150 B records, 700 true matches):
- Embeddings. Normalized names embedded with
gemini-embedding-001at its default 3,072 dimensions. Each A record is linked to its highest-cosine B record if the cosine is at or above 0.8992, a threshold tuned on seed 1 and evaluated on seed 2. - Embeddings plus LLM. For each A record whose top candidate scored at or above the tuned threshold minus 0.12 (0.779),
gemini-2.5-flash-liteis shown the two raw names and asked, as structured JSON at temperature 0, whether they are the same company. Its yes or no decides the link. The prompt asks the model to consider abbreviations, legal suffixes, typos and word order, but to treat different business words as different companies.
| Method | Precision | Recall | F1 | Wrong links |
|---|---|---|---|---|
| Jaro-Winkler on normalized names | 0.971 | 0.821 | 0.890 | 17 |
| Fellegi-Sunter style weights | 1.000 | 0.806 | 0.892 | 0 |
| Token overlap (Jaccard) | 1.000 | 0.784 | 0.879 | 0 |
| Character 3-gram TF-IDF cosine | 0.902 | 0.853 | 0.877 | 65 |
| Embeddings, top-1 candidate | 0.925 | 0.861 | 0.892 | 49 |
| Embeddings + LLM confirmation | 0.909 | 0.827 | 0.866 | 58 |
Source: our Gemini benchmark run, test seed 2. We ran the Gemini mode twice with identical results (the second run reused the cached embeddings, so only the LLM step was repeated). The LLM step made 989 calls (48,129 input and 6,995 output tokens), which is $0.0076 at the published gemini-2.5-flash-lite prices. We did not compute an embedding cost because gemini-embedding-001 is not on Google's current pricing page.
| Method | Typo | Word order | Dropped word | Abbreviation | Acronym |
|---|---|---|---|---|---|
| Jaro-Winkler | 0.73 | 0.49 | 0.07 | 0.85 | 0.00 |
| Token overlap | 0.14 | 0.81 | 0.00 | 0.85 | 0.00 |
| Character 3-gram TF-IDF | 0.33 | 0.66 | 0.85 | 0.95 | 0.00 |
| Embeddings | 0.68 | 0.77 | 0.36 | 0.88 | 0.00 |
| Embeddings + LLM | 0.32 | 0.62 | 0.58 | 0.91 | 0.00 |
n = 85 typo, 77 word-order, 55 dropped-word, 80 abbreviation and 30 acronym matches. A match can carry more than one corruption and then counts under each.
What the numbers show, on this data:
- Embeddings alone matched the best string methods overall. F1 0.892 against 0.877 to 0.892 for the four best string methods; with one run, a few points of difference are not meaningful. The trade-off differs: recall 0.861 is slightly above trigram TF-IDF’s 0.853 with higher precision (0.925 against 0.902), but precision is below Jaro-Winkler, token overlap and the Fellegi-Sunter style model, which all made fewer wrong links.
- No method won every error type. Embeddings sit in the middle on typos, word order and dropped words. Jaro-Winkler was better on typos (0.73), token overlap on word order (0.81) and trigram TF-IDF on dropped words (0.85).
- Acronyms stayed at zero. Embeddings found 0 of 30. These acronyms are initials of invented company names, so a model is unlikely to have seen them; that matches the limitation described above.
- The LLM step lowered F1 from 0.892 to 0.866. Of the 645 correct top candidates that reached the LLM, it rejected 66. Of the 344 wrong top candidates that reached it, it accepted 58: 26 that the embedding threshold had already accepted, and 32 from the extra band below the threshold. It did raise dropped-word recall from 0.36 to 0.58, most likely because the extra band lets in candidates that score below the embedding threshold.
- The design caps recall. For 55 of the 700 true matches the highest-cosine candidate was the wrong record, so linking only the top candidate cannot exceed a recall of 0.921 here, whatever the LLM does.
We recorded the pairs the LLM got wrong. Of the first 12 correct candidates the LLM rejected, 8 involve a typo, such as Dakota Insurance Holdings Limited against Dakota Insurance Holdings Limtied, and typo recall fell from 0.68 to 0.32. The prompt asked the model to consider typos and word order but also to treat different business words as different companies. Why it rejected misspelled names anyway we do not know; we did not test other prompts. Among the wrong candidates it accepted are look-alikes such as Foxglove Technologies Global Company and Foxglove Technologies Group Company. The generator creates such one-word-changed siblings deliberately as non-matches, but in real data two names like these could be either, and nothing in the names alone settles it. The embedding saw normalized names while the LLM saw the raw ones; we did not test whether giving it normalized names changes the result.
What this does not show. It is one synthetic dataset, one prompt, one embedding model and one LLM, at one tuned threshold. It does not show that LLM confirmation is a bad idea: a different prompt, more fields (city, state, revenue) in the prompt, or an LLM that only reviews a narrow band around the threshold might behave differently, and we did not try them on the test set because tuning a prompt on the data you report on inflates the result. It does show that the step needs its own evaluation on labelled pairs, including typos and look-alike siblings, before you rely on it.
The 989 LLM calls cost $0.0076 at the published gemini-2.5-flash-lite prices; embedding cost depends on your key’s billing. The benchmark code is not published, so these figures are our measurements and cannot be rerun from this page. Because the data is synthetic, the evaluation that should decide your design is the one on your own labelled pairs.
In Match Data Studio: the pipeline runs string and numeric pre-filters with no AI first, then computes AI extractions and embeddings (
gemini-embedding-001at its default 3,072 dimensions) only for rows that survived, then scores candidate pairs by cosine similarity with embedding and numeric thresholds and optional LLM confirmation usinggemini-2.5-flash-lite(a per-check option can selectgemini-3.1-flash-lite-preview). File columns are skipped for embeddings; to use images, add an AI extraction that describes them. The product’s matching accuracy has not been benchmarked.
Keep reading
- Deterministic vs probabilistic matching — measured results for the string methods used here
- Understanding similarity thresholds — how to pick and validate a cutoff
- How to set up blocking keys — reducing pair counts before any scoring
- From Soundex to LLMs — where each matching method came from
- Can ChatGPT do fuzzy matching? Yes — but here’s where it breaks down
Ready to match your data?
Create a free account, upload your CSV files, and configure matching in plain English. 100 free credits — no credit card required.
Free account No credit card Match your files in minutes