Product matching for price comparison: a worked example
A runnable walkthrough on 20 sample product rows: clean prices, check barcodes, block on brand and storage, score names, and see where variants break matches.
You have your own catalog in one CSV and a competitor’s catalog in another. The question is simple: for each of your products, what does the competitor charge for the same item? The hard part is “the same item”. Your file says iPhone 16 Pro 256GB Natural Titanium; theirs says Apple iPhone 16 Pro 256 GB - Natural Titanium. Both files also contain an iPhone 16 Pro 128GB, which is a different product at a different price, and whose name differs from the 256GB one only in storage and colour.
This post works through that problem on two real files, step by step, with code you can run. The outputs shown below are the real outputs of the code shown. The few counts in the text that no snippet prints (codes starting with 0, prices parsed, storage and brand agreement) come from a separate check on the same files.
The data
The files are the 20-row electronics test fixtures from the Match Data Studio repository, sample_data/electronics/client.csv and sample_data/electronics/competitor.csv. Both are public downloads, so the walkthrough is reproducible without the repository: sample_electronics_client.csv and sample_electronics_competitor.csv (if you download them, change the paths in the snippets). They are made-up test files, not scraped data. Their prices are written like €1099.00 and $1099.00. They use different column names for the same things:
| File | Name column | Price column | Code column | Example name | Example price |
|---|---|---|---|---|---|
| client.csv | product_name | price | ean_code | iPhone 16 Pro 256GB Natural Titanium | €1099.00 |
| competitor.csv | item_name | retail_price | barcode | Apple iPhone 16 Pro 256 GB - Natural Titanium | $1099.00 |
| client.csv | product_name | price | ean_code | Samsung Galaxy S25 256GB Mint | €899.00 |
| competitor.csv | item_name | retail_price | barcode | SAMSUNG Galaxy S25 256GB - Mint Green | $899.00 |
The competitor file also has manufacturer and storage_capacity columns; the client file has neither.
Twenty rows is a demonstration, not an accuracy measurement. The point is to see each failure mode with real strings, not to estimate how often it happens in your data.
Ground truth, labelled by hand. We read both files side by side and compared brand, model, storage, case size and colour. By that reading, every client row has exactly one counterpart, and it is the competitor row with the same number: D1_001 is COMP_001, and so on up to D1_020. Six pairs differ in colour wording (Mint / Mint Green, Iceblue / Ice Blue, Obsidian / Obsidian Black, White / Polar White, Jet Black / Jet Black Aluminium, and AirPods with no colour on the client side and White on the competitor side); we judged those to be the same product. Because every row has a true partner, this data cannot show you false matches for products the competitor does not sell. We simulate that case in step 4.
A note on collection: if your competitor file comes from scraping, collect it in line with each site’s terms of service and its robots.txt rules, and keep request rates modest. This is a practical note, not legal advice.
The snippets build on each other: run them in order in one Python session from the repository root. They need pandas and the standard library.
Step 1: clean the prices, and keep the currency
The obvious move is a regex that deletes everything except digits and dots. It works on these files, which contain prices like €1099.00 and $1099.00 with no thousands separators, but it silently breaks on a European-formatted price, and it throws the currency away. The parser below keeps the currency and handles both decimal conventions.
import re
import pandas as pd
client = pd.read_csv("sample_data/electronics/client.csv", dtype=str)
comp = pd.read_csv("sample_data/electronics/competitor.csv", dtype=str)
def parse_price(s):
"""Return (amount, currency) from strings like '€1099.00', '$1,099.00', '1.099,00 €'."""
s = s.strip()
cur = {"€": "EUR", "$": "USD", "£": "GBP"}.get(re.sub(r"[\d\s.,]", "", s)[:1])
num = re.sub(r"[^\d.,]", "", s)
if re.search(r",\d{2}$", num): # decimal comma: 1.099,00
num = num.replace(".", "").replace(",", ".")
else: # decimal point: 1,099.00
num = num.replace(",", "")
return float(num), cur
client[["price_amt", "price_cur"]] = client["price"].apply(lambda s: pd.Series(parse_price(s)))
comp[["price_amt", "price_cur"]] = comp["retail_price"].apply(lambda s: pd.Series(parse_price(s)))
print(client[["price", "price_amt", "price_cur"]].head(3).to_string(index=False))
print(comp[["retail_price", "price_amt", "price_cur"]].head(3).to_string(index=False))
print("currencies:", client.price_cur.unique(), comp.price_cur.unique())
print("naive [^0-9.] regex on '1.099,00 €':", float(re.sub(r"[^0-9.]", "", "1.099,00 €")))
for s in ["€1,099.00", "1.099,00 €", "$1099"]:
print(f"{s!r:14} ->", parse_price(s))
Output:
price price_amt price_cur
€1099.00 1099.0 EUR
€999.00 999.0 EUR
€1449.00 1449.0 EUR
retail_price price_amt price_cur
$1099.00 1099.0 USD
$999.00 999.0 USD
$1449.00 1449.0 USD
currencies: ['EUR'] ['USD']
naive [^0-9.] regex on '1.099,00 €': 1.099
'€1,099.00' -> (1099.0, 'EUR')
'1.099,00 €' -> (1099.0, 'EUR')
'$1099' -> (1099.0, 'USD')
Two findings already. The naive regex turns 1.099,00 € into 1.099, a price off by a factor of a thousand that no type check would catch. And the two files are in different currencies: the client prices in euros, the competitor in dollars. Hold on to that; it matters in step 5. More cleaning steps that pay off before any matching are in data cleaning before matching.
Step 2: use a shared identifier first, and check it
If both sides carry a product code, an exact join on it is the most direct link, and it does not depend on how anyone wrote the name. Here the client has ean_code and the competitor has barcode. Before joining, two checks.
First, read codes as text. A numeric read drops leading zeros, which are part of the code. Second, validate the GS1 check digit. The GS1 General Specifications define a GTIN as an 8, 12, 13 or 14-digit string whose last digit is computed from the others (weights 3 and 1 alternating from the right, then the distance to the next multiple of ten; GS1 shows a worked example). A code that fails the check is not a valid GTIN as written (for example, it was mistyped or truncated).
def check_digit(body: str) -> int:
"""GS1 check digit for the digits WITHOUT the check digit (weights 3,1,3,1 from the right)."""
total = sum(int(d) * (3 if i % 2 == 0 else 1) for i, d in enumerate(reversed(body)))
return (10 - total % 10) % 10
def is_valid_gtin(s: str) -> bool:
return s.isdigit() and len(s) in (8, 12, 13, 14) and check_digit(s[:-1]) == int(s[-1])
print("leading zeros, default read:", pd.read_csv("sample_data/electronics/client.csv")["ean_code"].iloc[12])
print("leading zeros, dtype=str: ", client["ean_code"].iloc[12])
for name, df, col in [("client", client, "ean_code"), ("competitor", comp, "barcode")]:
codes = df[col].fillna("")
print(f"{name:10} non-empty: {(codes != '').sum()}/20 "
f"valid GTIN check digit: {codes.map(is_valid_gtin).sum()}/20")
linked = client.merge(comp, left_on="ean_code", right_on="barcode")
print("rows linked by identical code:", len(linked))
print("example failure:", "194253890001", "expected check digit", check_digit("19425389000"))
Output:
leading zeros, default read: 27242920001
leading zeros, dtype=str: 027242920001
client non-empty: 20/20 valid GTIN check digit: 1/20
competitor non-empty: 20/20 valid GTIN check digit: 1/20
rows linked by identical code: 20
example failure: 194253890001 expected check digit 3
| Check | client.csv | competitor.csv |
|---|---|---|
| Rows with a code | 20 / 20 | 20 / 20 |
| Codes with a valid GS1 check digit | 1 / 20 | 1 / 20 |
| Codes that start with 0 (lost on a numeric read) | 2 / 20 | 2 / 20 |
| Rows linked by an identical code string | 20 | 20 |
The one code that passes, 194253890003, does so because its last digit happens to equal the computed check digit.
So the codes are identical on both sides, and the code join links all 20 rows to the pairs we labelled by hand. But 19 of the 20 codes fail the check digit, so for this exercise they act as arbitrary identifiers, not GTINs. Whether to trust codes like these is a policy decision, and a strict pipeline that only accepts valid GTINs would reject 19 of these rows.
That is the scenario for the rest of this post. We set the codes aside and match on names, which lets us use the code join as an answer key.
When codes do validate, normalise them before joining. GS1 recommends storing a GTIN as a 14-digit number, left-padded with zeros, and states that the leading zeros do not change its value. Padding with s.zfill(14) after validation makes a 12-digit UPC-A and the 13-digit form of the same item collapse into one key.
Step 3: extract attributes and block on them
For price comparison, the attributes that separate one sellable item from another (brand, storage, case size) matter more than the rest of the name. So we pull them out with regular expressions and use them as a blocking key: only rows with the same brand, storage and size are compared at all. The remaining words, model and colour, get a similarity score. Blocking is covered in more depth in how to set up blocking keys.
from difflib import SequenceMatcher
BRANDS = {"apple", "samsung", "google", "sony", "oneplus", "xiaomi", "motorola", "nothing"}
FAMILY = {"iphone": "apple", "ipad": "apple", "airpods": "apple"} # names that omit the brand
def attrs(name):
s = re.sub(r"[()\-,]", " ", name.lower())
tokens = s.split()
brand = tokens[0] if tokens[0] in BRANDS else FAMILY.get(tokens[0])
gb = re.search(r"(\d+)\s*gb\b", s)
mm = re.search(r"(\d+)\s*mm\b", s)
# model words: drop brand, storage, size; keep the rest (colour stays in)
rest = re.sub(r"\d+\s*(gb|mm)\b", " ", s)
words = [t for t in rest.split() if t not in BRANDS]
return brand, (gb.group(1) if gb else None), (mm.group(1) if mm else None), " ".join(words)
for df, col in [(client, "product_name"), (comp, "item_name")]:
df[["brand", "gb", "mm", "words"]] = df[col].apply(lambda s: pd.Series(attrs(s)))
df["block"] = df["brand"] + "|" + df["gb"].fillna("-") + "|" + df["mm"].fillna("-")
pairs = client.merge(comp, on="block", suffixes=("_a", "_b"))
pairs["score"] = [SequenceMatcher(None, a, b).ratio() for a, b in zip(pairs.words_a, pairs.words_b)]
print("pairs to score after blocking:", len(pairs), "of", len(client) * len(comp))
best = pairs.sort_values("score", ascending=False).drop_duplicates("product_id")
best["correct"] = best.product_id.str[-3:] == best.ref_id.str[-3:] # hand-checked truth: D1_i <-> COMP_i
print(best.sort_values("product_id")[["product_id", "ref_id", "words_a", "words_b", "score"]]
.to_string(index=False, float_format="%.2f"))
print("linked:", len(best), " correct:", best.correct.sum(),
" client rows with no candidate:", sorted(set(client.product_id) - set(best.product_id)))
Output:
pairs to score after blocking: 28 of 400
product_id ref_id words_a words_b score
D1_001 COMP_001 iphone 16 pro natural titanium iphone 16 pro natural titanium 1.00
D1_002 COMP_002 iphone 16 pro black titanium iphone 16 pro black titanium 1.00
D1_003 COMP_003 iphone 16 pro max desert titanium iphone 16 pro max desert titanium 1.00
D1_004 COMP_004 galaxy s25 ultra titanium black galaxy s25 ultra titanium black 1.00
D1_005 COMP_005 galaxy s25 mint galaxy s25 mint green 0.83
D1_006 COMP_006 galaxy a55 5g iceblue galaxy a55 5g ice blue 0.98
D1_007 COMP_007 pixel 9 pro hazel pixel 9 pro hazel 1.00
D1_008 COMP_008 pixel 9 obsidian pixel 9 obsidian black 0.84
D1_009 COMP_009 13 black eclipse 13 black eclipse 1.00
D1_010 COMP_010 15 ultra white 15 ultra polar white 0.82
D1_011 COMP_011 ipad air 11 inch m2 blue ipad air 11 inch m2 blue 1.00
D1_012 COMP_012 galaxy tab s9 fe lavender galaxy tab s9 fe lavender 1.00
D1_013 COMP_013 wh 1000xm5 black wh 1000xm5 wireless headphones black 0.62
D1_014 COMP_014 airpods pro 2nd generation airpods pro 2nd gen white 0.78
D1_015 COMP_015 galaxy buds3 pro white galaxy buds3 pro white 1.00
D1_016 COMP_016 watch series 10 jet black watch series 10 jet black aluminium 0.83
D1_017 COMP_017 galaxy watch7 green galaxy watch 7 green 0.97
D1_018 COMP_018 xperia 1 vi black xperia 1 vi black 1.00
D1_019 COMP_019 edge 50 pro black edge 50 pro black 1.00
D1_020 COMP_020 phone 3a black phone 3a black 1.00
linked: 20 correct: 20 client rows with no candidate: []
Blocking cut the work from 400 possible pairs to 28. Taking the best-scoring candidate in each block linked all 20 client rows, and all 20 links agree with the hand labels. The extracted storage values also agree with the competitor’s own storage_capacity column on the 15 rows where the competitor lists storage (15 of 15 agree), and the extracted brand agrees with its manufacturer column on all 20.
So the method works on this tidy example, and that is all the result shows. It is not evidence of accuracy in general: the brand dictionary was written by looking at these 20 names, every row has a true partner, and the names are clean. Twelve of the 20 pairs are identical once case, punctuation, brand and storage are stripped. A note on the dictionary itself: one of the brands is nothing, an ordinary English word, so a brand list like this needs care on scraped titles.
Step 4: where it breaks
The interesting question is how close the wrong answers came. Three checks:
cols = ["product_id", "ref_id", "words_a", "words_b", "score"]
same = lambda df: df.product_id.str[-3:] == df.ref_id.str[-3:]
# 1. Inside the blocks: the closest WRONG candidates
print(pairs[~same(pairs)].nlargest(3, "score")[cols].to_string(index=False, float_format="%.2f"))
# 2. No blocking: every client row against all 20 competitor rows (400 pairs)
allp = client.merge(comp, how="cross", suffixes=("_a", "_b"))
allp["score"] = [SequenceMatcher(None, a, b).ratio() for a, b in zip(allp.words_a, allp.words_b)]
top = allp.sort_values("score", ascending=False).drop_duplicates("product_id")
print("\nno blocking, best candidate correct:", same(top).sum(), "of 20")
print(allp[~same(allp)].nlargest(3, "score")[["product_name", "item_name", "score"]]
.to_string(index=False, float_format="%.2f"))
# 3. What if the competitor does not stock the 256GB iPhone 16 Pro?
missing = allp[(allp.ref_id != "COMP_001") & (allp.product_id == "D1_001")]
print("\nD1_001 without COMP_001 -> best:", missing.nlargest(1, "score")[["item_name", "score"]]
.to_string(index=False, header=False, float_format="%.2f"))
blocked = pairs[(pairs.product_id == "D1_001") & (pairs.ref_id != "COMP_001")]
print("with brand|storage blocking ->", blocked[["item_name", "score"]]
.to_string(index=False, header=False, float_format="%.2f"))
Output:
product_id ref_id words_a words_b score
D1_012 COMP_006 galaxy tab s9 fe lavender galaxy a55 5g ice blue 0.60
D1_006 COMP_005 galaxy a55 5g iceblue galaxy s25 mint green 0.57
D1_006 COMP_012 galaxy a55 5g iceblue galaxy tab s9 fe lavender 0.57
no blocking, best candidate correct: 20 of 20
product_name item_name score
iPhone 16 Pro 256GB Natural Titanium Apple iPhone 16 Pro 128 GB - Black Titanium 0.83
iPhone 16 Pro 128GB Black Titanium Apple iPhone 16 Pro 256 GB - Natural Titanium 0.83
iPhone 16 Pro 256GB Natural Titanium Apple iPhone 16 Pro Max 512 GB - Desert Titanium 0.79
D1_001 without COMP_001 -> best: Apple iPhone 16 Pro 128 GB - Black Titanium 0.83
with brand|storage blocking -> Apple iPad Air 11-inch M2 256 GB - Blue 0.26
What this shows:
- The margin is thin. Inside the blocks, the worst correct pair scored 0.62 (
WH-1000XM5 BlackagainstWH-1000XM5 Wireless Headphones - Black, where the competitor added two descriptive words) and the best wrong pair scored 0.60 (a Galaxy tablet against a Galaxy phone that share a brand and storage size). Any threshold between them is tuned on the same 20 rows it is judged on. - Variants score like matches. Without blocking, best-candidate selection still got 20 of 20, but only because every true partner was present. The wrong pairs at the top of the list are other storage and colour variants of the iPhone 16 Pro, plus the Pro Max, and at 0.83 they score higher than three of the 20 correct pairs.
- A missing product exposes it. Remove the competitor’s 256GB iPhone 16 Pro and, without blocking, the client’s 256GB model links to the 128GB model at 0.83, a score that passes any threshold low enough to keep the correct pairs. That is a price comparison between two different products. With storage in the blocking key, the only remaining candidate is an iPad at 0.26, which a threshold rejects, and the row correctly ends up unmatched.
This is the central trap for price comparison: variants, because a match to the wrong variant produces a wrong price gap with no visible error. Storage, colour, pack size, bundle contents and condition all change the price, and a name similarity score treats them as minor differences in spelling. Extract them and make them hard constraints. Colour is the awkward one: here White and Polar White were judged the same item by hand, and a strict colour key would have split them.
Which string method you use is a separate choice, and it is not neutral either. In our synthetic benchmark of company names, methods differed in which errors they recovered: Jaro-Winkler found 73% of matches with a typo, against 1% for normalized exact matching; token overlap was much better on swapped word order; character trigrams found 85% of matches with a dropped word, where no other method passed 7%. The details are in deterministic vs probabilistic matching, and how to choose a matching algorithm turns that into a decision procedure.
Step 5: compute the price differences
With pairs in hand, the price comparison is a join and a subtraction, once both prices are in one currency. The rate below is made up for illustration; use a dated rate from your own source.
RATE = 1.10 # ASSUMED USD per EUR, made up for this example. Use your own dated rate.
m = best[best.score >= 0.61].copy() # between best wrong (0.60) and worst correct (0.62): tuned on these same rows
m["client_eur"] = m.price_amt_a
m["comp_usd"] = m.price_amt_b
m["comp_in_eur"] = (m.comp_usd / RATE).round(2)
m["gap_eur"] = (m.client_eur - m.comp_in_eur).round(2)
m["gap_pct"] = (100 * m.gap_eur / m.comp_in_eur).round(1)
print(m[["product_id", "ref_id", "client_eur", "comp_usd", "comp_in_eur", "gap_eur", "gap_pct"]]
.sort_values("product_id").head(6).to_string(index=False))
print("pairs:", len(m), " identical nominal amounts:", (m.client_eur == m.comp_usd).sum())
Output:
product_id ref_id client_eur comp_usd comp_in_eur gap_eur gap_pct
D1_001 COMP_001 1099.0 1099.0 999.09 99.91 10.0
D1_002 COMP_002 999.0 999.0 908.18 90.82 10.0
D1_003 COMP_003 1449.0 1449.0 1317.27 131.73 10.0
D1_004 COMP_004 1399.0 1399.0 1271.82 127.18 10.0
D1_005 COMP_005 899.0 899.0 817.27 81.73 10.0
D1_006 COMP_006 449.0 449.0 408.18 40.82 10.0
pairs: 20 identical nominal amounts: 20
All 20 matched pairs carry the same number on both sides, in different currencies. Compare the raw numbers and the answer is “no price difference anywhere”. Convert at the assumed rate and every product appears 10.0% more expensive on the client side, which is the exchange rate, not anyone’s pricing decision. In this fixture the currency is the whole story. In real data it is one of several things to put on the same basis before subtracting: currency, whether tax is included, shipping, and the date each price was captured.
What we found, in one table
| Step | Result |
|---|---|
| Price parsing | 40 / 40 prices parsed; client in EUR, competitor in USD |
| Shared code | 20 / 20 rows linked by identical code; 19 / 20 codes fail the GS1 check digit on each side |
| Name matching with brand + storage + size blocking | 28 pairs scored instead of 400; 20 linked, 20 correct by hand label |
| Closest wrong pair inside blocks vs worst correct pair | 0.60 vs 0.62 |
| Without blocking, after removing one true partner | links the 256GB phone to the 128GB variant at 0.83 |
| Price gap | nominal amounts identical in all 20 pairs; any gap comes from the exchange rate |
20 rows, one hand labelling, every row has a true partner: a demonstration of failure modes, not an accuracy estimate.
To evaluate this on your own data, label a sample of pairs that covers each product family and includes products the other side does not carry, and measure precision and recall as described in how to measure matching quality.
Where Match Data Studio fits
Match Data Studio runs the same order of operations as a pipeline. Stage 1 normalises columns, applies transformations, and runs string pre-filters (exact, prefix, contains) and numeric percentage-difference pre-filters without any AI. Stage 2 runs AI attribute extraction and gemini-embedding-001 embeddings only on rows that survived, stage 3 scores candidate pairs by cosine similarity against thresholds and can ask gemini-2.5-flash-lite to confirm a pair, and stage 4 writes a results CSV. None of those AI steps were run for this post, so it makes no accuracy claim for them.
Keep reading
- Competitive intelligence from scraped data with record matching
- How to set up blocking keys to speed up large matching jobs
- How to choose the right matching algorithm for your data
- How to measure matching quality: precision, recall, and F1 for data teams
- Why SQL and pandas can’t accurately match retail products — and what can
- From AI scraping to AI matching — building the data pipeline for competitive analysis
Ready to match your data?
Create a free account, upload your CSV files, and configure matching in plain English. 100 free credits — no credit card required.
Free account No credit card Match your files in minutes