A demand planner asks a simple question: how many units of item X sold at retailer Y last week? The point-of-sale rows exist. The shipment history exists. The answer can still come out wrong, because the retailer’s feed, the syndicated data and the ERP each identify item X with a different code, and the join between them silently drops whatever does not line up.

Call it the translation tax: every forecast is only as complete as the crosswalk that translates one system’s product identity into another’s. It is a product-identity problem before it is a forecasting problem, and the first layer of the fix is deterministic code that runs before anyone reaches for fuzzy matching.

One item, three identities

Here is a hypothetical item as three systems might record it. The GTIN is GS1’s own published example number, used as a stand-in; the other codes are invented.

Hypothetical example. Only the GTIN digits come from GS1's documentation.
SourceWho assigns the keyKey as delivered (hypothetical)Description as delivered (hypothetical)
ERP item masterYouFG-10042PNT BTR CRNCHY 16OZ
ERP item master, barcode fieldYou614141123452-
Retailer POS feedRetailer item number + GTINRTL-558201 / 0614141123452Crunchy Peanut Butter 16 oz
Syndicated data extractData vendor, UPC delivered in 14-digit GTIN form00614141123452PEANUT BUTTER CRUNCHY 16OZ JAR

The syndicated row matters because of how those vendors describe their data. NIQ describes its Retail Measurement Services as using point-of-sale data to capture in-store sales, pricing, distribution and promotion activity, and Circana (formerly IRI) describes its point-of-sale data as measuring sales, share and price down to the UPC or store level. If your crosswalk cannot connect your internal item code to those UPCs, that data cannot be attributed to your item.

The three GTIN strings in the table are the same item. Compared as raw strings, they are three different keys.

The deterministic backbone: GTINs done properly

What GS1 actually specifies

A GTIN is 8, 12, 13 or 14 digits long (GTIN-8, GTIN-12, GTIN-13, GTIN-14); the 12-digit UPC-A form is a GTIN-12. Per the GS1 General Specifications (Release 26.0):

  • Shorter GTINs are left-padded with zeros to 14 digits, and those zeros are filler: their presence or absence does not change the GTIN’s value. A GTIN-12 gets two zeros, a GTIN-13 one, a GTIN-8 six.
  • GS1 recommends storing GTINs as 14-digit numbers in databases. A field may hold them with or without the zeros, but the format must be consistent within the field.
  • Message formats disagree: GS1 XML requires the full 14 digits, while EANCOM requires the non-significant zeros removed. If two partners exchange files built for different standards, the same item arrives in two shapes.
  • A GTIN-12 can start with zeros that are part of the number and must be kept. Padding to 14 keeps them; stripping “leading zeros” to tidy a column destroys them.

The check digit

The last digit of every GTIN is computed from the others: starting next to the check digit and moving left, multiply alternately by 3 and 1, sum, and subtract from the nearest equal or higher multiple of ten. The same algorithm covers all fixed-length numeric GS1 keys. This standard-library code reproduces the GS1 worked examples shown below:

def check_digit(body: str) -> int:
    """GS1 check digit for the digits WITHOUT the check digit (weights 3,1,3,1 from the right)."""
    total = sum(int(d) * (3 if i % 2 == 0 else 1) for i, d in enumerate(reversed(body)))
    return (10 - total % 10) % 10

def is_valid_gtin(s: str) -> bool:
    return s.isdigit() and len(s) in (8, 12, 13, 14) and check_digit(s[:-1]) == int(s[-1])

def to_gtin14(s: str) -> str:
    if not is_valid_gtin(s):
        raise ValueError(f"not a valid GTIN: {s!r}")
    return s.zfill(14)

print(check_digit("629104150021"))       # GS1 worked example
print(check_digit("061414112345"))       # GS1 worked example
print(is_valid_gtin("6291041500213"))
print(is_valid_gtin("6291041500214"))    # last digit mistyped
print(to_gtin14("0614141123452"))        # 13-digit form
print(to_gtin14("614141123452"))         # 12-digit form, same item

We ran it on Python 3.13 (standard library only):

3
2
True
False
00614141123452
00614141123452

The 12-digit and 13-digit forms of the same item collapse to one 14-digit key. That 14-digit value is the join key every feed should be converted to before any comparison.

How leading zeros disappear, and how to get them back

One way GTINs break has nothing to do with typos: a tool decides the column is numeric. You can reproduce it in two lines, and then repair it with a normalizer that pads to 14 and re-checks the check digit (this block reuses check_digit from above):

code = "0614141123452"            # GS1's example GTIN-13, stored as text
as_number = int(code)             # what happens when a tool parses the column as a number
print(as_number, len(str(as_number)), "digits")
print(f"{float(code):.6g}")       # and if it is then displayed in 6 significant digits

def normalise(value) -> str:
    """Digits only, left-pad to 14, then insist the check digit still holds."""
    digits = "".join(ch for ch in str(value) if ch.isdigit())
    if not 8 <= len(digits) <= 14:   # too short or too long: review it, don't guess
        raise ValueError(f"bad length: {value!r}")
    g14 = digits.zfill(14)
    if check_digit(g14[:-1]) != int(g14[-1]):
        raise ValueError(f"check digit fails: {value!r}")
    return g14

feeds = {"erp (12 digits)": "614141123452",
         "retailer (13 digits)": "0614141123452",
         "parsed as a number": as_number,
         "with a stray space": " 0614141123452"}
for src, value in feeds.items():
    print(f"{src:22} {value!r:18} -> {normalise(value)}")
print("distinct raw values:", len({str(v) for v in feeds.values()}))
print("distinct keys:      ", len({normalise(v) for v in feeds.values()}))
614141123452 12 digits
6.14141e+11
erp (12 digits)        '614141123452'     -> 00614141123452
retailer (13 digits)   '0614141123452'    -> 00614141123452
parsed as a number     614141123452       -> 00614141123452
with a stray space     ' 0614141123452'   -> 00614141123452
distinct raw values: 3
distinct keys:       1

Two lessons. Lost zeros are recoverable, because padding to 14 restores them and the check digit is unaffected by leading zeros. Lost significant digits are not: once a value has been written out in a rounded form like the 6.14141e+11 above, the digits are gone, and no normalizer brings them back. Read identifier columns as text at every step (dtype=str in pandas, text-formatted columns in a spreadsheet), and keep the raw value next to the normalized one so failures can be traced. The data cleaning steps most people skip applies here as much as to names and addresses.

What a check digit catches, and what it cannot

Validation is not verification. This block (again reusing the functions above) tries every single-digit typo and every swap of neighbouring digits on GS1’s example number:

good = "0614141123452"
# 1. Every possible single-digit substitution
typos = [good[:i] + d + good[i+1:] for i in range(13) for d in "0123456789" if d != good[i]]
print("single-digit typos:", len(typos), "| still valid:", sum(map(is_valid_gtin, typos)))

# 2. Every swap of two neighbouring (different) digits
swaps = [good[:i] + good[i+1] + good[i] + good[i+2:] for i in range(12) if good[i] != good[i+1]]
print("neighbour swaps:", len(swaps), "| still valid:", [s for s in swaps if is_valid_gtin(s)])

# 3. For any 12-digit body, exactly one of the ten possible last digits validates
bodies = [f"{n:012d}" for n in range(1000)]
passes = sum(is_valid_gtin(b + str(d)) for b in bodies for d in range(10))
print("candidates:", len(bodies) * 10, "| valid:", passes)
single-digit typos: 117 | still valid: 0
neighbour swaps: 11 | still valid: ['0164141123452']
candidates: 10000 | valid: 1000

Every single-digit substitution failed the check. One of the eleven neighbour swaps passed: swapping 6 and 1, two digits that differ by 5, leaves the weighted sum unchanged modulo 10. And a check digit only says a number is well formed. A GTIN that belongs to a different product (the wrong row copied into the mapping, a sibling flavor’s code) passes validation every time. Catching that needs a second field: the description, the pack size, the brand.

Many-to-one: pack levels and GTIN-14

GS1 prints two GTIN-14 numbers for the item above: 10614141123459 (indicator 1) and 80614141123458 (indicator 8). The code below reproduces both by putting the indicator in front of the GTIN-13’s first 12 digits and computing a new check digit. The grouping key (digits 2 to 13) is our own convention for linking such codes back to the unit, not something GS1 defines:

def gtin14_with_indicator(gtin13: str, indicator: str) -> str:
    body = indicator + gtin13[:12]           # drop the old check digit, prefix indicator
    return body + str(check_digit(body))

base = "0614141123452"
print(gtin14_with_indicator(base, "1"), gtin14_with_indicator(base, "8"))

def grouping_key(code: str) -> str:
    """Digits 2-13 of the 14-digit form: indicator and check digit removed."""
    return to_gtin14(code)[1:13]

for code in ["614141123452", "0614141123452", "10614141123459", "80614141123458"]:
    print(f"{code:>14} -> {to_gtin14(code)} group {grouping_key(code)}")
10614141123459 80614141123458
  614141123452 -> 00614141123452 group 061414112345
 0614141123452 -> 00614141123452 group 061414112345
10614141123459 -> 10614141123459 group 061414112345
80614141123458 -> 80614141123458 group 061414112345

Two cautions. First, GS1 is explicit that “the indicators have no meaning”. Common usage often treats the digit as a packaging level, but by rule neither 1 nor 8 denotes a pack level. The quantity in each pack level must come from your item master, not from the digit. Second, the grouping key only links codes that were built from the unit’s number this way. It cannot find a pack-level code numbered any other way, and codes that share a key still need confirming against master data, so treat it as a candidate link, not as proof.

Conversion factors belong in the crosswalk, not in the forecast code:

Hypothetical crosswalk rows. A variety pack with one jar of each of four flavors is one unit of each item, not a quarter of one.
Retailer item (hypothetical)Link typeInternal itemUnits of the internal item per sale
Crunchy PB 16 ozDirectFG-100421
Crunchy PB 16 oz, 2-packMultipackFG-100422
PB variety 4-pack (4 flavors)BundleFG-10042 and three other items1 of each
Crunchy PB 40 ozDifferent itemFG-100431

When the key is missing or untrustworthy

Some feeds carry no GTIN at all, carry only a retailer item number, or carry GTINs you cannot trust. Then the link has to come from attributes: brand, description, size, unit of measure, pack count. That is the switch from rules to scoring described in deterministic vs probabilistic matching, and the choice of method depends on what kind of differences your descriptions have, covered in how to choose a matching algorithm.

There is measured evidence on how string methods behave, with an important caveat: it comes from our own offline benchmark on synthetic company names, not CPG product descriptions. On that data (one run, 700 true matches among 1,150,000 possible pairs):

Share of true matches found, by method and error type, from the benchmark's published results.
Finding (synthetic company names)Recall
Raw exact match on the name0.414 (precision 1.000)
Exact match after normalization (case, punctuation, suffixes, abbreviations)0.717 (precision 1.000)
Single-character typos: normalized exact match vs Jaro-Winkler0.01 vs 0.73
Word-order swaps: Jaro-Winkler vs token overlap0.49 vs 0.81
Dropped generic word: character 3-gram TF-IDF (the only method above 0.07)0.85
Acronyms (initials only), every method tested, n=300.00

Synthetic data, one run, no confidence intervals. Differences of a few points are not meaningful.

Two things carry over as hypotheses, not results. Normalization did a lot of the work before any similarity score was involved, which is the same lesson as the GTIN padding above. And no string method found initials-style matches; “PB” for “peanut butter” has the same shape, so expect abbreviation dictionaries, extra fields (size, brand) or a semantic model to be needed there. Test on your own labelled pairs before you trust either.

A checklist for a product crosswalk

  1. One golden key per item. Pick the internal item code as the record’s identity and attach every external key to it: 14-digit GTINs at each pack level, retailer item numbers per retailer, and the UPCs your syndicated extracts use.
  2. Normalize at the edge. Convert every incoming GTIN to 14-digit text with the check digit verified before it touches a join. Keep the raw value and the reason for any rejection.
  3. Model many-to-one explicitly. Store pack levels and bundles as rows with a unit conversion factor, sourced from master data rather than inferred from indicator digits.
  4. Version the links. Give every link a source and an effective-from date, and close links rather than overwrite them, so last year’s sales still resolve to the item they were at the time.
  5. Agree formats with partners. A data contract with each retail partner can fix the identifier field, its format (14 digits as text, or the shape their standard requires) and how rejected rows are reported back. That can reduce the translation work; it does not remove it.
  6. Review exceptions, not everything. Route check-digit failures, rows with no usable key, and attribute-based links below your threshold to a person. Understanding similarity thresholds covers where to draw that line.
  7. Count what fell out. Report how many POS rows did not resolve each period. An unresolved row is not zero demand; it is unknown demand.

The scale argument is arithmetic. In a hypothetical portfolio of 400 items sold through 12 retailers with 3 data sources each, there are 400 x 12 x 3 = 14,400 item-retailer-source links. If each link can break independently of the others (an assumption, not a measurement), every one of them is a separate thing to check.

Honest limits

  • The GTIN rules here cover structure and check digits only. They say nothing about who owns a number, whether it is assigned to the product you think, or GS1’s rules on allocating and reusing numbers.
  • A valid check digit does not mean a correct mapping. Wrong-but-valid GTINs pass every check in this post.
  • The string-matching numbers are from synthetic company names. Nothing in this post measures accuracy on product data, and no coverage, time or cost figures are claimed.
  • The code assumes digit-only identifiers. Retailer item numbers follow each retailer’s own format and need their own validation.

Where Match Data Studio fits

Match Data Studio matches CSV datasets of up to 50,000 rows each. String pre-filters (exact, prefix or contains rules) and numeric pre-filters (percentage difference) run first, without AI. AI attribute extraction and gemini-embedding-001 embeddings then run only on rows that survived, candidate pairs are scored by cosine similarity against thresholds, gemini-2.5-flash-lite can optionally confirm pairs, and the output is a results CSV. Normalize GTIN columns to 14 digits before uploading so an exact rule compares like with like.

Start matching your product data →

Keep reading