PDF data extraction for matching: schema, checks and cost
Plan PDF extraction you can check: a unit-normalised schema, validation code, a gold-set test and Gemini cost arithmetic from the documented token rules.
Here is a hypothetical case. A procurement team has a catalog row that says 2-inch stainless steel gate valve, and a folder of supplier spec sheets in PDF. Two suppliers offer something that fits that description, but only the spec sheets say which one is rated for the pressure and temperature the team needs, which material grade it uses, and what the manufacturer’s part number is. To match catalog rows to supplier products, those values have to come out of the PDFs and into columns.
Extraction with a multimodal model can do that. What it does not do is tell you whether the values are right. This post is about the parts you control: deciding whether you need extraction, designing the schema, validating every record, measuring accuracy on your own documents, and estimating cost before you run. The example values are made up; the code is real and the outputs below come from running it.
First decide whether you need extraction at all
Extraction adds cost and a new source of errors, so use it only for fields that decide the match and exist nowhere else.
| Situation | Approach |
|---|---|
| The deciding fields already exist as columns in both files (part number, manufacturer, size) | No extraction. Match on the columns. |
| PDFs have a text layer and one fixed layout (one supplier's template) | Pull the text with a PDF library and parse it with rules; validate the output the same way as below. |
| Layouts vary by supplier, pages are scanned, or values sit in tables and footnotes | Model-based extraction against a fixed schema. |
| You only need to know what kind of document each file is | Classification only: one field, a short list of allowed labels and an 'other' option. |
If a folder mixes document types, classify first, so that spec sheets, certificates and invoices each get their own schema. How AI classifies PDFs covers that step, and AI extraction vs AI enrichment explains the difference between pulling a value that is on the page and generating one that is not.
Step 1: a schema for one document type
Write one schema per document type. This one is for a valve spec sheet. Gemini’s structured output can make the response follow a JSON Schema with string, number, integer, boolean, object, array and null types. Not every JSON Schema feature is supported, so check the structured output documentation for the keywords you use.
{
"type": "object",
"properties": {
"document_type": {"type": "string", "enum": ["valve_spec_sheet", "other"]},
"manufacturer": {"type": ["string", "null"]},
"part_number": {"type": ["string", "null"], "description": "exactly as printed"},
"valve_type": {"type": ["string", "null"], "description": "gate, globe, ball, butterfly or check"},
"size": {"type": ["object", "null"], "properties": {
"value": {"type": "number"}, "unit": {"type": "string", "enum": ["mm", "in"]}},
"required": ["value", "unit"]},
"pressure": {"type": ["object", "null"], "properties": {
"value": {"type": "number"}, "unit": {"type": "string", "enum": ["bar", "psi", "kPa"]}},
"required": ["value", "unit"]},
"pressure_class": {"type": ["string", "null"], "description": "designation as printed, e.g. 'Class 150' or 'PN16'; never convert"},
"temp_min": {"type": ["object", "null"], "properties": {
"value": {"type": "number"}, "unit": {"type": "string", "enum": ["C", "F"]}},
"required": ["value", "unit"]},
"temp_max": {"type": ["object", "null"], "properties": {
"value": {"type": "number"}, "unit": {"type": "string", "enum": ["C", "F"]}},
"required": ["value", "unit"]},
"material": {"type": ["string", "null"]},
"certifications": {"type": "array", "items": {"type": "string"}},
"evidence": {"type": "string", "description": "short text copied from the page that supports the pressure and size values"},
"source_page": {"type": ["integer", "null"]}
},
"required": ["document_type", "manufacturer", "part_number", "valve_type", "size", "pressure",
"pressure_class", "temp_min", "temp_max", "material", "certifications", "evidence", "source_page"]
}
The design choices behind it:
- Value and unit as printed; convert in code. Ask the model to copy
2andin, not to compute 50.8 mm. Unit conversion is deterministic, so do it where you can test it. - Null means “not in the document”. Every field is required; most are also nullable. That gives the model an explicit way to abstain, and it lets you measure abstention separately from wrong answers (step 3).
- Rating labels stay as printed. If a sheet gives a label such as
PN16rather than a number with a unit, put it in its own text field and do not let it turn into a number. - An escape hatch for the wrong document.
document_typewith anothervalue stops a certificate from being forced into a spec-sheet shape. - Evidence and page number. A short copied snippet and a page number make human review fast.
Step 2: validate every record
Here is a validator for records shaped like that schema. It converts to canonical units (millimetres, bar, degrees Celsius), checks required fields and allowed values, and flags implausible numbers. Three conversions are exact by definition: 1 inch is 25.4 mm, 1 kPa is 0.01 bar, and °C = (°F − 32) × 5/9. The psi factor is derived (about 0.06894757 bar per psi); the code uses it rounded to 0.0689476. The range limits are placeholders; set them from your own catalog.
TO_MM = {"mm": 1.0, "in": 25.4} # 1 inch = 25.4 mm (exact)
TO_BAR = {"bar": 1.0, "psi": 0.0689476, "kpa": 0.01}
TO_C = {"c": lambda v: v, "f": lambda v: (v - 32) * 5 / 9}
RANGES = {"size_mm": (6, 1500), "pressure_bar": (1, 420), # placeholder limits:
"temp_min_c": (-200, 600), "temp_max_c": (-200, 600)} # set them from your catalog
REQUIRED = ["manufacturer", "part_number", "valve_type"]
VALVE_TYPES = {"gate", "globe", "ball", "butterfly", "check"}
def validate(rec):
"""Convert to canonical units; return (clean_record, problems)."""
out = {k: rec.get(k) for k in REQUIRED}
problems = [f"missing {k}" for k in REQUIRED if not rec.get(k)]
if rec.get("valve_type") and rec["valve_type"].lower() not in VALVE_TYPES:
problems.append(f"unknown valve_type {rec['valve_type']!r}")
for field, table, key in [("size", TO_MM, "size_mm"), ("pressure", TO_BAR, "pressure_bar"),
("temp_min", TO_C, "temp_min_c"), ("temp_max", TO_C, "temp_max_c")]:
q = rec.get(field)
if q is None:
continue
unit = str(q.get("unit", "")).lower()
if not isinstance(q.get("value"), (int, float)) or unit not in table:
problems.append(f"bad {field}: {q}")
continue
conv = table[unit]
out[key] = round(conv(q["value"]) if callable(conv) else q["value"] * conv, 2)
for key, (lo, hi) in RANGES.items():
if key in out and not lo <= out[key] <= hi:
problems.append(f"{key}={out[key]} outside {lo}..{hi}")
if out.get("temp_min_c", -1e9) > out.get("temp_max_c", 1e9):
problems.append("temp_min above temp_max")
return out, problems
Four toy records, made up to look like model output, each with a different problem except the first:
toy = [ # MADE-UP records, shaped like model output for a valve spec sheet
{"manufacturer": "Example Valve Co", "part_number": "EV-G50-316", "valve_type": "gate",
"size": {"value": 2, "unit": "in"}, "pressure": {"value": 150, "unit": "psi"},
"temp_min": {"value": -20, "unit": "F"}, "temp_max": {"value": 400, "unit": "F"}},
{"manufacturer": "Example Valve Co", "part_number": "", "valve_type": "ball",
"size": {"value": 50, "unit": "mm"}, "pressure": {"value": "PN16", "unit": "bar"}},
{"manufacturer": "Sample Flow Ltd", "part_number": "SF-B80", "valve_type": "Ball",
"size": {"value": 80, "unit": "mm"}, "pressure": {"value": 2500, "unit": "kPa"},
"temp_min": {"value": 120, "unit": "C"}, "temp_max": {"value": 90, "unit": "C"}},
{"manufacturer": "Sample Flow Ltd", "part_number": "SF-X1", "valve_type": "plug valve",
"size": {"value": 3, "unit": "inch"}, "pressure": {"value": 10000, "unit": "psi"}},
]
for rec in toy:
clean, problems = validate(rec)
print(clean)
print(" problems:", problems or "none")
Output:
{'manufacturer': 'Example Valve Co', 'part_number': 'EV-G50-316', 'valve_type': 'gate', 'size_mm': 50.8, 'pressure_bar': 10.34, 'temp_min_c': -28.89, 'temp_max_c': 204.44}
problems: none
{'manufacturer': 'Example Valve Co', 'part_number': '', 'valve_type': 'ball', 'size_mm': 50.0}
problems: ['missing part_number', "bad pressure: {'value': 'PN16', 'unit': 'bar'}"]
{'manufacturer': 'Sample Flow Ltd', 'part_number': 'SF-B80', 'valve_type': 'Ball', 'size_mm': 80.0, 'pressure_bar': 25.0, 'temp_min_c': 120, 'temp_max_c': 90}
problems: ['temp_min above temp_max']
{'manufacturer': 'Sample Flow Ltd', 'part_number': 'SF-X1', 'valve_type': 'plug valve', 'pressure_bar': 689.48}
problems: ["unknown valve_type 'plug valve'", "bad size: {'value': 3, 'unit': 'inch'}", 'pressure_bar=689.48 outside 1..420']
The second record has a rating label, PN16, in a numeric field. The third has a temperature range written backwards, which no type check catches. The fourth uses a unit spelling the schema’s enum should have prevented, which is the point: validate even when the schema says it cannot happen. An unknown valve type can also mean the allowed list is incomplete, so review it before discarding the record. Records with problems go to a review queue; they should not flow into matching unchecked.
Structured output does not remove the need for this step. Google’s documentation is direct about it: “While output is syntactically correct JSON, always validate values in your application.”
Step 3: measure extraction on your own gold set
Validation catches values that are impossible. It cannot catch values that are plausible and wrong. For that you need a gold set: a sample of your real documents where a person has typed the correct value for every field, or null when the field is not in the document.
Pick documents that cover every supplier layout, scanned and text PDFs, and single- and multi-page files. Keep the size in mind when reading results: if a field appears in 50 gold documents, each one moves that field’s accuracy by 2 percentage points.
Score each field separately, and separate four outcomes: correct, wrong, abstained (null where a value exists), and invented (a value where the gold set says there is none).
def same(gold, pred, rel_tol=0.01):
if isinstance(gold, (int, float)) and isinstance(pred, (int, float)):
return abs(gold - pred) <= rel_tol * max(abs(gold), 1e-9)
return str(gold).strip().lower() == str(pred).strip().lower()
def field_report(gold_set, predictions, fields):
"""gold_set / predictions: {doc_id: {field: value or None}}. None = not on the page / no answer."""
print(f"{'field':13}{'present':>8}{'correct':>8}{'wrong':>6}{'abstain':>8}{'invented':>9}{'accuracy':>9}")
for f in fields:
c = w = a = inv = 0
for doc, g in gold_set.items():
gv, pv = g.get(f), predictions.get(doc, {}).get(f)
if gv is None:
inv += pv is not None # value returned for something not in the document
elif pv is None:
a += 1 # model abstained on a field that exists
elif same(gv, pv):
c += 1
else:
w += 1
present = c + w + a # documents where the field really exists
print(f"{f:13}{present:>8}{c:>8}{w:>6}{a:>8}{inv:>9}{c / present:>9.0%}")
gold = { # TOY gold set: values typed by a person reading each (imaginary) PDF
"doc1": {"part_number": "EV-G50-316", "size_mm": 50.8, "pressure_bar": 10.34, "material": "316L"},
"doc2": {"part_number": "EV-B50", "size_mm": 50.0, "pressure_bar": None, "material": "CF8M"},
"doc3": {"part_number": "SF-B80", "size_mm": 80.0, "pressure_bar": 25.0, "material": None},
"doc4": {"part_number": "SF-X1", "size_mm": 76.2, "pressure_bar": 20.0, "material": "WCB"},
}
pred = { # TOY model output after validate() and unit conversion
"doc1": {"part_number": "EV-G50-316", "size_mm": 50.8, "pressure_bar": 10.3, "material": "316L SS"},
"doc2": {"part_number": "EV-B50", "size_mm": 50.0, "pressure_bar": 16.0, "material": "CF8M"},
"doc3": {"part_number": "SF-B8O", "size_mm": 80.0, "pressure_bar": 25.0, "material": None},
"doc4": {"part_number": "SF-X1", "size_mm": None, "pressure_bar": 20.0, "material": "WCB"},
}
field_report(gold, pred, ["part_number", "size_mm", "pressure_bar", "material"])
Output:
field present correct wrong abstain invented accuracy
part_number 4 3 1 0 0 75%
size_mm 4 3 0 1 0 75%
pressure_bar 3 3 0 0 1 100%
material 3 2 1 0 0 67%
These are toy numbers, invented to exercise each branch, and they say nothing about any model. What the toy run does show is why the four outcomes need separate counts:
pressure_barshows 100% accuracy on the documents where a pressure exists, yet it also has one invented value:doc2printed only a designation and the toy “model” turned it into 16 bar. Accuracy alone hides that.- The
part_numbererror is a letter O in place of a zero (SF-B8O). For matching, one wrong character in an identifier is as bad as a missing one. 316L SSagainst316Lcounts as wrong under exact comparison. Decide whether that is an extraction error or a normalisation rule you are missing, and write the rule down before you score, not after.- Numeric comparisons allow 1% relative tolerance, so a pressure of 10.3 against 10.34 counts as correct. Choose tolerances per field, from what the match needs.
Re-run the same gold set whenever you change the prompt, the schema or the model, and compare field by field.
Step 4: estimate cost before you run
Google’s document processing guide gives the building blocks: each PDF page counts as 258 tokens; text natively embedded in a PDF is extracted and passed to the model, and you are not charged for those native-text tokens; files can be up to 50MB or 1000 pages. The pricing page, as of October 2026, lists gemini-2.5-flash-lite at $0.10 per million input tokens and $0.40 per million output tokens, with thinking tokens included in the output price. The Batch API runs at 50% of the standard cost with a 24-hour target turnaround. Prices change; check the page before you budget.
The calculator below is arithmetic on those rules. Every input is an assumption you supply.
TOKENS_PER_PAGE = 258 # Gemini docs: each PDF page is equivalent to 258 tokens
PRICE_IN, PRICE_OUT = 0.10, 0.40 # gemini-2.5-flash-lite, USD per 1M tokens, standard, Oct 2026
BATCH_DISCOUNT = 0.50 # Batch API: 50% of the standard cost
MAX_PAGES, MAX_MB = 1000, 50 # per PDF file
def pdf_extraction_cost(docs, pages_per_doc, prompt_tokens, output_tokens, mb_per_doc, batch=False):
"""Arithmetic on YOUR assumptions. Native PDF text is not charged; page tokens are."""
if pages_per_doc > MAX_PAGES or mb_per_doc > MAX_MB:
raise ValueError("document over the 1000-page / 50MB limit: split it first")
tokens_in = docs * (pages_per_doc * TOKENS_PER_PAGE + prompt_tokens)
tokens_out = docs * output_tokens # output price includes thinking tokens
cost = tokens_in / 1e6 * PRICE_IN + tokens_out / 1e6 * PRICE_OUT
return tokens_in, tokens_out, cost * (BATCH_DISCOUNT if batch else 1)
# ASSUMPTIONS for this example only (not measured, not typical):
example = dict(docs=2000, pages_per_doc=4, prompt_tokens=400, output_tokens=300, mb_per_doc=2)
for batch in (False, True):
tin, tout, usd = pdf_extraction_cost(**example, batch=batch)
print(f"{'batch ' if batch else 'standard'} input {tin:>9,} tok output {tout:>7,} tok ${usd:.2f}")
Output:
standard input 2,864,000 tok output 600,000 tok $0.53
batch input 2,864,000 tok output 600,000 tok $0.26
| Assumption | Value |
|---|---|
| Documents | 2,000 |
| Pages per document | 4 |
| Prompt tokens per document (instructions + schema) | 400 |
| Output tokens per document, thinking included | 300 |
| Standard cost | $0.53 |
| Batch cost | $0.26 |
Change any assumption and the result changes. Measure your real counts first: the API's count_tokens call measures input before you send it.
Two cautions. The calculator assumes every page costs 258 tokens, so it is an upper bound on the page cost. The documentation says tokens from text natively embedded in a PDF are not charged, so a text-native PDF may cost less; check with count_tokens on a handful of your own files. And output tokens are the input you know least: if thinking is enabled, the output count includes it, so do not assume it is small.
Failure modes to check for
A checklist for reviewing a new document type or supplier:
- Scanned pages. No text layer, so there is no native text to pass along, only the page image. Include scanned files in the gold set.
- Rating labels read as numbers. A label such as
PN16ends up in a numeric field. - Units in the header, not the cell. A table column headed “Pressure (psi)” with bare numbers below.
- Ranges. “-20 to 400 °F” needs to become two fields with the same unit, in the right order.
- One sheet, many products. A spec sheet covering several sizes is several rows for matching, not one. Extract an array of variants or split the document.
- Tables across page breaks, and footnotes that change a value (“at 38 °C”, “for sizes above 4 in”).
- Look-alike characters in identifiers: O and 0, l and 1, S and 5.
- Decimal commas in documents from regions that use them.
- Superseded revisions. Two spec sheets for the same part with different dates; keep the revision as a field.
- Invented values for absent fields. Count them separately (step 3).
- Files over the limits (50MB or 1000 pages): split before sending.
From extracted fields to matches
Once each PDF is a validated row, treat the fields by type, the same way you would any other column:
| Field | Comparison |
|---|---|
| part_number | Exact after normalising case, spaces and hyphens; a deterministic rule |
| manufacturer | Normalised name, used as a blocking key |
| size_mm, valve_type | Exact equality: different sizes or types are different products |
| pressure_bar, temp ranges | Numeric tolerance, or 'meets or exceeds the requirement' |
| material, free-text description | String or semantic similarity, with review for borderline scores |
Identifiers and variant attributes (size, type) work best as hard rules or blocking keys; free text is where similarity scoring earns its place. Deterministic vs probabilistic matching explains the difference with a benchmark, and how to choose a matching algorithm turns it into a procedure for your data.
Where Match Data Studio fits
In Match Data Studio, a CSV column can be marked as a file column whose cells are public URLs (fetched by Gemini) or the names of uploaded files. An AI extraction step, using gemini-2.5-flash-lite, reads each file and writes the fields you ask for into new columns, only for rows that passed the earlier non-AI filters. Embeddings are text-only, so file columns are not embedded directly; to embed what a document says, extract it as text first. Datasets can have up to 50,000 rows each, and a sample run processes a small slice first. No extraction accuracy was measured for this post, so export a sample and run the validation and gold-set checks above on it.
Keep reading
- AI extraction vs AI enrichment: how structured data gets pulled from files
- Can Gemini read PDFs? How AI classifies PDFs
- Deterministic vs probabilistic matching
- How to measure matching quality: precision, recall, and F1 for data teams
- Extracting structured data from home inspection reports and insurance documents
Ready to match your data?
Create a free account, upload your CSV files, and configure matching in plain English. 100 free credits — no credit card required.
Free account No credit card Match your files in minutes