What should a document extraction confidence score actually tell you?

A document processing pipeline returns a value and a confidence score for every extracted field. The implementation seems straightforward: Set a threshold, allow high-confidence values to continue automatically, and route anything below that threshold to human review. Fields scoring below 0.85 enter a review queue; everything else moves into the database, ERP, or downstream application.

Then a value with 94 percent confidence turns out to be wrong — not slightly wrong, but pulled from an entirely different field in the document.

The model was confident, but the application has no way to determine what it was confident about. A value and a confidence number are returned, but nothing explains why the value received that number. That’s the limitation of many document extraction confidence scores: They tell you how decisive the model was when selecting an answer, but not whether the source document supports that answer.

What is a document extraction confidence score?

A document extraction confidence score is a numerical signal indicating how certain an extraction system is about a returned value. In many systems, that score is derived from the model’s internal ranking of possible answers — if one candidate ranks significantly higher than the alternatives, the model returns a high confidence score.

That’s useful when a model is genuinely unsure — a model choosing between three plausible dates should expose that uncertainty. But model certainty and extraction correctness aren’t the same thing: A model can be highly decisive and still choose the wrong candidate.

This distinction becomes important when confidence scores control automation. Once a threshold determines which values proceed without review, the score is no longer just diagnostic metadata — it becomes part of the application’s decision logic. For that logic to be reliable, developers need more than a probability. They need to know whether the extracted value is supported by evidence in the document.

The high-confidence extraction that came from the wrong column

Consider a Making Home Affordable Program form containing a three-column financial table:

• Monthly Household Income
• Total Assets
• Total Monthly Expenses

The labels are visually similar, the values share the same currency format, and the columns appear next to one another on the same page.

The extraction schema requests a field called total_gross_income. The correct value in the income column is $3,450. But the extraction returns $6,500 — a value taken from the adjacent Total Assets column.

The returned value looks plausible. Its data type is correct, it appears in the expected part of the document, and the model assigns it a high confidence score. In a score-only workflow, the value passes through automatically, and a reviewer presented with $6,500 and a high score has little reason to question it without manually searching the document.

Now add spatial grounding.

Instead of returning only the extracted value, the system ties it to the precise region of the page used as its source. When a reviewer hovers over total_gross_income, the corresponding bounding box is highlighted in the original document — and the highlight lands in the Total Assets column. The error becomes immediately visible.

Spatial grounding doesn’t merely tell the reviewer that something may be wrong. It shows where the extraction came from, making it possible to compare the returned value, requested field, and source evidence directly. A confidence score should account for that relationship: not just whether the system strongly preferred an answer, but whether the cited document evidence supports it.

From model decisiveness to source-text support

Nutrient Data Extraction API grounds extracted values in specific regions of the source document. Each result can be connected to page context and bounding-box coordinates, making it possible to inspect the evidence behind the extraction. The confidence score reflects whether that cited evidence supports the returned value.

This is different from measuring how strongly a model preferred one candidate over the alternatives. A model may be decisive and still select a value from the wrong field, column, or section of the document. Source-grounded confidence makes a more useful claim: The document evidence cited for this extraction supports the returned value.

A high score paired with a correctly grounded region gives the application confidence to proceed automatically. But as the RMA example shows, a high score alone doesn’t confirm the right region was used — the bounding box is what makes that verifiable. For teams routing extracted data by threshold, grounding adds a second layer, revealing not just how confident the model was, but where it looked.

Why source-grounded confidence is different

A confidence score tells you how sure the model was. A grounded confidence score also tells you what the model was sure about — and both are needed.

A score-only extraction API response for this field might look like the following:

{
  "field": "total_gross_income",
  "value": 6500,
  "confidence": 0.97
}

The response tells the application that the model strongly preferred this answer. But it leaves several important questions unresolved:

• Which part of the document produced the value?
• Did that region contain income, assets, or expenses?
• Was the value taken from a field label, a table cell, or nearby text?
• Can a reviewer verify the result without searching the page manually?

Here’s the actual Nutrient Data Extraction API response for the same field — from a real run against the Making Home Affordable RMA form before the schema description was corrected:

{
  "data": {
    "total_gross_income": {
      "amount": 6500,
      "iso_4217_currency_code": "USD"
    }
  },
  "metadata": {
    "total_gross_income": {
      "amount": {
        "confidence": 0.97,
        "confidenceComponents": {
          "groundingScore": 0.95,
          "formatScore": 1.0
        },
        "pageNumber": 2,
        "bbox": { "x": 4404, "y": 2975, "width": 258, "height": 62 }
      }
    }
  }
}

The confidence is 0.97 — and the extraction is wrong. The value came from the Total Assets column, not Monthly Household Income. The score alone wouldn’t have flagged this for review.

What the bbox does flag: When a developer or reviewer hovers over total_gross_income in the extraction UI, the bounding box highlights the exact cell the model read. That highlight lands in the wrong column, and the error is immediately visible — not from the score, but from the spatial evidence. The schema description was updated to explicitly anchor the field to the leftmost column, and the corrected extraction returned $3,450 with the highlight landing in the right place.

How Nutrient’s confidence scores changed

The example above reflects the problem with the previous generation of extraction confidence scores: The score was a weighted average computed internally during ranking. It was useful for ordering candidate answers, but it wasn’t meaningful as a signal to an API user. A high score indicated that one candidate ranked decisively above the alternatives — nothing more.

The updated confidence score is based on NLI groundedness (natural language inference). For each extracted field, the model evaluates whether the source text it cited actually supports the value it returned. The question shifts from “How strongly did the model prefer this answer?” to “Does the evidence in the document support this value?”

The practical difference: A high score now means the cited source text backs the returned value. A low score is a genuine signal to review — not an artifact of how close the candidates were.

The new scoring was calibrated against 194 SEC filings — dense, structured financial documents that stress-test extraction accuracy across a range of table layouts, numeric fields, and ambiguous labels. The English grounding model(opens in a new tab) is open source. The multilingual grounding model(opens in a new tab) powers Nutrient Data Extraction API and supports confidence scores across non-English documents.

For teams using thresholds to route extraction results, this matters. A threshold applied to the old score was drawing a line on an internal ranking metric. The same threshold applied to the NLI groundedness score is a statement about evidence quality: Results above the threshold have source support; results below it warrant a second look.

A second example: A signature isn’t a printed name

The same problem appears in an Indiana Certificate of Live Birth application.

In the applicant signature block, the form contains visually adjacent regions for a handwritten signature and a printed name. The requested field is printed_name. The system may return “O.G. Bennet,” interpreted from the cursive signature, when the correct value in the printed-name field is “Olivia Grace Bennett.”

Both regions belong to the same applicant and appear close together. A text-only result may not reveal why the wrong value was selected — but the bounding box does. Highlighting the source region shows whether the extraction came from the signature line or the printed-name field, making a mistake that looks plausible in JSON immediately understandable on the page.

These are the errors that conventional validation often misses: The output contains a string, the field isn’t empty, the response conforms to the schema — but the value came from the wrong evidence.

Confidence thresholds become more meaningful

Confidence thresholds are commonly used to divide straight-through processing from exception handling.

A simplified workflow might look like this:

if (result.confidence >= 0.85) {
  await continueAutomatically(result);
} else {
  await sendToHumanReview({
    result,
    page: result.source.page,
    region: result.source.boundingBox,
  });
}

The code is easy to write. The difficult question is what confidence >= 0.85 actually guarantees.

When the score represents only the model’s internal preference among candidates, the threshold tells you that the model made a decisive selection. It doesn’t guarantee that the selected source corresponds to the field the workflow requested.

When confidence is tied to source-text support, the threshold becomes more useful. It can help answer whether the cited evidence backs the returned value.

Confidence scoring is one layer in a reliable extraction workflow — grounding makes it more actionable, but it doesn’t replace business-rule validation, entity checks, or human review for high-impact fields.

Building document automation you can inspect

Document extraction isn’t valuable simply because a model is confident. It’s valuable when the returned data is reliable enough for the next system — or the next person — to act on. That requires outputs developers can inspect rather than scores they’re expected to accept.

As document volume increases, human review needs to become exception-based. When an extraction error arrives as a highlighted region on the source page, correction can take seconds. When it arrives as an unexplained value with a probability, the reviewer may need to read the whole document to understand what happened. That difference determines how much human review a pipeline can support and whether high-confidence errors remain hidden.

Source grounding creates the link between the structured result and the original document. Evidence-aware confidence makes that link part of the score itself. A high score should mean more than “the model strongly preferred this answer” — it should mean the evidence in the document supports the value.

That’s a confidence score that can be used to automate decisions, accelerate human review, and understand an extraction when something goes wrong.

FAQ