Classifying documents

Sorting incoming documents — is this an invoice, a contract, a resume, a bank statement? — is the first step in most document workflows. When you don’t have a fixed, pre-trained set of categories, you need zero-shot classification: you supply the candidate labels at call time and the model scores the document against exactly those.

This sample shows how to classify a document against your own set of candidate labels using Nutrient Java SDK. Classification combines two signals — the document’s text and its visual layout — and fuses them into a single ranked prediction. Both run locally.

Download sample

How Nutrient helps

Nutrient Java SDK runs the full zero-shot classification pipeline behind a single method call. The SDK handles:

Rendering the page to a bitmap at the resolution the image model expects
Scoring the document text against your candidate labels
Scoring the page image against your candidate labels
Fusing the two branches by the weights you choose into one ranked prediction
Serializing the result to JSON

The result is a predicted label plus the full ranked list of candidates with their probabilities — including the per-branch (text and image) contributions, so you can see why a label won.

Why two branches

Some document types are obvious from their text (a contract’s legal language), others from their layout (an invoice’s table of line items, an ID card’s photo block). Scoring both and fusing them is more robust than either alone. You control the balance with two weights — lean on text, lean on image, or weight them equally (the default).

Choosing where the text comes from

The text branch needs the document’s text. By default it derives the text from the document by running the extraction pipeline selected by Vision Settings (AdaptiveOcr for fast OCR, Icr for local models, VlmEnhancedIcr for VLM-enhanced); multi-page documents are extracted in full. This is the zero-config path.

To classify text you already have, call setText(...) — that text is scored directly and extraction is skipped. To run image-only and skip the text extraction cost entirely, set the text weight to 0 with setTextWeight(0f).

Classification requires the vision image classification feature in your license.

Multi-page documents

When you classify a multi-page document, every page contributes. The image branch scores each page independently; the text branch reads text from every page. Pages with a confident, decisive result count for more, while pages that look like boilerplate are softly silenced, so the most distinctive page drives the prediction.

When the text is derived from the document, the text branch reads the full structure of every page — paragraphs, tables, key-value regions, and form fields — in reading order, not just running paragraphs. This matters for invoices, statements, and forms, where the identifying signal often lives in tabular numbers or filled fields rather than prose.

Complete implementation

Specify a package name and create a new class:

package io.nutrient.Sample;

Import the classes used in the sample:

import io.nutrient.sdk.Document;
import io.nutrient.sdk.Vision;
import io.nutrient.sdk.requests.ClassificationRequest;
import io.nutrient.sdk.exceptions.NutrientException;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

Loading the document

Open the document in a try-with-resources block so resources are cleaned up after processing:

public class ClassifyDocument {
    public static void main(String[] args) {
        try (Document document = Document.open("input_forms_detection.pdf")) {

Configuring classification

Build a ClassificationRequest carrying your candidate labels (at least two are required), and tune the behavior on the document’s classification settings. This example derives the text from the document via the extraction pipeline and weights both branches equally:

            var classification = document.getSettings().getDocumentClassificationSettings();

            // Balance the text and image branches (defaults are 0.5 each).
            classification.setTextWeight(0.5f);
            classification.setImageWeight(0.5f);

            // Required: at least two candidate classes to score against (zero-shot).
            var request = new ClassificationRequest();
            request.addCandidateLabel("Invoice");
            request.addCandidateLabel("Contract");
            request.addCandidateLabel("Resume");
            request.addCandidateLabel("Bank statement");
            request.addCandidateLabel("ID card");

Describing each class

A bare label like “Invoice” works, but ambiguous or domain-specific classes match better when you describe them. Pass an optional description as the second argument — the label is the short name shown in your results; the description is longer context used only to improve matching. Adding a label that’s already on the request replaces its entry, so you can start bare and upgrade to descriptions:

            request.addCandidateLabel("Invoice", "An itemized bill listing goods or services and the total amount due");
            request.addCandidateLabel("Contract", "A legal agreement between parties stating terms and obligations");
            request.addCandidateLabel("Resume", "A summary of a person's work experience, skills, and education");
            request.addCandidateLabel("Bank statement", "A periodic summary of account transactions and balances");
            // "ID card" keeps its bare label — descriptions are optional per class.

Labels and descriptions are plain strings with no delimiter rules — commas, pipes, and newlines are all fine. The result still reports only the label; the description never appears in the output.

Choosing the label language

Classification runs fully offline in any language. The text branch picks a multilingual model automatically when the document or the labels aren’t English, and a dedicated English model when both are English — there’s nothing to configure, the language is detected from the text and the labels. A document that mixes languages is treated as multilingual and uses the multilingual model too.

Predictions are sharpest when the document and its labels share a language, so:

Prefer English labels. English documents with English labels take the dedicated English model (the strongest path), and English labels stay reliable even on non-English documents.
When the source text isn’t English, give the labels in the document’s language or fall back to English labels.
An English document with non-English labels is the lowest-confidence pairing — prefer one of the two options above.

A language mismatch — say an English document with non-English labels — still classifies correctly (the multilingual model is selected automatically), but with lower confidence. Keeping the document and labels in the same language, or using English labels for non-English documents, keeps the prediction sharp.

Classifying the document

Create a vision instance bound to the document with Vision.set(document), then call classify(request):

            Vision vision = Vision.set(document);
            String resultJson = vision.classify(request);

            Files.writeString(Path.of("output.json"), resultJson);
        } catch (NutrientException | IOException e) {
            System.err.println("Error: " + e.getMessage());
        }
    }
}

Understanding the output

classify(request) returns JSON with a top-level classification object:

predictedLabel — The highest-ranked label.
confidence — The fused probability of the predicted label (0.0 to 1.0).
predictions — Every candidate, ranked by fused probability descending. Each entry carries label, probability (fused), textProbability, and imageProbability.

Because the per-branch contributions are included, you can tell whether a prediction was driven by the text, the layout, or both — useful for debugging and for tuning the weights.

Text-only classification

When you already have the text — an email body, a database field, your own extraction pipeline — and no file to open, classify it directly with the static Vision.classifyText(request, text). It takes the same ClassificationRequest as classify() — build it with addCandidateLabel exactly as above. No document is opened, nothing is rendered, and only the text branch runs; it returns the same JSON shape as classify().

When you already have a document open and only want to skip the extraction cost, the equivalent is to set the text on its settings with classification.setText(...) and turn off the image branch with classification.setImageWeight(0f) (text branch only). To classify on layout alone instead, call classification.setTextWeight(0f). Either way the disabled branch is skipped and the fused probability equals the remaining branch’s probability.

Error handling

Vision API raises VisionException (a NutrientException) when classification fails. Common failure scenarios include an unreadable document, a page that produces no renderable image, a missing or unlicensed model, or fewer than two candidate labels supplied. In production code, catch NutrientException, return a clear error message, and log failure details for debugging.

Conclusion

The workflow for zero-shot document classification is:

Open the source document using try-with-resources for automatic resource cleanup.
Build a ClassificationRequest with at least two candidate labels (optionally with descriptions), and tune the branch weights (or supply text directly) on the classification settings.
Create a vision instance with Vision.set().
Call classify(request) to score the document and export the ranked result as JSON.
Write the JSON to a file for routing or downstream processing.
Handle NutrientException for robust error recovery.

For related image extraction workflows, refer to the Java SDK guides.

Download this ready-to-use sample package to explore document classification.