Vision

Provides machine learning and computer vision capabilities for document processing. Enables AI-powered document description and content extraction.

from nutrient_sdk import Vision

Construction

Class Methods

classify_text

@classmethod
def classify_text(cls, request: ClassificationRequest, text: str) -> str

Classifies caller-supplied text against the candidate labels carried by request (zero-shot) with no document. The text is scored directly by the text branch and the image branch is skipped, so nothing is rendered and no file is opened — use this when you already have the text (an email body, a database field, your own extraction pipeline) rather than a document on disk.

Parameters:

Returns: str - JSON with the predicted label and the ranked candidate probabilities.

decide

@classmethod
def decide(cls, state_json: str, questions_json: str) -> str

Calls the nutrient-document-decision model’s native decision primitive directly: full caller control over both the shared state and the questions to answer against it. No document, schema, or template list is involved — this is the raw decision-model API for callers who want calibrated yes/no, multiple-choice, or ordinal-scale answers over their own structured state. The same concepts as llama.cpp’s own /v1/systemone endpoint carry over (a shared state, and per-question type/instructions/options), but the concrete wire shape differs: questionsJson here is a JSON array with each question’s id as a field, not an object keyed by id. usage.promptTokens is exact (summed from the native decode); there is no exact pre-call tokenizer, so budget roughly 1 token per 3-4 characters of rendered state/ instructions text (closer to 1 per 2.5-3 for JSON-heavy state) for cost estimates before calling.

Parameters:

Returns: str - JSON: {"answers": {questionId: {type, chosenIndex, chosenKey, confidence, probabilities}}, "usage": {"promptTokens": N, "outputTokens": M}}.

detect_languages_text

@classmethod
def detect_languages_text(cls, text: str) -> str

@classmethod
def detect_languages_text(cls, text: str, settings: DocumentSettings) -> str

Detects the language of caller-supplied text, fully offline, with no document. CLD3 scores the text directly — nothing is rendered and no file is opened. Use this when you already have the text (an email body, a database field, your own extraction pipeline) rather than a document on disk.

Parameters:

Returns: str - JSON with the predicted language and text direction.

generate_schema

@classmethod
def generate_schema(cls, request: SchemaGenerationRequest) -> str

@classmethod
def generate_schema(cls, request: SchemaGenerationRequest, settings: DocumentSettings) -> str

Generates a JSON Schema for a class of documents using a vision model, with default document settings. The schema is designed for downstream LLM structured-output extraction tasks: it is hard-clamped to the knobs in SchemaGenerationSettings and to the target structured-output dialect selected there, so the same call with the same settings always yields a schema the target accepts.

Parameters:

Returns: str - A JSON envelope object with a schema property (the generated JSON Schema) and a constraints array of cross-field rules in standard JsonLogic. The array is empty unless SchemaGenerationSettings.include_constraints is enabled.

generate_schema_to_file

@classmethod
def generate_schema_to_file(cls, request: SchemaGenerationRequest, output_path: str) -> None

@classmethod
def generate_schema_to_file(cls, request: SchemaGenerationRequest, settings: DocumentSettings, output_path: str) -> None

Generates a JSON Schema for a class of documents using a vision model and writes it to a file, with default document settings.

Parameters:

set

@classmethod
def set(cls, document: Document) -> Vision

Creates a Vision instance for the specified document.

Parameters:

Returns: Vision - A Vision instance ready to perform analysis on the document. Raises:

• NullOrEmptyParameterException: Thrown when document is null.

Methods

classify

def classify(self, request: ClassificationRequest) -> str

Classifies the document against the candidate labels carried by request (zero-shot) and exports the ranked result. Branch weights, supplied text, and the other knobs are read from DocumentClassificationSettings on the document’s settings. The result is always JSON; output_format is not consulted because only the JSON exporter serializes the classification result.

Parameters:

Returns: str - The exported content as a string (JSON), including the predicted label and ranked candidate probabilities.

classify_to_file

def classify_to_file(self, request: ClassificationRequest, output_path: str) -> None

Classifies the document against the candidate labels carried by request (zero-shot) and writes the exported result to a file. The result is always JSON; output_format is not consulted because only the JSON exporter serializes the classification result.

Parameters:

describe

def describe(self) -> str

Generates an AI-powered description of the document content.

Returns: str - A string containing the document description.

detect_forms

def detect_forms(self) -> str

Detects form fields on the document and exports the result. Output format is determined by output_format (JSON or IR Lite). Each detected field carries its type and bounding box. To also assign AI semantic labels (e.g. “First name”), set FormLabelingSettings.enable_ai_labeling on the document’s settings before calling — no separate method is needed.

Returns: str - The exported content as a string (JSON or IR Lite JSON depending on settings).

detect_forms_to_file

def detect_forms_to_file(self, output_path: str) -> None

Detects form fields on the document and writes the exported result to a file. Output format is determined by output_format (JSON or IR Lite).

Parameters:

detect_languages

def detect_languages(self) -> str

Detects the language and text direction of the document, fully offline, and exports the result. Runs the offline cascade (Tesseract OSD for script → script-model OCR → fastText for the specific language) on every page and reports one detection per page. Output format is determined by output_format.

Returns: str - JSON with the predicted language, text direction, and per-page detections.

detect_languages_to_file

def detect_languages_to_file(self, output_path: str) -> None

Detects the language and text direction of the document, fully offline, and writes the exported result to a file. Output format is determined by output_format.

Parameters:

extract_content

def extract_content(self) -> str

def extract_content(self, settings: DocumentLayoutJsonExportSettings) -> str

Extracts structured content from the document using machine vision processing. The pipeline used is determined by the engine setting and the output format by output_format.

Parameters:

Returns: str - The exported content as a string (JSON, Markdown, or IR Lite JSON depending on settings).

extract_content_to_file

def extract_content_to_file(self, output_path: str) -> None

def extract_content_to_file(self, output_path: str, settings: DocumentLayoutJsonExportSettings) -> None

Extracts structured content from the document and writes it to a file. The pipeline used is determined by the engine setting and the output format by output_format.

Parameters:

extract_structured

def extract_structured(self, request: StructuredExtractionRequest) -> str

Extracts structured data from the document, shaped to the JSON Schema carried by the request’s {"schema": ...} envelope. The document is first read by the extraction pipeline selected by engine, then an AI model fills the schema from the recognized content. Provider, model, endpoint, and confidence reporting are driven by AiProcessingSettings on the document’s settings.

Parameters:

Returns: str - A JSON string with two top-level nodes: extraction (the schema-shaped extracted fields) and metadata (per-field source locations and grounding labels).

extract_structured_to_file

def extract_structured_to_file(self, request: StructuredExtractionRequest, output_path: str) -> None

Extracts structured data from the document, shaped to the JSON Schema carried by the request’s {"schema": ...} envelope, and writes the JSON result to a file. See extract_structured for the result shape.

Parameters:

split

def split(self) -> str

Splits a merged document into its constituent sub-documents (page-stream segmentation) and exports the result. Every page is scored for whether it starts a new sub-document — fusing a per-page image signal with the page’s text — and the contiguous page ranges are returned. The threshold and the other knobs are read from DocumentSplitSettings on the document’s settings. The result is always JSON; output_format is not consulted because only the JSON exporter serializes the split result.

Returns: str - The exported content as a string (JSON), carrying the detected sub-document segments (1-based inclusive page ranges with their boundary confidence).

split_to_file

def split_to_file(self, output_path: str) -> None

Splits a merged document into its constituent sub-documents (page-stream segmentation) and writes the exported result to a file. The result is always JSON; output_format is not consulted because only the JSON exporter serializes the split result.

Parameters:

warmup

def warmup(self) -> None

Preloads (warms up) all resources needed for vision processing. This downloads all model files based on the document’s VisionSettings before execution. Call this to avoid download delays during extract_content().