Extracting data from images using ICR

Use ICR to extract structured document data from images with local models.

Common use cases include:

• Air-gapped document processing
• Privacy-sensitive workflows with local-only processing
• High-volume extraction with predictable runtime cost
• Pipelines that need layout and semantic structure

ICR returns more than plain text. It detects layout and semantic elements such as tables, key-value regions, headings, and equations.

How Nutrient helps

Nutrient .NET SDK handles local model loading, layout analysis, and JSON output generation.

The SDK handles:

• Local model deployment and loading details
• Table detection and cell boundary extraction
• Semantic element classification and hierarchy parsing
• Bounding box and reading-order calculation

Prerequisites

Before following this guide, ensure you have:

• .NET 8.0 or higher installed
• Nutrient .NET SDK referenced by your project
• An image file to process (PNG, JPEG, or other supported formats)

For initial SDK setup and configuration, refer to the getting started guide.

Complete implementation

This example extracts structured JSON from an image using the ICR engine:

using Nutrient;

Configuring ICR mode

Open the image with a using statement(opens in a new tab) and set the vision engine to ICR.

In this sample:

• Setting Engine to VisionEngine.Icr selects local ICR mode.
• ICR is the default engine, so this step is optional.

ICR is the default engine, so this call is optional but shown here for illustration purposes.

try
{
    using Document document = Document.Open("input_ocr_multiple_languages.png");
    // Configure ICR engine for local processing (this is the default)
    document.Settings.VisionSettings.Engine = VisionEngine.Icr;

Creating a vision instance

Create a vision instance with Vision.Set(document) to bind extraction to the opened document:

    var vision = Vision.Set(document);

Performance tip: Preload Vision resources

The first ICR run may take longer because required Vision resources/models are downloaded and initialized on first use. To avoid this startup delay during extraction, call vision.Warmup() once before ExtractContent():

    // Preload/download required Vision resources (first run can take time)
    vision.Warmup();

After warmup completes successfully, subsequent extractions are typically faster because resources are already cached locally.

Extracting content

Call ExtractContent() to run local layout analysis. It returns structured JSON as a string, and processing runs locally when the engine is ICR:

    string contentJson = vision.ExtractContent();

Write the JSON string to a file for downstream use.

Use the output for storage, indexing, or custom analysis:

    File.WriteAllText("output.json", contentJson);
}
catch (NutrientException e)
{
    Console.Error.WriteLine($"Error: {e.Message}");
    Environment.Exit(1);
}

Understanding the output

ExtractContent() returns structured JSON with layout and semantic information.

ICR output includes:

• Document elements — Paragraphs, headings, tables, figures, equations, and detected barcodes
• Barcode data — Decoded barcode values with symbology information for supported 1D and 2D barcode types
• Bounding boxes — Pixel coordinates for detected regions
• Reading order — Element order for content flow reconstruction
• Element classification — Semantic labels such as paragraph, table, heading, and barcode
• Hierarchical structure — Parent-child relationships across sections and blocks

Use this JSON for extraction pipelines, structured storage, and search indexing.

Error handling

Vision API throws a NutrientException when extraction fails.

Common failure scenarios include:

• The image file can’t be read because of path or permission issues.
• Image data is corrupted or truncated.
• ICR models are missing or inaccessible.
• Available memory is insufficient for model loading.
• Image format or encoding is unsupported.

In production code:

• Catch NutrientException.
• Return a clear error message.
• Log failure details for debugging.

Conclusion

Use this workflow for ICR-based extraction:

1. Open the image document with a using statement for automatic resource cleanup.
2. Set Engine to VisionEngine.Icr in the vision settings for local AI processing.
3. ICR is the default engine, making this configuration optional but useful for explicit control.
4. Create a vision instance with Vision.Set() to bind content extraction operations to the document.
5. Optionally call Warmup() once to pre-download models and avoid first-run latency.
6. Call ExtractContent() to invoke local AI models for document layout analysis.
7. The ICR engine loads AI models, detects semantic elements (tables, equations, headings, and barcodes), and determines reading order.
8. The method returns a JSON-formatted string containing complete document structure with bounding boxes in pixel coordinates.
9. All processing occurs locally without external API calls, ensuring data privacy and offline capability.
10. Write the JSON content to a file for downstream pipelines, including data extraction, database storage, and search indexing.
11. Handle NutrientException failures for robust error recovery in production environments.
12. ICR mode is ideal for air-gapped environments, sensitive document processing, and high-volume workflows.

For related image extraction workflows, refer to the .NET SDK guides.

Download this ready-to-use sample package to explore the Vision API capabilities with preconfigured ICR settings.