---
title: "Generate schemas with a custom local VLM in C# | Nutrient .NET SDK"
canonical_url: "https://www.nutrient.io/guides/dotnet/csharp/extraction/generate-extraction-schema-with-custom-local-vlm/"
md_url: "https://www.nutrient.io/guides/dotnet/csharp/extraction/generate-extraction-schema-with-custom-local-vlm.md"
last_updated: "2026-10-28T00:00:00.000Z"
description: "Generate JSON extraction schemas from example documents with the Nutrient.SDK package using a custom local OpenAI-compatible vision endpoint in C#."
---

# Generate extraction schemas with a custom local vision endpoint in C#

Structured extraction starts with a JSON Schema: the schema tells the extraction model exactly which fields to fill, and an LLM's structured-output mode guarantees the result matches it. Writing that schema by hand is the slow part — you study sample documents, name every field, decide what's nullable, what repeats, what's an enum. *Schema generation* drafts it for you: describe the document type, state your requirements in plain language, attach a few example documents, and get back a schema ready for structured-output extraction.

This sample generates an extraction schema from example documents using a custom local OpenAI-compatible vision model endpoint. The sample selects the Custom provider explicitly; the SDK's default Local provider runs a model in process.

[Download sample](https://www.nutrient.io/downloads/samples/csharp/generate-extraction-schema-with-custom-local-vlm.zip)

## How Nutrient helps

Nutrient.NET SDK runs the full schema generation workflow behind a single method call. The SDK handles:

- Rendering the example documents' pages and sending them to the vision model in one request

- Drafting a schema grounded in the document type, your requirement, and the examples

- Validating and hard-clamping the draft — field budget, nesting depth, and the feature set of your target structured-output dialect

- Optionally drafting cross-field constraint rules (sums, orderings) and validating each one against the schema

- Serializing the result to JSON

The result always satisfies the limits you configured, regardless of what the model drafted — the same call with the same settings yields a schema your extraction target accepts.

## How generation works

Three inputs shape the schema:

- **Document type** (required) — The class of documents the schema must represent, such as "invoice" or "application form". It grounds the schema's vocabulary.

- **Requirement** (optional) — A free-form description of what the schema must capture: fields of interest, granularity, naming preferences — anything you'd tell a colleague.

- **Example documents** (optional, up to five) — Opened documents whose pages are rendered and shown to the model as representative samples. The schema is designed so *other* documents of the same type also fit, not just the examples. At most 20 pages are sent in total across all examples; pages beyond that bound aren't rendered.

Generation requires a reachable vision model endpoint. Select the Custom provider and configure its endpoint and model on the standalone document settings passed to schema generation. Settings on the example documents do not select the provider for this call. Schema generation requires the vision schema generation feature in your license.

## Prerequisites

This sample explicitly selects the Custom provider, which connects to an external OpenAI-compatible vision model server. The SDK doesn't provision or start that server.

- Start a server such as LM Studio, Ollama, or vLLM and load a vision-capable model.

- Set the API endpoint and model identifier to match your server. Replace `your-vision-model` in the code with the loaded model's identifier.

- Set an API key in the custom VLM API settings when your server requires authentication.

- Ensure the endpoint is reachable before processing documents.

Local server setup examples:

- **LM Studio** — Start server mode with a vision model loaded; the default endpoint is `http://localhost:1234/v1`.

- **Ollama** — Run `ollama serve` with a vision model; the default endpoint is `http://localhost:11434/v1`.

- **vLLM** — Launch a vision-capable model with an OpenAI-compatible server; configure its port and API key to match your deployment.

Use [custom VLM API settings](https://www.nutrient.io/api/csharp/settings/vision/advanced/custom-vlm-api-settings.md) to configure the endpoint, model, authentication, and generation parameters.

## Targeting a structured-output dialect

LLM providers each accept a different subset of JSON Schema in their structured-output modes — one rejects `oneOf`, another rejects `maxItems`, a third silently ignores keywords it doesn't know. The `Compatibility` setting picks the target dialect, and the SDK guarantees the generated schema stays inside it:

- **`SchemaCompatibility.Portable`** (default) — The intersection accepted by the major structured-output modes and local grammar-constrained inference engines. The safest choice when you don't know where the schema will run.

- **`SchemaCompatibility.OpenAI`**, **`SchemaCompatibility.Anthropic`**, **`SchemaCompatibility.Gemini`** — Unlock the keywords that specific provider documents as supported, while enforcing its documented rejections.

- **`SchemaCompatibility.None`** — No dialect filter; any standard JSON Schema feature the other knobs enable.

Every generated schema also satisfies the strict-mode invariants shared by all providers: an object root, every property listed in `required`, `additionalProperties: false` on every object, and optionality expressed as a null type union rather than a missing requirement.

## Bounding the schema

Generated schemas stay within the budgets you set — properties beyond `MaxFields` are truncated and structures deeper than `MaxNestingDepth` are flattened, so a verbose model can't produce an unusable schema. Feature knobs (`AllowEnums`, `AllowNullable`, `AllowStringFormats`, and others) control which JSON Schema features the draft may use.

## Cross-field constraints

A schema describes each field in isolation; it can't say "the subtotal equals the sum of the line item amounts." Setting `IncludeConstraints` to `true` asks the model to also encode such relationships as [JsonLogic](https://jsonlogic.com) constraint rules — inert JSON expressions your validation layer can evaluate after extraction. Every rule is checked against the generated schema: a rule referencing a field that doesn't exist is dropped, so the constraints always match the schema they ship with.

## Complete implementation

Import the Nutrient namespace:

```csharp

using Nutrient;

```

## Building the request

Open the example documents in [using statements](https://learn.microsoft.com/dotnet/csharp/language-reference/statements/using) so resources are cleaned up after processing, then describe what the schema must represent:

```csharp

try
{
    using Document firstExample = Document.Open("input_forms.pdf");
    using Document secondExample = Document.Open("input_forms_detection.pdf");

    using SchemaGenerationRequest request = new SchemaGenerationRequest();
    request.DocumentType = "application form";
    request.Requirement = "Capture the applicant's identity (full name, date of birth, contact details), " +
        "every checkbox group as an enumerated field, and the signature and date block. " +
        "Repeating rows belong in arrays of objects.";
    request.AddExampleDocument(firstExample);
    request.AddExampleDocument(secondExample);

```

## Configuring the schema shape

Create standalone document settings, configure the Custom provider connection, bound the field budget, and opt in to cross-field constraints:

```csharp

    using DocumentSettings settings = new DocumentSettings();
    settings.VisionSettings.Provider = VlmProvider.Custom;
    CustomVlmApiSettings vlmSettings = settings.CustomVlmApiSettings;
    vlmSettings.ApiEndpoint = "http://localhost:1234/v1";
    vlmSettings.Model = "your-vision-model";
    SchemaGenerationSettings schemaGeneration = settings.SchemaGenerationSettings;
    schemaGeneration.MaxFields = 40;
    schemaGeneration.IncludeConstraints = true;

```

## Generating the schema

Call the static `Vision.GenerateSchema()` with the request and settings:

```csharp

    string resultJson = Vision.GenerateSchema(request, settings);

    File.WriteAllText("output.json", resultJson);
}
catch (NutrientException e)
{
    Console.Error.WriteLine($"Error: {e.Message}");
    Environment.Exit(1);
}

```

## Understanding the output

Schema generation returns a JSON envelope with two members — the shape is the same whether or not constraints are enabled:

- **`schema`** — The generated JSON Schema, ready to pass to an LLM structured-output mode. Every property carries a `description` telling the extractor what to put there.

- **`constraints`** — Cross-field rules (empty unless constraints are enabled). Each carries a short `id`, a one-sentence `description`, and a `rule` — a plain, standard JsonLogic expression that evaluates to `true` when the extracted document is internally consistent, for example, `{"==": [{"var": "total"}, {"+": [{"var": "subtotal"}, {"var": "tax_amount"}]}]}`.

Pass `schema` to your extraction call, and after extraction, evaluate each constraint rule against the extracted data with any JsonLogic implementation — a `false` result flags the document for review. Comparison policy (such as tolerating sub-cent rounding differences in numeric checks) is yours to apply in that evaluation step.

## Error handling

The SDK throws a `NutrientException` when generation fails. Common failure scenarios include a missing document type, more than five example documents, an unreachable vision model endpoint, or a model response that isn't valid JSON after a retry. In production code, catch `NutrientException`, return a clear error message, and log failure details for debugging.

## Conclusion

The workflow for generating an extraction schema is:

1. Open up to five representative example documents using [using statements](https://learn.microsoft.com/dotnet/csharp/language-reference/statements/using) for automatic resource cleanup.

2. Build a `SchemaGenerationRequest` with the document type (required), a natural-language requirement, and the examples.

3. Select the Custom provider and configure the endpoint and model on the standalone `DocumentSettings`, then configure the schema shape through `SchemaGenerationSettings`.

4. Call the static schema generation method on `Vision` to draft, validate, and clamp the schema.

5. Write the envelope to a file; feed `schema` to your structured-output extraction and `constraints` to your validation layer.

6. Handle `NutrientException` for robust error recovery.

For configuring a custom vision model connection, refer to the [custom local AI endpoint guide](https://www.nutrient.io/guides/dotnet/csharp/extraction/describe-image-with-custom-local-ai.md). For related extraction workflows, refer to the [.NET SDK guides](https://www.nutrient.io/guides/dotnet/csharp.md).

Download [this ready-to-use sample package](https://www.nutrient.io/downloads/samples/csharp/generate-extraction-schema-with-custom-local-vlm.zip) to explore schema generation.

To use Nutrient's built-in model without a server, refer to the [Local provider version of this sample](https://www.nutrient.io/guides/dotnet/csharp/extraction/generate-extraction-schema.md).
---

## Related pages

- [Nutrient .NET SDK extraction guides](/guides/dotnet/csharp/extraction.md)
- [Applying OCR to a PDF page](/guides/dotnet/csharp/extraction/apply-ocr-to-pdf-page.md)
- [Applying OCR to a PDF document](/guides/dotnet/csharp/extraction/apply-ocr-to-pdf.md)
- [Classifying documents](/guides/dotnet/csharp/extraction/classify-document.md)
- [Generating image descriptions using Claude](/guides/dotnet/csharp/extraction/describe-image-with-claude.md)
- [Describe images with a custom local AI endpoint in C#](/guides/dotnet/csharp/extraction/describe-image-with-custom-local-ai.md)
- [Generating image descriptions using local AI](/guides/dotnet/csharp/extraction/describe-image-with-local-ai.md)
- [Generating image descriptions using OpenAI](/guides/dotnet/csharp/extraction/describe-image-with-openai.md)
- [Detecting document language](/guides/dotnet/csharp/extraction/detect-document-language.md)
- [Extracting data from images using ICR](/guides/dotnet/csharp/extraction/extract-data-from-image-icr.md)
- [Extracting data from images using OCR](/guides/dotnet/csharp/extraction/extract-data-from-image-ocr.md)
- [Extracting data from images using vision language models](/guides/dotnet/csharp/extraction/extract-data-from-image-vlm.md)
- [Extract image data with a custom local vision endpoint in C#](/guides/dotnet/csharp/extraction/extract-data-from-image-with-custom-local-vlm.md)
- [Extracting data from specific pages](/guides/dotnet/csharp/extraction/extract-data-from-specific-pages.md)
- [Extracting form fields from images](/guides/dotnet/csharp/extraction/extract-form-fields-from-image.md)
- [Extracting structured data from documents](/guides/dotnet/csharp/extraction/extract-structured-data.md)
- [Generating extraction schemas](/guides/dotnet/csharp/extraction/generate-extraction-schema.md)
- [Extracting JSON data from a PDF document](/guides/dotnet/csharp/extraction/json-data-extraction.md)
- [Label form fields with a custom local vision endpoint in C#](/guides/dotnet/csharp/extraction/label-form-fields-with-custom-local-vlm.md)
- [Labeling form fields with a vision language model](/guides/dotnet/csharp/extraction/label-form-fields-with-vlm.md)
- [Parsing a document into structured content](/guides/dotnet/csharp/extraction/parse-document.md)
- [Extracting text from PDF documents](/guides/dotnet/csharp/extraction/pdf-to-text.md)
- [Reading barcodes with vision extraction](/guides/dotnet/csharp/extraction/read-barcodes-with-vision.md)
- [Extracting text from multilingual images](/guides/dotnet/csharp/extraction/read-text-from-image-multi-language.md)
- [Extracting text from images](/guides/dotnet/csharp/extraction/read-text-from-image.md)
- [Searching document text](/guides/dotnet/csharp/extraction/search-document-text.md)
- [Speeding up first ICR operation by predownloading models](/guides/dotnet/csharp/extraction/speed-up-first-icr-by-downloading-requirements.md)