This HTML page is not optimized for LLM or AI agent consumption. Fetch the Markdown version instead: /guides/document-engine/configuration/full-text-search.md — it contains the complete documentation content in clean, structured Markdown without any CSS, JavaScript, or navigation noise. Speed up search with a full-text index | Nutrient

Speed up search with a full-text index

Markdown version of this page, suitable for AI agents and automation

Document Engine can answer plain text search requests from a cached full-text search (FTS) index instead of rescanning the source PDF on every query. For large documents, this turns searches that take multiple seconds into ones that complete in less than a second.

The feature is opt-in and disabled by default. Deploying a release that includes it doesn’t change search behavior until you enable it, and the /search response shape is unchanged — SDK clients need no changes. FTS has been available since Document Engine 1.17.

When to use it

Previously, every /search request reopened the source PDF and scanned the requested pages from scratch. That cost grows with document size, so search on large blueprint- or manual-style PDFs could take several seconds per query.

The FTS path builds an index once per source PDF, persists it, and reuses it across requests and nodes. The improvement scales with document size and is largest for hit-heavy queries, where the previous path spent the most time scanning. For small documents, where a scan is already fast, the two paths perform similarly.

Enable it

FTS is controlled by two environment variables on the document-engine container. A typical rollout enables them in this order:

  1. Set SEARCH_FTS_BUILD_ON_WRITE=true first, so indexes are built in the background as documents are created or their source PDFs are replaced. This lets indexes accumulate before any query relies on them.
  2. Set SEARCH_FTS_ENABLED=true to route eligible plain text searches through the index. When a query targets a document that doesn’t yet have a built index, Document Engine enqueues a build and transparently falls back to the previous per-query scan for that request.

Both default to false. A third option, SEARCH_FTS_ENRICHMENT, controls how matches are turned into preview text and highlight rectangles; the supported value is daemon (the default), where Document Engine computes previews and rectangles while querying the index.

Refer to the search options in the configuration options guide for the full list.

What uses the FTS path

To keep behavior safe and predictable, FTS only handles plain text search. The following requests continue to use the previous per-query scan:

  • Searches that include annotations (include_annotations=true)
  • Regex and preset searches
  • Case-sensitive searches (case_sensitive=true)
  • Any document without a built or reachable index — a build is enqueued and the request falls back for that query

How indexing works

  • The index is content-addressed by the source PDF, so two documents that share the same source PDF share one index, and replacing a document’s source PDF naturally invalidates its index.
  • Indexes are built by background jobs and persisted to your configured asset storage backend (S3, Azure Blob Storage, or the built-in PostgreSQL storage). A node restart, or a request routed to a different node, reuses the persisted index rather than rebuilding it.
  • Index builds never block document upload, source-PDF replacement, or deletion.