Getting started with AI Assistant and Nutrient Android SDK

AI Assistant provides Nutrient Android SDK with AI functionality. Using intelligent document processing (IDP) technology, AI Assistant enables users to query, summarize, translate, compare document text on the fly.

To set up a fully functional AI system, you’ll need a Docker container service and a library working in unison:

• Nutrient Android SDK — A document viewer for Android that also exposes a user interface (UI) for the AI features.
• AI Assistant — A service to process the AI requests and process documents.

Prerequisites

AI Assistant is distributed as a Docker container. To run it on your computer, you need to install a Docker runtime distribution for your operating system.

Obtaining an OpenAI API key

AI Assistant requires an API key from either of these LLM providers:

• OpenAI
• Azure OpenAI

This example will use OpenAI, but if you want to use Azure OpenAI, refer to the Azure OpenAI guide.

If you don’t have an OpenAI key, create one by following the steps in the next section. Otherwise, skip to the Setting up AI Assistant section.

Creating an OpenAI account

To create an OpenAI account, sign up(opens in a new tab) to obtain an API key(opens in a new tab).

The OpenAI API has attained SOC 2 Type 2 compliance (see the official announcement(opens in a new tab)).

Save your API key somewhere safe, as you’ll need it in the Setting up AI Assistant step.

Setting up AI Assistant

AI Assistant requires a PostgreSQL database with the pgvector(opens in a new tab) extension to operate.

Copy the code snippet below and save it anywhere on your computer in a file called docker-compose.yml. Replace the <your-openai-api-key> placeholder with your OpenAI API key:

version: "3.8"

services:
  ai-assistant:
    image: pspdfkit/ai-assistant:nightly
    environment:
      OPENAI_API_KEY: <your-openai-api-key>
      PGUSER: db-user
      PGPASSWORD: password
      PGDATABASE: ai_assistant
      PGHOST: db
      PGPORT: 5432
      API_AUTH_TOKEN: secret
      JWT_PUBLIC_KEY: |
        -----BEGIN PUBLIC KEY-----
        MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2gzhmJ9TDanEzWdP1WG+
        0Ecwbe7f3bv6e5UUpvcT5q68IQJKP47AQdBAnSlFVi4X9SaurbWoXdS6jpmPpk24
        QvitzLNFphHdwjFBelTAOa6taZrSusoFvrtK9x5xsW4zzt/bkpUraNx82Z8MwLwr
        t6HlY7dgO9+xBAabj4t1d2t+0HS8O/ed3CB6T2lj6S8AbLDSEFc9ScO6Uc1XJlSo
        rgyJJSPCpNhSq3AubEZ1wMS1iEtgAzTPRDsQv50qWIbn634HLWxTP/UH6YNJBwzt
        3O6q29kTtjXlMGXCvin37PyX4Jy1IiPFwJm45aWJGKSfVGMDojTJbuUtM+8P9Rrn
        AwIDAQAB
        -----END PUBLIC KEY-----
      JWT_ALGORITHM: RS256
      DASHBOARD_USERNAME: dashboard
      DASHBOARD_PASSWORD: secret
      SECRET_KEY_BASE: secret-key-base
    ports:
      - 4000:4000
    depends_on:
      db:
        condition: service_healthy
  db:
    image: pgvector/pgvector:pg16
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U db-user -d ai_assistant" ]
      interval: 3s
      timeout: 3s
      retries: 10
    environment:
      POSTGRES_USER: db-user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: ai_assistant
      POSTGRES_INITDB_ARGS: --data-checksums
      PGDATA: /var/lib/postgresql/data/pgdata
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Starting AI Assistant

Now open a terminal emulator.

Go to the directory where you saved the docker-compose.yml file:

cd <path-to-directory-with-docker-compose-yml>

Run the following:

docker-compose up

This command might take a while to run, depending on your internet connection speed. Wait until you see the following message in the terminal:

ai_document_assistant  | info: AI Assistant started

AI Assistant is now up and running!

Setting up AI Assistant on Android

To attach AI Assistant directly to a document, follow the steps below.

1. Add the following dependencies to your Gradle file, as AI Assistant depends on them:

   implementation("io.noties.markwon:core:4.6.2")
   implementation("io.noties.markwon:html:4.6.2")
   implementation("io.noties.markwon:linkify:4.6.2")
   implementation("io.noties.markwon:ext-tables:4.6.2")
   implementation("io.noties.markwon:ext-strikethrough:4.6.2")
   implementation("io.socket:socket.io-client:2.1.1")

2. Create an instance of AI Assistant using the standaloneAiAssistant method and passing AiAssistantConfiguration into it. This configuration specifies the URL to access the AI Assistant server, a JSON Web Token (JWT) for authentication, a session identifier, and an optional user identifier.

   • It’s best practice for the JWT to be generated by your own server. For more details, refer to the guide on generating a JWT.
   • Use one session for each document if you want to restore the chat history each time. Use multiple sessions per document if multiple users access the same document in your app on one device.

3. Attach the AI Assistant object to PdfDocument using the setAiAssistant method.

4. To display the AI Assistant icon in your toolbar, enable it with setAiAssistantEnabled in PdfActivityConfiguration.

Now you can access AI Assistant in your Activity by clicking the AI Assistant icon in the Nutrient MainToolbar.

For an interactive example of AI Assistant, check out the AiAssistant example(opens in a new tab) in the Catalog app.

Advanced usage

If you need more control, the initialize method and individual steps in this section provide advanced flexibility. The responseState flow enables observing responses, and messages can be sent using emitMessage.

AI Assistant initialization

val aiAssistant = AiAssistant.standaloneAiAssistant(
  AiAssistantConfiguration(
    "https://your-ai-assistant-server.com",
    "your-jwt-token",
    "your-session-id",
    "your-user-id" // Optional.
  ))

With a configured instance of AI Assistant, you have more control over the setup. It enables you to ingest a document early — before loading a document in the user interface (UI) — which can drastically reduce the time before the UI is ready for chatting with the assistant, especially when working with large documents. You can also programmatically retrieve the data from the assistant to get the session history or process the responses from the assistant in your own code.

For AI Assistant to begin processing the document, call the initialize method, which requires a data provider and a list of document identifiers.

Attaching a document to AI Assistant

To attach a document to AI Assistant, use initialize with the permanent and changing IDs of the document, which can be taken from the PdfDocument instance:

val permanentDocumentId = StringUtils.byteToHex(pdfDocument.permanentId)
val documentChangingId = StringUtils.byteToHex(pdfDocument.changeId)
val identifier = DocumentIdentifiers(permanentDocumentId, documentChangingId)

aiAssistant.initialize(dataProvider, identifier)

The initialize method automatically:

• Checks if the document is already ingested.
• Ingests the document if necessary.
• Establishes a socket connection.
• Retrieves the session history.

If you prefer to do this manually, follow the steps outlined below.

1. Check if a document is already ingested:

   val result = aiAssistant.checkIfDocumentIsAlreadyIngested(documentId, fileHash)

2. Ingest a document:

   val ingestionResponse = aiAssistant.ingestDocument(dataProvider, jwtToken)

3. Establish a socket connection:

   aiAssistant.initializeSocketConnection()

4. Retrieve the session history:

   val history = aiAssistant.getSessionHistory()

Observing responses

After initialization, observe responses using the responseState flow and update your UI accordingly:

val responseState: Flow<CompletionResponse?> = aiAssistant.responseState

Emitting messages

To interact with AI Assistant, use the emitMessage method:

aiAssistant.emitMessage("Your query here", permanentDocumentId)

This sends a message to AI Assistant and receives a response using responseState.

Terminating AI Assistant

To stop AI Assistant, call the terminate method:

aiAssistant.terminate()