Set Up Otari with PostgreSQL and Localtonet

What the Otari PostgreSQL stack provides

Otari sits between applications and upstream model providers. Applications send requests to one gateway endpoint instead of integrating separately with every provider. The gateway authenticates the request, resolves the required provider credentials, checks applicable budgets before dispatch, sends the provider request, and records usage afterward.

The project documents compatibility with both OpenAI-style and Anthropic-style requests. Its core completion routes include /api/v1/chat/completions, /api/v1/messages, and /api/v1/responses. OpenAI-compatible clients can use http://localhost:8000/api/v1 as their local base URL. The same server also presents an administrative dashboard at http://localhost:8000/, Swagger UI at /api/v1/docs, and its OpenAPI document at /api/v1/openapi.json.

Otari offers a short Docker command for an ephemeral evaluation, but that container stores SQLite data inside the container. When the disposable container stops and is removed, that database is deleted with it. The full Docker Compose deployment is the appropriate path when persistent gateway state matters because it runs Otari with PostgreSQL.

Choose the appropriate runtime mode

Otari documents three runtime modes. This tutorial focuses on a standalone deployment because the objective is to operate a self-contained gateway with local storage and PostgreSQL. Understanding the other modes is still important when reviewing an existing configuration.

When OTARI_MODE is not set, the presence of OTARI_AI_TOKEN selects hybrid mode. Otherwise, Otari defaults to standalone mode. Check these values carefully if a deployment unexpectedly behaves as a connected hybrid gateway rather than a self-contained standalone instance.

Prerequisites and deployment planning

Prepare the host before cloning the project. The machine needs Git to retrieve the repository and a working Docker installation with the Docker Compose command available as docker compose. It also needs sufficient local resources for Otari, PostgreSQL, and any optional services you enable. Otari's supplied evidence does not establish a universal CPU, memory, disk, Docker version, or operating-system minimum, so this guide does not invent one.

Confirm that the following commands are available:

git --version
docker --version
docker compose version

Each command should return version information rather than a command-not-found message or a Docker daemon connection error. If the Docker client is installed but cannot contact the daemon, fix that local Docker problem before continuing. Container startup, PostgreSQL persistence, and local networking cannot be validated until Docker itself is operational.

You will also need at least one provider credential if you want to submit a real model request. Treat that credential as a secret. Do not place it in shell history, screenshots, support tickets, Git commits, or an article example. Otari keeps provider credentials behind the gateway so client applications can use gateway-issued keys instead of receiving the original upstream secret.

Plan the network boundary

The documented server address is http://localhost:8000/. Before remote access is introduced, port 8000 only needs to be reachable on the Otari host. Check whether another process already occupies that port. The project evidence does not define an alternative port procedure for this exact Compose stack, so consult the actual Compose file before changing port mappings.

For Localtonet, install our client on the same machine or another device that can reach the Otari host and port. If the client runs on the same host, the local target can normally be the loopback service. If it runs elsewhere, the target must be an address reachable from that client device. Do not assume that localhost on another machine refers to the Otari server. It always refers to the machine on which the connecting process runs.

Prepare secrets before deployment

The full-stack setup tells you to set a master key, provider credentials, and pricing in config.yml. Choose a strong, unique master key rather than a sample or reused password. Provider credentials should have only the permissions needed for the models and operations you intend to expose.

If you plan to store provider keys through the dashboard, Otari also requires OTARI_SECRET_KEY to be set to a Fernet key generated by:

otari gen-secret-key

Run that command in an environment where the Otari CLI is installed and available. Supply the generated value to the Otari service as OTARI_SECRET_KEY through your deployment's secure environment configuration. The available evidence does not specify a single universal secret manager or Compose override pattern, so use the method approved for your host and inspect the repository's current Compose configuration before editing it.

Install the persistent Otari stack with Docker Compose

Run the following workflow on the machine that will host Otari. It follows the project's documented persistent installation sequence: clone the repository, enter it, copy the example configuration, configure the required values, pull the images, and start the Compose stack.

git clone https://github.com/mozilla-ai/otari

cd otari

cp config.example.yml config.yml

docker compose pull

docker compose up -d

Startup can take longer on the first run because images may need to be extracted, PostgreSQL must initialize, and Otari must prepare its database. Do not create the public tunnel merely because the Compose command returned successfully. First inspect the service state and verify the HTTP endpoint locally.

Use the standard Compose status and log commands from the repository directory:

docker compose ps
docker compose logs

The status output should not show a required service repeatedly restarting or exiting. Read the logs for database connection failures, malformed configuration, missing credentials, migration errors, port conflicts, or permission problems. Avoid copying complete logs into public channels because startup output can contain sensitive operational information.

Optional service profiles

Otari's Compose setup includes optional profiles for code execution, web search, and guardrails. Start them only when your workload requires those capabilities and after reviewing their security and resource implications:

docker compose --profile code-exec --profile web-search --profile guardrails up -d

This command enables all three documented optional profiles. They expand the deployment beyond the core gateway and PostgreSQL stack. Optional code execution deserves particularly careful isolation and access review because it adds an execution capability to the environment. Do not enable optional services solely because they are available.

Configure keys, providers, models, and clients

A running container is not yet a useful gateway. Otari needs a valid configuration, at least one usable provider path for real inference, and a gateway API key for client authentication. Separate these credential types mentally and operationally:

Capture the first bootstrap key safely

On the first start with an empty database, Otari creates an API key and prints it once. The output identifies it as the bootstrap key and tells you to save it. Review the initial Compose logs in a private terminal, record the key in an approved secret manager, and do not expect the same plaintext value to be displayed again.

If the database already contains keys, Otari should not be treated as a fresh installation. This can occur when restarting a persistent Compose deployment or reusing existing PostgreSQL storage. Do not delete persistent data merely to make a bootstrap message reappear. Use the dashboard and the project's supported key-management workflow instead.

Use scoped gateway keys for applications

The bootstrap key is useful for initial administration and verification, but it should not automatically become a permanent shared credential for every application. Otari supports revocable API keys with user, workspace, and model scope. Create separate keys according to workload and trust boundary so one compromised client does not require rotating every consumer.

Budget and model restrictions should match the application's purpose. A testing tool may need access to one inexpensive model and a low budget, while an internal production service may require a different model set and accounting boundary. Scope and budget decisions belong in the gateway configuration even when network access is also restricted.

Configure OpenAI-compatible clients

For a client running on the Otari host, set its base URL to:

http://localhost:8000/api/v1

Supply an Otari API key as the bearer credential expected by that client. Do not place the upstream provider key in the application when the purpose of the gateway is to keep provider credentials centralized. After Localtonet is configured, remote clients use the assigned public HTTPS URL with /api/v1 appended, while preserving the same API route structure.

Verify the dashboard, API schema, and model route locally

Verification should proceed from the least dependent layer to the most dependent one. First confirm that containers stay running. Next confirm that Otari serves HTTP. Then confirm authentication and API routing. Finally, send a provider-backed completion request. This sequence makes failures easier to isolate.

1. Confirm the dashboard responds

Open the following address in a browser on the Otari host:

http://localhost:8000/

The Otari dashboard should load. Its navigation can vary according to runtime mode and the signed-in person's authority, so not every installation will display identical administrative options. In a standalone deployment, the dashboard supports management of providers, models, routing, tools, keys, members, budgets, settings, activity, and usage.

If the browser cannot connect, return to docker compose ps and docker compose logs. A dashboard failure at this stage is local and should not be investigated as a Localtonet issue because no tunnel is required for loopback access.

2. Open the API documentation

Otari publishes Swagger UI at:

http://localhost:8000/api/v1/docs

It also publishes the machine-readable OpenAPI document at:

http://localhost:8000/api/v1/openapi.json

You can retrieve the OpenAPI document from the command line:

curl http://localhost:8000/api/v1/openapi.json

A returned OpenAPI document confirms that the HTTP server and route registration are working. It does not prove that provider credentials, model names, budget checks, or upstream connectivity are correct.

3. Submit an authenticated completion request

Replace the placeholder below with the bootstrap key or another Otari API key stored in your secret manager. The documented example uses the openai:gpt-4o-mini model identifier:

curl http://localhost:8000/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_OTARI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai:gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Say hello."
      }
    ]
  }'

A successful response verifies much more than the dashboard test. It demonstrates that the request reached the completion route, the supplied Otari key was accepted, a configured model and provider path could be resolved, applicable budget checks passed, and the upstream call returned.

Interpret failures by layer. An immediate connection refusal points to the local listener or container state. An authentication response points to the Otari API key. A model or provider error points to model configuration, credentials, permissions, quota, or upstream behavior. A budget rejection indicates gateway policy rather than a network tunnel failure.

Operate, stop, update, and extend the Compose deployment

Run operational commands from the repository directory so Compose uses the intended project definition. Standard Compose commands make it possible to inspect, stop, start, and remove containers without changing the Otari application configuration.

Inspect current state and recent output

docker compose ps
docker compose logs

Use status output to spot exited or restarting services. Use logs to understand startup sequencing and application failures. Sanitize logs before sharing them because an initial bootstrap key is printed once and operational output may reveal request or configuration details.

Stop and restart the services

docker compose stop
docker compose start

Stopping containers preserves the Compose deployment so it can be started again. Because the Localtonet tunnel depends on a reachable local service, remote requests will fail while Otari is stopped even if the Localtonet client remains connected.

Recreate the application after configuration changes

docker compose up -d

Running docker compose up -d again applies the current Compose definition and starts the required services. Whether a particular configuration change is reloaded dynamically or requires container recreation depends on the application and Compose mapping. Confirm the active behavior through the dashboard, API, and logs rather than assuming the file was reloaded.

Update deliberately

Before updating, back up the PostgreSQL data using a process appropriate for your deployment, review the target Otari release notes, compare the current configuration example with your existing config.yml, and plan a rollback. The supplied evidence does not define one universal backup command because database container names, credentials, volumes, and storage policies can vary.

After reviewing compatibility, pull the configured images and recreate the stack:

docker compose pull
docker compose up -d

Repeat local verification after every update. Check the dashboard, OpenAPI document, authentication, model routing, budgets, and at least one representative completion request before reopening remote access.

Remove containers carefully

docker compose down

This removes the Compose containers and network created for the stack. Docker normally treats named volumes separately unless volume removal is explicitly requested, but you must inspect the project's actual storage configuration before relying on that behavior. Never add destructive volume-removal options during routine troubleshooting unless permanent PostgreSQL deletion is intentional and a tested backup exists.

Publish the verified Otari HTTP service with Localtonet

Once http://localhost:8000/ works on the Otari host, an HTTP tunnel can provide a public HTTPS address for the same service. Our client establishes an outbound connection to a Localtonet relay server, so this workflow does not require inbound router port forwarding, a public IP address, VPN setup, or inbound firewall changes.

Creating a tunnel configuration does not automatically make it active. The selected Localtonet client device must be connected, and the tunnel must be started. The public endpoint remains available only while that client is connected and the tunnel is running.

HTTP tunnels can use a generated subdomain, a selected subdomain where supported, or a custom domain. These process types serve the same content through a public HTTPS address. Custom-domain DNS requirements can change, so check the current Localtonet documentation before making DNS changes rather than relying on an old record type or target.

For current tunnel options and dashboard behavior, use the Localtonet documentation. Stop the tunnel when remote access is no longer required, and delete obsolete configurations that should not be reused.

Verify the API through the public URL

Replace YOUR_PUBLIC_HOST with the hostname assigned to the running tunnel. Do not paste a private or temporary production endpoint into public documentation.

curl https://YOUR_PUBLIC_HOST/api/v1/openapi.json

Then repeat the authenticated completion test against the public route:

curl https://YOUR_PUBLIC_HOST/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_OTARI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai:gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Say hello."
      }
    ]
  }'

The public request should behave like the successful local request. If the local request succeeds but the public one fails, investigate the Localtonet client connection, tunnel state, selected device, local target, and hostname. If both fail in the same way, investigate Otari or the provider path first.

Security controls for a remotely reachable AI gateway

A working public HTTPS address changes the service's threat model. Remote users can now reach the routes exposed by the Otari HTTP server, subject to Otari's authentication and authorization. The tunnel is transport and reachability infrastructure, not a replacement for application-level access control.

Require Otari authentication for API consumers

Give each application an appropriate revocable gateway key. Avoid sharing the bootstrap or master key broadly. Use user, workspace, model, and budget scope where those controls fit the deployment. Revoke keys that are no longer needed and rotate a key immediately if it may have been disclosed.

Protect administrative routes

The same HTTP service hosts both the dashboard and API documentation. Before publishing the root URL, verify the access behavior of the dashboard for an unauthenticated remote visitor. Do not assume that a route is protected merely because it was previously reachable only through localhost.

If the available application controls do not meet your remote administrative access requirements, keep the dashboard private and expose only what your architecture can safely authorize. The supplied evidence does not establish route-level filtering in a Localtonet HTTP tunnel, so this guide does not claim that the tunnel can independently publish one Otari path while blocking another.

Keep provider credentials server-side

Client applications should receive Otari keys, not provider keys. Store provider credentials in the gateway's protected configuration or supported dashboard workflow. When storing provider keys through the dashboard, configure the generated Fernet value as OTARI_SECRET_KEY. Protect that secret alongside the master key and database backups.

Apply least privilege and budgets

Restrict each key to the models and organizational scope it needs. Set budgets that match the workload. Network restrictions reduce who can send traffic, but gateway controls determine what an authenticated identity can spend and which models it can invoke.

Control tunnel lifecycle

Start the tunnel only when the service is ready for remote use. Stop it during maintenance, incident investigation, or configuration changes that could expose an unsafe state. A stopped tunnel removes the public route, while the local Otari service can remain available for diagnosis.

Troubleshooting the Otari and Localtonet workflow

The Compose stack does not start

Run docker compose ps and docker compose logs from the repository directory. Look for an invalid config.yml, a port conflict on 8000, a PostgreSQL startup or connection failure, image pull errors, missing required values, or host storage permissions. If the Docker daemon itself is unavailable, resolve that before changing Otari configuration.

The dashboard does not open locally

Confirm that the URL is exactly http://localhost:8000/ when testing on the Otari host. Verify that the required containers remain running. If Otari is hosted on another machine, localhost in your browser refers to your own computer, not the server. Test from the host first or use an address that the testing machine can route to.

The API documentation works, but completion requests fail

This usually means the HTTP server is available while a later request stage is failing. Check the bearer key, model identifier, provider configuration, upstream provider credential, account permissions, quota, budget policy, and provider availability. Compare the application error with sanitized Otari logs.

No bootstrap key appears

Otari creates and prints the bootstrap key on the first empty database. A persistent PostgreSQL instance that has already been initialized is not an empty database. Review the earliest private startup logs and use existing administrative key-management mechanisms. Do not delete the PostgreSQL data simply to force bootstrap behavior.

Provider keys cannot be stored through the dashboard

Confirm that OTARI_SECRET_KEY has been set to a Fernet key generated with otari gen-secret-key. Ensure the value reaches the Otari service through the deployment environment, then recreate or restart the service as required by the active Compose setup. Keep the generated value private and stable for the deployment.

The Localtonet URL does not respond

Confirm that the selected Localtonet client device is connected and that the tunnel is running. Recheck the chosen relay and the local target. From the client device itself, verify that it can reach the Otari address and port configured in the tunnel. A target of localhost:8000 is valid only when the Localtonet client and Otari listener are on the same host.

The public dashboard works, but API requests return unauthorized

Reachability and Otari authentication are separate. Ensure the request sends Authorization: Bearer YOUR_OTARI_API_KEY and that the key is active and allowed to use the requested model and workspace. Never substitute the Localtonet device token for an Otari API key.

The local API works, but the public API fails

Compare the paths exactly. If the client used http://localhost:8000/api/v1 locally, its remote base URL should use the assigned HTTPS hostname followed by /api/v1. Confirm that the tunnel targets the Otari HTTP port and not PostgreSQL or an optional service. Also verify that the tunnel remains started and the selected client remains connected.

The public endpoint stops after a reboot

Check both independent service lifecycles. Otari and PostgreSQL must be running through Docker Compose, and the Localtonet client must be connected with the HTTP tunnel started. The supplied evidence does not establish a universal operating-system startup method for either component, so configure automatic startup only through mechanisms supported by your host and current client version.

An update changes dashboard or configuration behavior

Review the target release notes and compare the current config.example.yml with your deployment configuration. Back up PostgreSQL first, pull the intended images, recreate the stack, and repeat local verification before restarting public access. Do not assume that a configuration copied from an older release remains complete.

Frequently asked questions