How to Self-Host Ollama and Access It Remotely

Understand the architecture and trust boundaries

Ollama runs models on a host you control and exposes a local HTTP API, normally on port 11434. The native API includes Ollama-specific endpoints such as /api/tags and /api/chat. Ollama also implements parts of the OpenAI API under paths such as /v1/chat/completions, but that compatibility should not be treated as a universal drop-in replacement. Applications can depend on unsupported endpoints, provider-specific options, authentication behavior, tool-calling details, streaming formats, model names, or context settings.

Test every integration against the exact Ollama release, model tag, and application version you intend to use. Some tools provide a dedicated Ollama provider. Others accept an OpenAI-compatible base address. A third-party application may also require a placeholder API key even though that value does not authenticate requests to the normal local Ollama API. Follow the application's own configuration requirements rather than assuming that changing one URL is sufficient.

The traffic path is:

Remote client
    |
    | Public connection, authorization must be enforced separately
    v
Assigned Localtonet relay host and port
    |
    | Existing outbound connection from the Localtonet client
    v
Localtonet client on the Ollama host
    |
    | Local target
    v
127.0.0.1:11434
    |
    v
Ollama and the selected model

What self-hosting does and does not mean

Local inference can give you more control over model selection, storage, prompts, and service operation. It does not automatically make every workflow private or offline. Initial installation and model downloads require network access. A remote Localtonet workflow sends request and response traffic through a relay. Third-party chat interfaces, plugins, telemetry, web search, cloud model tags, or external tools can also transmit data outside the Ollama host.

Local execution also has real operating costs. There may be no per-token vendor charge for a locally executed model, but the host still consumes electricity, storage, memory, accelerator capacity, administrator time, and network bandwidth. Throughput and concurrency remain constrained by the selected model, context size, hardware, and workload. Treat privacy, cost, and performance as properties of the complete deployment rather than automatic properties of installing Ollama.

Prerequisites and capacity planning

Check the host before downloading a model. Model families often provide many tags with different parameter counts, quantizations, context capabilities, and sometimes local or cloud-oriented variants. A model family name alone is not enough to determine download size or runtime requirements.

Memory and accelerator planning

Parameter count is useful for comparing broad model classes, but it is not a reliable storage or memory figure by itself. Quantization changes the size of the weights, and runtime memory also depends on context length, parallel requests, implementation details, and accelerator offloading. A smaller quantized tag may run on a machine where a larger tag cannot, but it can have different quality and performance characteristics.

Start with a small tag, verify that it loads, and observe the host while generating a response. Increase model size only after checking memory pressure, response latency, temperature, and available disk space. On a shared workstation, reserve capacity for normal user applications. On a server, account for simultaneous users and the possibility that a long context or several requests will consume more resources than a short interactive prompt.

Privacy and data classification

Decide what data the service is allowed to process. Prompts can contain source code, customer information, internal documents, credentials, or regulated data. Model output can reproduce prompt content and may be recorded by clients, gateways, shell history, application logs, or monitoring systems. Review every component in the path, including the remote client and any authentication proxy, before sending sensitive information.

Install Ollama on Windows, macOS, Linux, or Docker

Use the current Ollama download page for the platform package. Installation interfaces and supported hardware can change, so confirm the requirements shown for the package you are about to install.

Linux installation

Ollama publishes the following installation command for Linux:

curl -fsSL https://ollama.com/install.sh | sh

Piping a downloaded script into a shell executes it immediately. If your environment requires script review, download and inspect the installer before running it. After installation, verify the executable and inspect the service:

ollama --version
systemctl status ollama

If the service is not active, start it with the operating system's service manager:

sudo systemctl start ollama

Whether Ollama starts automatically after a reboot depends on the installed service state. Confirm that behavior on your distribution instead of assuming persistence from a successful interactive test.

macOS installation

The current macOS download requires macOS 14 Sonoma or later. Download the macOS package from the Ollama download page, install the application as directed, and launch Ollama. The application must be running before local API tests will succeed.

Open Terminal and verify the command is available:

ollama --version
ollama list

Login-startup behavior can vary by application and release. Check the current Ollama application settings and test an actual reboot if unattended operation is required.

Windows installation

Select Windows on the Ollama download page and run the current installer. Launch Ollama after installation, then open a new PowerShell or Command Prompt window so any PATH changes are available.

ollama --version
ollama list

If the command is not found, close and reopen the terminal, confirm that installation completed, and verify the current Windows installation instructions. Also confirm that the Ollama application or service is running before testing port 11434.

Docker installation

Docker is useful when you want container lifecycle controls and a named volume for model data. The following baseline keeps the published port on host loopback instead of exposing it on every interface:

docker volume create ollama

docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

Confirm that the container is running and inspect its initial logs:

docker ps --filter name=ollama
docker logs ollama

Hardware acceleration in Docker requires a compatible host, driver, container runtime, and Ollama image configuration. Do not add GPU device flags copied from another platform without checking the current instructions for your exact hardware. The baseline above is intended to establish a portable local service path, not to promise accelerator use.

Choose, pull, and run a model tag

Browse the current Ollama library and select a tag suited to the task and host. The library includes families with many different sizes. For example, the listed options include Llama 3.2 tags in 1B and 3B sizes, DeepSeek R1 tags across several sizes, Gemma 3 tags from compact variants through larger models, and Qwen families with multiple text, coding, vision, and embedding choices.

Those parameter labels are not fixed download-size recommendations. Open the model page, inspect the available tags, read the model's license and capability notes, and select one exact tag. Features such as tools, vision, embeddings, audio, or cloud execution vary by family and tag.

This tutorial uses llama3.2:1b as a concrete small-tag example. Confirm that it remains available and suitable before using it:

ollama pull llama3.2:1b
ollama list
ollama run llama3.2:1b

Enter a simple prompt and wait for a complete response. Type /bye to leave the interactive session. Pulling and running are separate operations: pull downloads the selected tag, while run loads it and begins an interactive request.

In Docker, run the same Ollama command inside the container:

docker exec -it ollama ollama pull llama3.2:1b
docker exec -it ollama ollama list
docker exec -it ollama ollama run llama3.2:1b

If the pull fails, check free storage, DNS, outbound network access, and the exact tag spelling. If the process is terminated while loading, inspect memory pressure before trying a larger tag.

Verify the local Ollama API

Do not create a tunnel until Ollama works locally. Local testing separates an Ollama problem from a Localtonet configuration problem and gives you a known-good request to repeat remotely.

List installed models through the native API

curl http://127.0.0.1:11434/api/tags

A successful response is JSON. Confirm that llama3.2:1b, or your chosen replacement tag, appears in the returned model list.

Generate a native chat response

curl http://127.0.0.1:11434/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:1b",
    "messages": [
      {
        "role": "user",
        "content": "Reply with the words local API verified."
      }
    ],
    "stream": false
  }'

The first request can take longer because the model may need to load into memory. A successful response should be JSON containing an assistant message. The wording can vary, so validate the response structure rather than requiring an exact natural-language sentence.

Test the OpenAI-compatible endpoint when needed

curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:1b",
    "messages": [
      {
        "role": "user",
        "content": "Reply with the words compatibility path verified."
      }
    ]
  }'

Configure remote access with Localtonet

With Localtonet, the client application on the Ollama host establishes an outbound connection to one of our relay servers. A TCP tunnel can then forward an assigned public host and port to 127.0.0.1:11434. This avoids inbound router port forwarding and does not require Ollama to listen on every network interface.

The following procedure describes the documented Localtonet lifecycle. Creating a tunnel is not the same as starting it, and running a Localtonet client does not prove that the specific tunnel is connected.

Test from an authorized remote machine

Store the assigned endpoint in temporary shell variables rather than committing it to a repository. Replace the placeholders locally:

export OLLAMA_REMOTE_HOST="YOUR_ASSIGNED_HOST"
export OLLAMA_REMOTE_PORT="YOUR_ASSIGNED_PORT"

curl "http://${OLLAMA_REMOTE_HOST}:${OLLAMA_REMOTE_PORT}/api/tags"

The expected result is JSON similar in structure to the local /api/tags response. Confirm that the requested model tag is present. Next, repeat the native chat request:

curl "http://${OLLAMA_REMOTE_HOST}:${OLLAMA_REMOTE_PORT}/api/chat" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:1b",
    "messages": [
      {
        "role": "user",
        "content": "Reply with the words remote path verified."
      }
    ],
    "stream": false
  }'

If you use an authenticated gateway, the remote request must include the credential format required by that gateway. Do not place long-lived credentials directly in source code or command history. Prefer the application's protected secret store or another approved credential mechanism.

Configure a third-party client carefully

First determine whether the client supports Ollama natively. If it does, use its Ollama provider and enter the authorized endpoint according to that application's documentation. If it supports only an OpenAI-compatible provider, confirm that it allows a custom base address and test each required feature.

A client configured for the compatibility path will generally need a base address ending in /v1, but exact behavior varies. Some SDKs require a non-empty API-key field even when the backend does not validate it. Such a placeholder is not protection for the public endpoint. Real authorization must be enforced by the gateway or access-control layer in front of Ollama.

Secure a remotely reachable Ollama service

Threat model

Assume an unauthorized party may learn or scan the public host and port. If that party can reach an unprotected Ollama API, they may consume CPU, GPU, memory, bandwidth, and power. They may enumerate locally available model names, submit arbitrary prompts, trigger large model loads, or degrade service for legitimate users. Depending on the surrounding application, they may also reach tools or data sources that the application has connected to the model.

Also account for authorized-user mistakes. A legitimate user can submit oversized contexts, start many simultaneous requests, paste sensitive data, or configure an agent with excessive tool permissions. Authentication identifies or admits a client, but it does not replace authorization, quotas, request limits, data classification, and least privilege.

Validate the security boundary

Test the service as both an authorized and unauthorized client. An unauthenticated request should be rejected by the access-control layer and should not reach Ollama. An authorized request should reach the intended endpoint without granting access to unrelated local services. Verify logout, credential revocation, expiration, and service restart behavior.

Do not claim end-to-end encryption unless the exact client, tunnel type, gateway, TLS termination point, and backend connection have been verified in current documentation and in your deployment. Likewise, do not assume that an HTTP-oriented sign-on feature applies to a raw TCP tunnel. This tutorial makes no such claim.

Use a narrow local listener

Keep Ollama on 127.0.0.1:11434 when the Localtonet client runs on the same host. There is normally no reason to set Ollama to listen on 0.0.0.0 for this topology. A wildcard listener expands access to other interfaces and can expose the API to the local network or to container networks, depending on the host configuration.

Operate, update, and shut down the deployment

Routine health checks

A useful operational check covers every layer rather than only the public endpoint:

1. Confirm the host has sufficient free memory and storage.
2. Confirm the Ollama process, application, service, or container is running.
3. Call http://127.0.0.1:11434/api/tags locally.
4. Send a small local inference request with the intended model tag.
5. Confirm the Localtonet device is connected.
6. Confirm the intended tunnel is started and connected.
7. Test authorization from a remote client.
8. Send a small remote inference request and compare its behavior with the local test.

This sequence identifies the failing layer. For example, a successful local request with a failed remote request points toward the client, tunnel, relay selection, endpoint, authorization layer, or remote network. A failed local request should be fixed before changing the tunnel.

Ollama logs and service control

On a systemd-based Linux installation, inspect service status and follow logs with:

systemctl status ollama
journalctl -u ollama -f

For Docker, use:

docker ps --filter name=ollama
docker logs -f ollama
docker stats ollama

On Windows and macOS, confirm that the Ollama application is running and use the current application's diagnostic or log guidance. Log locations can change, so this article does not hardcode unverified paths.

Persistent Localtonet service mode

Interactive Localtonet operation is appropriate for short tests. For unattended operation, our service mode can register the client with Windows Service, systemd, or launchd. Service mode keeps the client process available, but it does not create or start a tunnel. The selected device must connect and the tunnel must still be configured and started.

On Windows, use an Administrator Command Prompt or PowerShell:

localtonet --install-service --authtoken <YOUR_TOKEN>
localtonet --start-service --authtoken <YOUR_TOKEN>
localtonet --status-service --authtoken <YOUR_TOKEN>

On systemd-based Linux or macOS, use an elevated shell:

sudo localtonet --install-service --authtoken <YOUR_TOKEN>
sudo localtonet --start-service --authtoken <YOUR_TOKEN>
sudo localtonet --status-service --authtoken <YOUR_TOKEN>

For the full lifecycle, including stop, removal, platform verification, and update precautions, follow our guide to starting Localtonet automatically. After configuring service mode, reboot the host and verify the operating system service, Localtonet device, tunnel state, local Ollama API, and authorized remote request.

Updates

Plan updates as a controlled maintenance operation. Record the Ollama version, Localtonet client version, model tags, client configuration, and authentication-gateway configuration before changing them. Re-run the local and remote verification tests afterward.

For a native Ollama installation, use the current update method provided on the Ollama download page for your platform. For Docker, pull the current image and recreate the container while preserving the named model volume:

docker pull ollama/ollama
docker stop ollama
docker rm ollama

docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

Pulling a model tag again can retrieve the currently published content for that tag. If exact reproducibility is important, record all available model metadata and test updates before promoting them. Model behavior can change even when an application request remains syntactically valid.

Safe shutdown and cleanup

Stop access in the reverse order of exposure:

1. Stop accepting new work in the remote client or gateway.
2. Use the Localtonet Stop action for the tunnel.
3. Confirm the public endpoint is no longer reachable.
4. Delete the tunnel if it will not be used again.
5. Stop or remove Localtonet service mode if the device should no longer connect.
6. Stop Ollama or its container when local inference is not required.
7. Revoke or rotate exposed credentials and device tokens according to your security process.

For a Docker deployment, stop the Ollama container with:

docker stop ollama

Removing the container does not remove the named ollama volume automatically. Delete model data only after confirming that it is no longer needed and that your retention policy permits removal.

Troubleshoot common failures

Diagnose timeouts methodically

A model request can time out even when connectivity is correct. The first request may load model weights, a large context may increase processing time, or the host may be swapping. Send /api/tags first because it does not require generating a model response. Then send a short chat prompt locally. Only after that succeeds should you repeat the same prompt remotely.

If local and remote requests both take a long time, investigate Ollama and the host. If only the remote request fails, inspect the access gateway, Localtonet device and tunnel state, relay endpoint, and remote network. Avoid increasing every timeout before identifying which operation is slow.

Diagnose resource exhaustion

Watch available memory while loading the model. In Docker, check whether the container has explicit memory or device restrictions. Reduce the model size, context, or concurrent requests and retry. A host that can answer one short request is not necessarily sized for several users or agent workflows.

Disk exhaustion can also appear during a pull or update. Use ollama list to review installed tags, remove only models you no longer need through the supported Ollama workflow, and preserve any model required by an active application.

Appropriate remote Ollama use cases

Team sharing and agent use require more than distributing a base address. Verify user authentication, per-user authorization, resource controls, data handling, failure behavior, and application compatibility. If an application can invoke tools, access files, or call other systems, those permissions must be reviewed independently from model access.

Frequently asked questions