How to Self-Host Ollama and Access It Remotely

What Is Ollama?

Ollama is an open-source tool that lets you download and run large language models locally on your own computer. Think of it as a package manager for AI models you pull a model with one command and start chatting with it immediately, with no cloud subscription, no internet dependency after the download, and no data sent to third parties.

Ollama exposes a REST API on port 11434 that is fully compatible with the OpenAI API format. That means any application built for ChatGPT's API Cline, Continue, AnythingLLM, Open WebUI can be pointed at your local Ollama instance and work without modification.

Step 1 — Install Ollama

🐧 Linux (Recommended for servers)

The official one-liner detects your architecture and installs Ollama as a systemd service automatically.

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation and check the service status:

ollama --version
systemctl status ollama

🍎 macOS

Download the Ollama app from ollama.com/download, open the ZIP, and drag Ollama.app to your Applications folder. Launch it and Ollama runs in the menu bar. Requires macOS 14 Sonoma or later.

🪟 Windows

Download the OllamaSetup.exe installer from ollama.com/download. Run it no Administrator rights required. Ollama installs to your user directory and adds itself to the PATH.

Step 2 — Pull and Run Your First Model

Browse the full model library at ollama.com/library. Below are some recommended starting points depending on your hardware and use case.

Pull a model and start an interactive chat session:

# Pull the model (downloads once, runs locally forever)
ollama pull llama3.2

# Start an interactive chat
ollama run llama3.2

# Exit the chat
/bye

List all models you have downloaded:

ollama list

Step 3 — Test the Local API

Ollama exposes a REST API on http://localhost:11434 by default. The API is fully compatible with the OpenAI API format, so any tool or library that works with OpenAI can point at Ollama instead just change the base URL.

# Verify Ollama is running
curl http://localhost:11434

# Send a chat request
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{ "role": "user", "content": "Hello! What can you do?" }],
  "stream": false
}'

Using the OpenAI-compatible endpoint (works with any OpenAI SDK):

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

Step 4 — Access Ollama Remotely with Localtonet

By default, Ollama only listens on localhost it's not reachable from other machines. There are two approaches to make it remotely accessible. The simplest and most secure is using Localtonet to create an encrypted tunnel without changing any Ollama settings or opening firewall ports.

Test the remote connection from any machine:

# Replace with your actual Localtonet relay address
curl http://example.localto.net:3327/api/chat -d '{
  "model": "llama3.2",
  "messages": [{ "role": "user", "content": "Hello from remote!" }],
  "stream": false
}'

Use with the OpenAI Python SDK from a remote machine:

from openai import OpenAI

client = OpenAI(
    base_url="http://example.localto.net:3327/v1",
    api_key="ollama"  # Required but not validated
)

response = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Explain quantum computing simply."}]
)

print(response.choices[0].message.content)

# Linux — set via systemd override
sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf <<'EOF'
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
EOF
sudo systemctl daemon-reload
sudo systemctl restart ollama

# macOS — set via launchctl
launchctl setenv OLLAMA_HOST "0.0.0.0"
# Then restart the Ollama app from the menu bar

Security Considerations

What You Can Do with a Remote Ollama Instance

Frequently Asked Questions