ollama as openai endpoint

Ollama does not have a built-in "API Key" authentication system for its local server. By default, it is designed for open local access. However, because Ollama provides an OpenAI-compatible endpoint, you can easily put it behind a Reverse Proxy (like Nginx) to add an API Key layer.

Here is how to set this up on your Ubuntu machine so other platforms can consume it securely.

Phase 1: Enable Network Access

By default, Ollama only listens on localhost. To let other platforms connect, you must allow it to listen on your network interface.

1. Edit the Ollama service configuration:

   sudo systemctl edit ollama.service
   

2. Add these lines (this opens a blank file or section):

   [Service]
   Environment="OLLAMA_HOST=0.0.0.0"
   

3. Reload and Restart:

   sudo systemctl daemon-reload
   sudo systemctl restart ollama
   

Phase 2: Add API Key Security (Nginx)

Since Ollama doesn't validate keys, we use Nginx to act as a "bouncer." It will check for a Bearer token before letting the request reach Ollama.

1. Install Nginx:

   sudo apt update && sudo apt install nginx -y
   

2. Create a Configuration File: Create /etc/nginx/sites-available/ollama-proxy:

   server {
       listen 80; # Or 443 with SSL
       server_name your_ip_or_domain;
   
       location / {
           # 1. Check for your custom API Key
           if ($http_authorization != "Bearer YOUR_CHOSEN_SECRET_KEY") {
               return 401;
           }
   
           # 2. Forward request to Ollama
           proxy_pass http://127.0.0.1:11434;
           proxy_set_header Host $host;
           proxy_set_header X-Real-IP $remote_addr;
   
           # 3. Necessary for streaming responses
           proxy_http_version 1.1;
           proxy_set_header Connection "";
           proxy_buffering off;
           proxy_read_timeout 600s;
       }
   }
   

3. Enable the site and restart Nginx:

   sudo ln -s /etc/nginx/sites-available/ollama-proxy /etc/nginx/sites-enabled/
   sudo nginx -t && sudo systemctl restart nginx
   

Phase 3: Consuming it (OpenAI Style)

Now, any platform (like LangChain, Dify, or a custom SDK) can consume your bge-m3 model using the standard OpenAI format.

Example: cURL

curl http://<YOUR_SERVER_IP>/v1/embeddings \
  -H "Authorization: Bearer YOUR_CHOSEN_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-m3",
    "input": "Hello world"
  }'

Example: Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://<YOUR_SERVER_IP>/v1",
    api_key="YOUR_CHOSEN_SECRET_KEY" # This is now validated by Nginx!
)

response = client.embeddings.create(
    model="bge-m3",
    input="Testing my secure embedding server"
)

Summary Comparison Table

Pro Tip: For production, you should also set up SSL (HTTPS) using Let's Encrypt (certbot) so your API Key isn't sent in plain text over the network.

Are you planning to expose this over the public internet, or just to other servers within your local private network?