How to Run Your Own LLM and Connect It to Codex

A practical setup for running a local LLM through Ollama and Open WebUI, then wiring it into Codex CLI so your code and data never leave your machine. This isn't a review of a single tool — it's a full working pipeline, from pulling a model to actually using it inside a coding agent.

📌 Key takeaway: a local model won't replace a frontier model on hard coding tasks, but it covers something the cloud simply can't: working with private data on your own hardware. The sensible setup is hybrid — frontier models for heavy lifting, a local LLM for sensitive or routine work.

What This Setup Solves

Cloud models are convenient, but every request goes to someone else's servers. For some workloads that's a non-starter: customer personal data, internal documents, trade secrets, code under NDA.

A local setup solves three problems:

• Privacy. Prompts and files never leave your machine.
• Cost at scale. No per-token billing — you pay once for hardware.
• Autonomy. Works offline and isn't affected by provider rate limits or outages.

In exchange, you take on hardware costs, model updates, and the fact that a local model is weaker than a top-tier cloud one.

What the Setup Looks Like

Three components: the model runtime, a human-facing interface, and the coding agent. All three talk to the same local model through a single API.

flowchart LR
A["Your hardware<br>Mac / Linux / VPS"] --> B["Ollama<br>model runtime"]
B --> C["OpenAI-compatible API<br>localhost:11434"]
C --> D["Open WebUI<br>chat and RAG for the team"]
C --> E["Codex CLI<br>coding agent"]
E --> F["Code and data<br>stay local"]

💡 Why this works: Ollama serves the model over an OpenAI-compatible protocol. That's what lets both Open WebUI and Codex CLI connect to it using the exact same settings they'd use for cloud OpenAI — only the address changes.

Prerequisites

Before you start, make sure you have:

• Hardware for the model. The main constraint is memory. See the table below for ballpark figures.
• Docker — for Open WebUI, installs with a single command.
• Codex CLI — installed and authenticated.
• A chosen model. For coding, pick a model with tool-calling support (e.g., the Qwen Coder family).

Model size	Memory needed (RAM/VRAM)	Best for
7–8B	~8 GB	laptop, quick drafts
14B	~16 GB	balance of quality and speed
32B	~24–32 GB	serious coding work
70B+	48 GB and up	maximum quality, needs a workstation

⚠️ Warning: never expose Ollama directly to the internet. By default it only listens on localhost — keep it that way. For access from other devices, use a VPN or a private tunnel; otherwise anyone can reach your model.

Step 1: Run the Model in Ollama

Install the Ollama runtime and pull a model:

# macOS and Linux
curl -fsSL https://ollama.com/install.sh | sh
# pull a coding model (example)
ollama pull qwen2.5-coder:14b

On macOS there's also a second path — the desktop app (.dmg) from the website; both install the same ollama CLI.

Ollama runs as a background service and listens on port 11434. Verify the model responds via the OpenAI-compatible API:

curl http://localhost:11434/v1/models

If your model shows up in the response, the runtime is ready.

Step 2: Set Up Open WebUI

Open WebUI gives you a human-friendly interface to the local model: chat, history, team access, and RAG over your own documents.

Run it with Docker:

docker run -d -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui \
ghcr.io/open-webui/open-webui:main

Open http://localhost:3000 and create a local admin account. Open WebUI automatically finds Ollama at host.docker.internal:11434. If the model doesn't show up, set the address manually in the connection settings.

💡 Tip: RAG in Open WebUI runs on the same local model, so any documents you upload for search also never touch the cloud. It's a convenient way to give your team a private chat over an internal knowledge base.

Step 3: Connect Codex CLI to the Local Model

Codex CLI can work with any OpenAI-compatible provider. First, describe the local provider in ~/.codex/config.toml:

# Local provider on top of Ollama
[model_providers.ollama]
name = "Ollama (local)"
base_url = "http://localhost:11434/v1"
# Ollama doesn't validate the key, but the field may be required — any string works
env_key = "OLLAMA_API_KEY"

In recent Codex CLI versions, the profile itself is described in a separate file next to the main config — for example, ~/.codex/local.config.toml. Keys go at the top level, without a nested table:

model = "qwen2.5-coder:14b"
model_provider = "ollama"

⚠️ Version note: older setups wrote the profile as a [profiles.local] table directly inside config.toml. Recent Codex CLI versions don't read that format, and the legacy block can sometimes break startup — the profile needs to live in its own <name>.config.toml file. Field names change between versions, so double-check the current documentation.

Run Codex on the local profile:

export OLLAMA_API_KEY="local"
codex --profile local

If the open gpt-oss models are good enough for your needs, there's a shortcut: the --oss flag spins up the Ollama pairing automatically, no manual config editing required.

codex --oss -m gpt-oss:20b

The Same Setup in Codex App and the IDE CLI

Codex CLI isn't the only client. The desktop app and the editor extension read the same ~/.codex/config.toml, so they connect to the local model with identical settings.

Codex App (macOS/Windows). The easiest route is through Ollama — the command sets up the local endpoint and launches the app for you:

ollama launch codex-app
# with a specific model
ollama launch codex-app --model qwen2.5-coder:14b

Codex App support arrived in Ollama version 0.24.0 and later.

IDE extension (VS Code / Codium). Works off the same config.toml: open Codex settings → "Open config.toml" and point it at the local provider. Switching models currently requires editing the config and restarting the editor — there's no built-in selector for local models yet.

⚠️ Note: switching to a local model changes the Codex App interface — instead of the cloud model selector, you get a reasoning-effort toggle. Switching back to cloud models can occasionally hang, so change providers deliberately.

Step 4: Verify Everything Works

• curl http://localhost:11434/v1/models returns your model
• Open WebUI opens and responds in chat
• Codex on the local profile completes a simple task (e.g., explains a file)
• While Codex is running, there are no outgoing requests to api.openai.com (check in a network monitor)

That last check is the one that matters most — it confirms your data is genuinely staying local.

Where This Setup Fits Best

• NDA-protected code and private repos. Reviewing, refactoring, and explaining legacy code that can't be sent to the cloud. Codex on the local profile reads and edits files right in your working directory.
• Sensitive data. Logs with personal information, database exports, internal reports — have Codex write and run a processing script instead of feeding it the raw data; the data itself never leaves the machine.
• A private chat over your internal knowledge base. Open WebUI with RAG turns the local model into a reference tool for your team's documents, with no content leakage.
• Offline and air-gapped work. On a plane, at a site with no internet, in an isolated network — the model is already on your hardware and doesn't depend on connectivity.
• High-volume routine work. Generating tests, docstrings, draft commit messages, and repetitive edits where you'd rather not pay per token or hit rate limits.
• A hybrid workday. Sensitive and routine tasks on the local model, complex architecture work by switching to cloud Codex. One tool, different profiles.

💡 Example: you need to parse a log dump containing personal data. Run codex --profile local in the folder with the file and ask: "read access.log, find the top 20 IPs by number of 5xx errors, and produce a CSV." The model reads the file, writes a script, and returns the result — the log never goes anywhere.

Common Mistakes

• The model doesn't fit in memory. The system starts swapping and responses take minutes. Drop to a smaller model or a quantized version.
• Codex goes silent or fails on tool calls. A coding agent needs a model with tool-calling support — regular chat models won't cut it.
• Open WebUI can't see Ollama. Usually a Docker networking issue: check the --add-host=host.docker.internal:host-gateway flag and the connection address.
• Port 11434 isn't reachable from outside. That's expected — access is closed by default. Only open it through a VPN or a tunnel.

Is It Worth It?

⚖️ The trade-off: a local LLM wins on privacy and cost at scale, but loses to frontier models on complex code quality and on speed if your hardware is modest. Keep it hybrid: sensitive and routine tasks on the local model, heavy development on cloud Codex.

Anti-Patterns to Avoid

• ❌ Running a 70B model on a laptop and expecting decent speed — you'll hit a memory wall
• ❌ Picking a chat model for a coding agent without tool-calling support
• ❌ Exposing Ollama to the internet for remote access instead of using a VPN
• ❌ Abandoning frontier models entirely — a local model struggles on genuinely complex code
• ❌ Feeding private documents into a cloud RAG setup when the entire point of this exercise is keeping things local

If you're working with sensitive data — or simply don't want to depend on the cloud — this kind of local setup buys real peace of mind: the model, the code, and the documents all stay on your own hardware.