You can run LOPs entirely against a local model server — no cloud API key required. Start the server, point ChatTD at it, pick a model, and test from an Agent.

Local Provider Path ⤢ expand

localhost /v1model defaultstool callsresults

◇ external

Local Server

LM Studio / Ollama / llama.cpp

▲ process

ChatTD

provider routing

◆ agent

Agent

model call

■ tool

Tools

optional

LM Studio

LM Studio connects through its OpenAI-compatible local server. LOPs routes through LiteLLM’s /v1 interface.

1. Start the LM Studio local server.
2. In ChatTD, choose LM Studio or set up a Custom OpenAI-compatible endpoint.
3. Use the model prefix expected by the route — lm_studio/{model} when going through LiteLLM.
4. Refresh the model list or type the model ID.
5. Send a plain Agent message to confirm it works before enabling tools.

Ollama

Ollama runs at http://localhost:11434 by default. ChatTD routes Ollama calls through LiteLLM with model names like ollama_chat/{model}.

1. Pull the model you want in Ollama.
2. Choose the Ollama provider in ChatTD.
3. Select or type the model name.
4. Test with a simple Agent prompt.

llama.cpp and Other OpenAI-Compatible Servers

For llama.cpp-style servers, set up a Custom OpenAI-compatible endpoint in ChatTD or route through a LiteLLM proxy.

CAUTION Tool calling needs --jinja

llama.cpp needs chat template support for tool calling. Run the server with the --jinja flag — without it, tool calls may come back as raw text instead of structured calls.

Testing Tool Support

Not every local model handles tool calling reliably. Before building a workflow around a local model, test this sequence:

1. Plain chat response.
2. Structured JSON response, if your workflow needs it.
3. A simple tool call — try Tool Request or another low-stakes tool.
4. A follow-up response after the tool result.

If any step fails, the issue is usually the model or its template support, not the Agent or tool wiring.

When to Use Local vs. Cloud

Use local models when you want offline operation, lower recurring API costs, local data handling, or custom/fine-tuned models. Use cloud providers when you need capabilities your local setup does not handle reliably — tool calling, vision, or large context windows in particular.

Local does not automatically mean private. LOPs traces, provider logs, model downloads, and sidecar caches may still be written to disk. See Local Data and Files for what gets stored.