OpenClaw LLM Setup: Connect Local Models, Cloud Providers, and Fallbacks

To set up an LLM in OpenClaw, run openclaw onboard on the machine running the Gateway, connect your provider or local inference server, and select a model as provider/model. Check the model catalog and authentication, then test a short reply and one harmless tool task in the chat you intend to use.

For local Ollama, choose Local only during onboarding and use the native Ollama connection. That route uses /api/chat; adding /v1 selects a different, OpenAI-compatible route. For cloud models, authenticate the account that will serve the Gateway and choose a model that account can actually access. For recovery, configure fallbacks on the default model, then leave the chat on Default: manually choosing a specific model creates a strict session selection.

The procedures below follow OpenClaw documentation checked on October 4, 2026. They are documented setup instructions; this update did not include a live provider call or local hardware benchmark.

Choose the connection you can operate

Start with where inference should run and which account or server you already control. A local model is a useful starting point when you want to keep inference on your own hardware. A cloud provider is appropriate when you want hosted inference without maintaining a local model server. Neither choice establishes that a particular model can complete your agent's work; test that separately.

For an always-on Gateway, OpenClaw's authentication guide recommends an API key as the most predictable option. An interactive native CLI login can fit a personal workstation, provided the Gateway runs as the user who owns that login and can launch the executable.

A local inference route also needs a data boundary. Cloud fallback, online tools, plugins, and other model-backed features can send content outside the local server. If a task must remain local, keep its model selection strict and check those features as well. Ollama's Cloud + Local mode and models ending in :cloud perform remote inference even when requests pass through your own daemon. Ollama setup explains the modes.

Connect Ollama on the Gateway host

The simplest local setup puts Ollama and the Gateway on the same machine. Install Ollama from its official download page, start its desktop application or service, and confirm that its CLI works:

ollama list

If you are running Ollama manually and no daemon is already running, start it in a separate terminal:

ollama serve

Keep that process running while you connect OpenClaw. Pull a model you intend to test. OpenClaw's current local recipe uses gemma4; this is a documented example, not a claim that it fits every computer:

ollama pull gemma4
ollama list

Model fit depends on weights, quantization, available memory, context length, and the runtime. Choose a size and context your machine can load rather than relying on a fixed RAM-per-parameter rule. Downloading the model successfully does not establish that it can run with the agent's tool schemas and context.

Run the guided local connection

openclaw onboard

Choose Ollama, then Local only. Keep the default local host when Ollama runs beside the Gateway. Select the installed model and let setup complete its live check. If desktop Model Setup does not offer an installed but idle model automatically, choose Choose connection on the Ollama card, then Local only.

Fresh guided automatic selection considers models already loaded in memory, reported by /api/ps, with tool support and at least 16K context confirmed by /api/show. An installed model sitting idle on disk is not an automatic candidate. The explicit local setup path can prepare an eligible installed model for the live check. Automatic discovery does not pull or load an idle model, and setup saves the selected route only after a real completion. These conditions come from the current Ollama setup documentation.

After onboarding, inspect the catalog and selection:

openclaw models list --provider ollama
openclaw models status

If you need to select the documented example explicitly, and that exact name resolves on your server:

openclaw models set ollama/gemma4

Use the exact installed name or tag when your catalog differs. General implicit catalog discovery queries /api/tags and /api/show; it is distinct from the already-loaded gate used by fresh guided setup. A nonempty models.providers.ollama.models list selects manual models and skips discovery. Model discovery describes both the catalog behavior and exact-ref resolution.

For a manually operated same-host CLI setup, OLLAMA_API_KEY=ollama-local is the nonsecret marker documented for local discovery. Interactive onboarding is preferable for a service because it avoids assuming that an environment variable exported in your terminal will reach the daemon.

Keep the native API and the correct host

Native Ollama configuration uses api: "ollama" and a base URL without /v1. OpenClaw's native adapter supports streaming and tool calling together. Older instructions to disable streaming universally or add /v1 for every Ollama connection no longer describe this route. See Ollama configuration and streaming configuration.

If Ollama runs on another computer, supply that computer's reachable address during setup. The server must listen on an interface reachable from the Gateway, and the network must permit the connection. A daemon reachable from your laptop is insufficient when the Gateway runs on a VPS, another service host, or in a container with a different network namespace. A hostname alone cannot make a loopback-only listener accessible.

OpenClaw uses the ollama-local marker for local, private-network, and .local Ollama hosts. Public remote servers and direct Ollama Cloud need real credentials. For direct hosted use without a daemon, choose the dedicated ollama-cloud provider. These are host-specific authentication rules, not a general permission to use an arbitrary key on every Ollama endpoint.

Connect a cloud provider and select its model

On the Gateway host, run openclaw onboard, choose the provider and supported authentication method, and complete the prompts. In the Control UI, Settings → Models → Connect provider configures an agent's model credential. A personal account connected under Settings → Profile → Connected accounts has a different scope; it does not replace the agent's shared credential. OpenClaw's OpenAI authentication page documents that distinction.

Use the provider's current catalog instead of copying a model ID from an old recommendation:

openclaw models list --provider openai
openclaw models status

Change openai to the provider ID you connected. Copy the exact provider/model ref you intend to use and run openclaw models set with that ref. For example, the local command above selects ollama/gemma4; the same command works with a supported cloud ref.

A successful login usually makes the provider available while preserving an existing primary model. To change the default deliberately, use models set or a supported auth command with --set-default. The provider quick rules explain this preservation behavior.

Keep these provider differences in mind when choosing the authentication method:

• OpenAI: current model refs use openai/ for both API-key and supported ChatGPT/Codex login routes. The credential and endpoint determine access and billing; the prefix alone does not select the Codex runtime. Legacy openai-codex/ and codex/ configurations need the documented migration rather than new copies of those prefixes. See OpenAI setup.
• Anthropic: direct API-key access and native Claude CLI are separate routes. A native CLI route needs the installed, authenticated claude executable available to the Gateway user. OpenClaw does not own or refresh that executable's native login. See Anthropic setup.
• Google: the canonical provider is google; standard setup uses a Google AI Studio API key. The optional google-gemini-cli runtime is a separate runtime choice, not a different model identity or a promise of consumer OAuth access. See Google setup.
• Bedrock: amazon-bedrock uses the AWS SDK credential chain, region, and model permissions. An Anthropic API key does not authenticate this route. Follow the Bedrock provider instructions for your AWS environment.

Model credentials and the Gateway access token serve different connections. One authorizes inference; the other lets a client connect to OpenClaw. Provider endpoint settings belong in configuration, while current managed authentication uses SQLite with shared credentials and agent-local overrides. Avoid hand-editing an old auth-profiles.json as though it were the current runtime store. OAuth storage documentation describes the ownership and migration rules.

Use LM Studio, vLLM, or a custom endpoint

For LM Studio, install its desktop application or headless llmster, prepare a model, and start the server:

lms server start --port 1234
openclaw onboard

Choose LM Studio and select the model at the default-model prompt. If the server's authentication is disabled, leave the key blank. If it is enabled, use its configured token, supported through LM_API_TOKEN.

LM Studio uses native /api/v1/models discovery and load endpoints, then /v1/chat/completions for inference. Its model keys can include an author namespace: qwen/qwen3.5-9b becomes lmstudio/qwen/qwen3.5-9b in OpenClaw. Fresh guided selection requires reported tool training and at least 16K effective context; a loaded instance's actual context takes precedence over a larger advertised maximum. Follow the LM Studio guide for the exact key and loading options.

vLLM uses api: "openai-completions" and compatible /v1 endpoints. Discovery is enabled through VLLM_API_KEY; a nonsecret value is appropriate only when your server does not enforce authentication. Tool execution also depends on the model's parser and chat template. Printed JSON describing a tool is not an executed tool call. The vLLM provider guide gives the model-specific configuration path.

For either remote server, the Gateway must reach the actual listener. Preserve /v1 for these compatible inference routes; the native Ollama instruction to omit it does not apply universally.

A custom relay needs its own models.providers entry with the correct baseUrl, API adapter, model IDs, and capabilities. An alias in agents.defaults.models alone does not register a new runtime model. Set metadata to the server's real limits: contextWindow describes native capacity, contextTokens caps active input, and maxTokens limits output. A zero in cost metadata is not a billing guarantee. Custom provider documentation explains these fields and which native-provider features compatible endpoints do not inherit.

Verify the chat and actual tool use

Check four things before giving the agent a large job:

1. Inventory: the intended provider and exact model appear or resolve through the provider's supported discovery path.
2. Agent: openclaw models status shows the expected configured default, fallback policy, authentication, and runtime. If multiple agents exist, inspect the affected agent explicitly with --agent and its actual ID.
3. Session: run /model status in the chat. Check its model, runtime, and selected account. The CLI's agent status does not inspect a chat's model override.
4. Execution: send a short message, then ask for a harmless tool task in that same chat.

For the message, ask: “Reply with exactly: ready.” For the tool task, create a disposable file named setup-check.txt in the agent's workspace containing a sentence you can verify. Ask the agent: “Read setup-check.txt using your file tool, quote its first line, and do not modify any files.” Confirm a real tool invocation and the matching result. Text that merely claims to have read a file does not pass this check.

For a narrow local Ollama test, the documentation also provides the inference CLI:

openclaw infer model run \
  --local \
  --model ollama/gemma4 \
  --prompt "Reply with exactly: ready" \
  --json

Use your exact installed ref if it differs. This bypasses chat tools, memory, and session context. A successful reply establishes a narrower inference path; it does not prove that the normal agent can use tools. Ollama smoke tests explain that scope.

Status without --probe is not a model-call test. Direct models status --probe makes real requests, may consume paid usage, and requires exclusive state-directory ownership: stop the running Gateway first and wait for probe cleanup before restarting it. For initial setup, the short reply in your intended chat is the more direct check. See the models CLI reference.

Configure fallback strategies without losing a strict selection

A fallback chain recovers from eligible provider failures; it does not automatically assign cheap models to easy tasks. Configure only alternatives you have authenticated and tested. For a local primary plus an Ollama cloud backup, the official hybrid recipe uses this model object:

{
  agents: {
    defaults: {
      model: {
        primary: "ollama/gemma4",
        fallbacks: ["ollama/kimi-k2.5:cloud"],
      },
    },
  },
}

Merge this object into your existing openclaw.json configuration rather than replacing the entire file. This example assumes a signed-in Ollama daemon, exact model availability, and permission to send task content to cloud inference. Run ollama signin on that daemon's host to configure hybrid cloud access, then verify the backup separately before relying on it. The Ollama recipes document this connection.

If ollama/gemma4 is already your default, you can add the backup through the CLI instead of editing the file. Review the existing chain first, then add the verified model and inspect the result:

openclaw models fallbacks list
openclaw models fallbacks add ollama/kimi-k2.5:cloud
openclaw models status

These commands manage the global default fallback list; an agent-specific model object can have its own policy. The models CLI reference describes the command scope.

OpenClaw first attempts bounded recovery for eligible temporary failures and rotates usable auth profiles within the provider before advancing to another model. The current model failover documentation distinguishes selection sources:

To use the default and its recovery policy in an existing chat, run:

/model default
/model status

Changing the configured primary does not clear an existing session pin. To keep a sensitive task on one exact local model, select that model deliberately instead:

/model ollama/gemma4

If it becomes unreachable, that strict selection fails visibly. An operator can then decide whether to repair the server or choose another model.

Fallback cannot guarantee uninterrupted work. A cancellation, terminal run deadline, budget stop, or exhausted chain can end the turn. A recovery may also follow partial output or completed tool actions: inspect the recorded work before repeating an action. /status can show the selected model and the fallback that actually answered. Current fallback execution is turn-local; older stored automatic overrides from releases 2026.4.26 through 2026.6.0 have separate recovery behavior described in the failover documentation.

Fix the setup symptom you actually see

Use the following current checks with the illustrated error categories. The exact provider, process, and selection matter more than a generic quick fix.

For credential recovery, the OpenClaw 401 and missing-header guide separates upstream provider auth, native CLI login, agent-local overrides, and Gateway/client tokens. Those failures need different repairs.

Current model policy uses agents.defaults.modelPolicy.allow; agents.defaults.models holds aliases and settings. An omitted or empty explicit policy does not make an unsupported model, unknown provider, or inaccessible account usable. Older model maps may still need migration. Model selection documentation explains this distinction.

Compare local, proxy, and cloud inference on your own task

Choose a model by a small repeatable workflow test. Use the same disposable file, prompt, permissions, and intended output on each route. Record whether the model used the tool correctly, returned the correct result, how long the first and later turns took, and the cloud usage charged to the selected account.

Separate model loading from steady-state replies, and separate a proxy's network route from the model it forwards to. If the proxy and direct provider serve different models, accounts, context budgets, or tool schemas, the result does not isolate proxy overhead. A local text-only reply also cannot establish parity with a cloud agent completing file and tool work.

This gives you a useful local-versus-cloud comparison without inventing universal token-per-second or GPU-speed claims. There is no measured local/proxy/cloud winner established by this guide. Keep the route that completes your actual task within your acceptable latency, data boundary, and budget; use a different default or agent for work with different requirements.

FAQ

Which local LLM is best for OpenClaw?

Start with a tools-capable model that fits your hardware and effective context, then run the short reply and read-only file test. A parameter count or successful model download cannot establish agent reliability. Fresh guided Ollama setup requires tools and at least 16K context; general catalog discovery has a different scope. Ollama setup describes the selection gate.

Does Ollama need /v1 in OpenClaw?

Native Ollama uses api: "ollama" with a base URL without /v1, and supports streaming plus tools through /api/chat. LM Studio and vLLM compatible inference routes use /v1. Choose the path for the server and adapter you are actually using. See native Ollama configuration.

Why did adding a provider leave my old model selected?

Adding or renewing provider authentication preserves an existing primary unless you intentionally set a new default. An existing chat can also retain its own model pin. Set the configured default with openclaw models set, then use /model default in the chat when you want that default. Provider quick rules document both behaviors.

Will OpenClaw fall back if I manually choose a local model?

An explicit user session selection is strict. If that local model fails, OpenClaw reports the failure rather than silently sending the task to an unrelated configured cloud backup. A chat using the configured default can use that default's fallbacks. Model failover defines these selection rules.

Is running a local model free and completely private?

Self-hosted local inference avoids a hosted token charge for that inference, but hardware, electricity, and other services can still cost money. It also does not make every OpenClaw feature local. Check cloud models, fallbacks, tools, plugins, and other model-backed calls before treating a workflow as local-only.