# Hermes Agent local model won't start or goes silent > Fix the context-window and tool-calling mismatches behind most local model failures with Ollama, llama.cpp, vLLM and LM Studio. Canonical: https://hermeswatcher.com/troubleshoot/local-model-context-too-small Section: Troubleshooting ยท Models, providers & cost Last reviewed: Oct 20, 2026 Basis: Explained from the official Hermes Agent docs and repository (Nous Research). Steps follow the linked official sources. ## What you see You point Hermes at a local server and one of these happens: - Hermes refuses to start and says the server's context window is too small. - The startup line shows something like `Context limit: 2048 tokens`. - Replies are empty, an empty-stream error appears, or the turn ends with `Context length exceeded: compression could not reduce the rebuilt request below the safe threshold`. - The model loses track of the conversation after a few turns, or answers incoherently. - Tool calls show up as JSON text in the reply instead of actually running. - Long prompts time out before the first word appears. ## Likely causes Hermes needs at least 64,000 tokens of context for agent work with tools, and rejects smaller windows at startup. The system prompt and tool schemas take up a large share of the window before you've typed anything. The docs specifically note that Hermes' full default toolset can overflow a 32k window and cause an empty-stream error on llama.cpp-family servers. Most local servers then quietly drop older messages instead of raising an error. The second most common cause is a mismatch between what the server actually serves and what Hermes believes it serves. Ollama's model info reports the model's maximum context, not the `num_ctx` you set, and you can't set context length through the OpenAI-compatible API at all. On llama.cpp, parallel slots split the context, so `-c 64000 -np 4` leaves each slot with only 16k. Tool calls that appear as text mean the server's tool-calling support isn't switched on. ## Check this first - What Hermes thinks: the `Context limit` line at startup, or `/usage` during a session. - What Ollama serves: `ollama ps`, in the CONTEXT column. - What llama.cpp serves: `curl http://localhost:8080/props` and look for `n_ctx` under the default generation settings. - What vLLM serves: the `--max-model-len` value it was started with. - Whether the endpoint answers at all: `curl http://localhost:11434/v1/models` (use your own port) should return a JSON list of models. ## Fix First, raise the context on the server, since that's where it's actually decided. For Ollama, start the server with `OLLAMA_CONTEXT_LENGTH=64000 ollama serve`, set the same variable on a systemd-managed service, or bake it into a model with a Modelfile containing `PARAMETER num_ctx 64000`. For llama.cpp, start with `-c 64000` (and don't split it across parallel slots below the minimum). For vLLM, use `--max-model-len`. Next, tell Hermes the same number. Set `model.context_length` in `config.yaml`, for example `context_length: 64000` beneath your `default`, `provider: custom` and `base_url` lines. For a named custom provider, set it per model under `providers:`. On local endpoints, `model.ollama_num_ctx` is also honoured as the window the server really serves. The two numbers must match: if Hermes believes it has more room than the server provides, requests get truncated. - Tool calls appearing as text: add `--jinja` to llama.cpp, `--enable-auto-tool-choice --tool-call-parser hermes` to vLLM, or the right `--tool-call-parser` to SGLang. For Ollama, confirm the model supports tools with `ollama show `. LM Studio needs version 0.3.6 or later and a model with native tool support. - The window really is tight: start with fewer tools, for example `hermes chat -t file,web`, and switch off unused toolsets with `hermes tools`. - Timeouts during long prompts: Hermes already raises the streaming read timeout for local endpoints. If prompts still time out, set `HERMES_STREAM_READ_TIMEOUT=1800` in `~/.hermes/.env`. - Replies cut off mid-sentence: that's usually the server's own output limit, which Hermes doesn't control, or a full context. `/compress` frees up room. ## If it still fails If you see 'The model server rejected this request as too large, but this conversation is only about N tokens', another request is probably holding a single-slot server (LM Studio and Ollama are common examples). The docs name a background memory review from an earlier session as the usual culprit, which shows up as `thread=bg-review` in `logs/agent.log`. Wait a moment and run `/retry`. If it keeps happening with no other Hermes process running, the server loaded the model with a smaller window than Hermes expects. If Hermes runs in WSL2 and the model server runs on Windows, `localhost` won't reach it under WSL's default networking. Enable mirrored networking (`networkingMode=mirrored` in `.wslconfig`, then `wsl --shutdown`), or use the Windows host's address. If none of this helps, test the endpoint in another client to confirm it really is OpenAI-compatible, then run `hermes doctor`. ## Related guides - [How to choose a model for Hermes Agent](https://hermeswatcher.com/learn/models-and-effective-configuration.md) - [Hermes Agent forgets things mid-conversation](https://hermeswatcher.com/troubleshoot/forgets-mid-conversation.md) - [Hermes Agent rate limits and provider errors](https://hermeswatcher.com/troubleshoot/rate-limits-and-provider-errors.md) - [Hermes Agent still using the wrong model](https://hermeswatcher.com/troubleshoot/model-change-not-taking-effect.md) ## Related tools - [Models and providers: configure the actual inference route](https://hermeswatcher.com/tools/models.md) - [Fallback providers: keep working when your main model fails](https://hermeswatcher.com/tools/fallback-providers.md) - [CLI: run bounded tasks and inspect the working environment](https://hermeswatcher.com/tools/cli.md) ## Sources - [AI Providers: Troubleshooting Local Models (official docs)](https://hermes-agent.nousresearch.com/docs/integrations/providers) - [FAQ: Context length exceeded (official docs)](https://hermes-agent.nousresearch.com/docs/reference/faq) - [Run Local LLMs on Mac (official guide)](https://hermes-agent.nousresearch.com/docs/guides/local-llm-on-mac) - [Quickstart: minimum context (official docs)](https://hermes-agent.nousresearch.com/docs/getting-started/quickstart) --- HermesWatcher is an independent guide. It is not affiliated with or endorsed by Nous Research. The [official Hermes Agent documentation](https://hermes-agent.nousresearch.com/docs) and [repository](https://github.com/NousResearch/hermes-agent) from Nous Research are the authority; each page here lists the official sources it was written from. If this site and the official docs disagree, trust the official docs. Index for agents: https://hermeswatcher.com/llms.txt