Hermes Agent local model won't start or goes silent

What you see

You point Hermes at a local server and one of these happens:

• Hermes refuses to start and says the server's context window is too small.
• The startup line shows something like Context limit: 2048 tokens.
• Replies are empty, an empty-stream error appears, or the turn ends with Context length exceeded: compression could not reduce the rebuilt request below the safe threshold.
• The model loses track of the conversation after a few turns, or answers incoherently.
• Tool calls show up as JSON text in the reply instead of actually running.
• Long prompts time out before the first word appears.

Likely causes

Hermes needs at least 64,000 tokens of context for agent work with tools, and rejects smaller windows at startup. The system prompt and tool schemas take up a large share of the window before you've typed anything. The docs specifically note that Hermes' full default toolset can overflow a 32k window and cause an empty-stream error on llama.cpp-family servers. Most local servers then quietly drop older messages instead of raising an error.

The second most common cause is a mismatch between what the server actually serves and what Hermes believes it serves. Ollama's model info reports the model's maximum context, not the num_ctx you set, and you can't set context length through the OpenAI-compatible API at all. On llama.cpp, parallel slots split the context, so -c 64000 -np 4 leaves each slot with only 16k. Tool calls that appear as text mean the server's tool-calling support isn't switched on.

Check this first

• What Hermes thinks: the Context limit line at startup, or /usage during a session.
• What Ollama serves: ollama ps, in the CONTEXT column.
• What llama.cpp serves: curl http://localhost:8080/props and look for n_ctx under the default generation settings.
• What vLLM serves: the --max-model-len value it was started with.
• Whether the endpoint answers at all: curl http://localhost:11434/v1/models (use your own port) should return a JSON list of models.

Fix

First, raise the context on the server, since that's where it's actually decided. For Ollama, start the server with OLLAMA_CONTEXT_LENGTH=64000 ollama serve, set the same variable on a systemd-managed service, or bake it into a model with a Modelfile containing PARAMETER num_ctx 64000. For llama.cpp, start with -c 64000 (and don't split it across parallel slots below the minimum). For vLLM, use --max-model-len.

Next, tell Hermes the same number. Set model.context_length in config.yaml, for example context_length: 64000 beneath your default, provider: custom and base_url lines. For a named custom provider, set it per model under providers:. On local endpoints, model.ollama_num_ctx is also honoured as the window the server really serves. The two numbers must match: if Hermes believes it has more room than the server provides, requests get truncated.

• Tool calls appearing as text: add --jinja to llama.cpp, --enable-auto-tool-choice --tool-call-parser hermes to vLLM, or the right --tool-call-parser to SGLang. For Ollama, confirm the model supports tools with ollama show <model>. LM Studio needs version 0.3.6 or later and a model with native tool support.
• The window really is tight: start with fewer tools, for example hermes chat -t file,web, and switch off unused toolsets with hermes tools.
• Timeouts during long prompts: Hermes already raises the streaming read timeout for local endpoints. If prompts still time out, set HERMES_STREAM_READ_TIMEOUT=1800 in ~/.hermes/.env.
• Replies cut off mid-sentence: that's usually the server's own output limit, which Hermes doesn't control, or a full context. /compress frees up room.

If it still fails

If you see 'The model server rejected this request as too large, but this conversation is only about N tokens', another request is probably holding a single-slot server (LM Studio and Ollama are common examples). The docs name a background memory review from an earlier session as the usual culprit, which shows up as thread=bg-review in logs/agent.log. Wait a moment and run /retry. If it keeps happening with no other Hermes process running, the server loaded the model with a smaller window than Hermes expects.

If Hermes runs in WSL2 and the model server runs on Windows, localhost won't reach it under WSL's default networking. Enable mirrored networking (networkingMode=mirrored in .wslconfig, then wsl --shutdown), or use the Windows host's address. If none of this helps, test the endpoint in another client to confirm it really is OpenAI-compatible, then run hermes doctor.

Related guides

• How to choose a model for Hermes Agent
• Hermes Agent forgets things mid-conversation
• Hermes Agent rate limits and provider errors
• Hermes Agent still using the wrong model

Spotted something wrong or out of date? Tell us on X at @HermesWatcher↗ (opens in a new tab) and we will fix it.

New Hermes Agent updates every day on X. Follow @HermesWatcher