# witcheer's Wingtips #29: local models context floor

Written by [witcheer](https://x.com/witcheer) at Nous Research. Republished with credit.

Canonical: https://hermeswatcher.com/wingtips/29  
Published: 2026-07-24  
[Original](https://notwitcheer.github.io/hermes-recipes/wingtips/29-local-models-context-floor/) · [On X](https://x.com/witcheer/status/2080682763594899763) · [Full collection](https://notwitcheer.github.io/hermes-recipes/) · [Official docs](https://hermes-agent.nousresearch.com/docs/integrations/providers)

[Source Markdown](https://raw.githubusercontent.com/notwitcheer/hermes-recipes/8653fb699a137a32e81936fa33cabacd3f4f2095/docs/wingtips/29-local-models-context-floor.md): use this file as the base for relative links in the verbatim text below.

Sources: [Hermes docs: Providers](https://hermes-agent.nousresearch.com/docs/integrations/providers)


Hermes Wingtips #29: local models context floor

you point Hermes Agent at Ollama, the first few turns are fine, then the agent starts forgetting instructions or repeating work. 

the usual cause is that Ollama's default context depends on your VRAM, and under 24 GB it is 4,096 tokens. Hermes needs at least 64,000 for agent work, because the system prompt and tool schemas alone take 4k to 8k, and most servers drop your oldest messages once the window fills.

here is what to do server-side:

(1) Ollama: `OLLAMA_CONTEXT_LENGTH=64000 ollama serve`, then check the CONTEXT column in ollama ps

(2) vLLM: `--max-model-len 64000`

(3) llama.cpp: `llama-server -m model.gguf -c 64000`

LM Studio users can skip this since Hermes asks it to load the model at 64K by default.