#29 · local models context floor
Hermes Wingtips #29: local models context floor
you point Hermes Agent at Ollama, the first few turns are fine, then the agent starts forgetting instructions or repeating work.
the usual cause is that Ollama's default context depends on your VRAM, and under 24 GB it is 4,096 tokens. Hermes needs at least 64,000 for agent work, because the system prompt and tool schemas alone take 4k to 8k, and most servers drop your oldest messages once the window fills.
here is what to do server-side:
(1) Ollama: OLLAMA_CONTEXT_LENGTH=64000 ollama serve, then check the CONTEXT column in ollama ps
(2) vLLM: --max-model-len 64000
(3) llama.cpp: llama-server -m model.gguf -c 64000
LM Studio users can skip this since Hermes asks it to load the model at 64K by default.