Smart Model Swap
From its catalog listing
Run a big and a small local LLM on one GPU rig and let the agent switch between them by task: the small one for quick or voice chat and to free VRAM for GPU jobs (ComfyUI), the big one for deep work. SmartModelProvider launches and stops llama.cpp's llama-server itself, swaps in the background without cutting replies, checks per-GPU VRAM before loading, restores the big model when idle, and ships a launch-parameter editor in the dashboard and Desktop, plus a bundled routing skill. Ollama, vLLM and LM Studio are detection-only.
Good to know before installing
starts and stops llama-server as a long-running background process (in a transient systemd-run --user scope when available), autostarts the primary model when the gateway starts and swaps back to it if a smaller model was left serving (GPU memory), and keeps a detached runner per swap and a watcher that restores the primary after inactivity and, by default, loads another registered model if the serving one dies; sends SIGINT/SIGKILL only to the engine process it manages or an engine of the same kind holding its port; runs nvidia-smi, ss, systemctl --user show, systemd-run and llama-server --help/--version with a minimal environment (no provider keys); reads /proc of other processes to identify the engine and GPU users, and its guard reads shell scripts a terminal command is about to run; adds one-line hints to turns, can veto terminal/execute_code calls that manage the engine by hand, and can end a turn with an explanation while the LLM is unloaded on purpose; writes its own settings to config.yaml and, only on request, installs a model-provider plugin under plugins/model-providers/smart-model-provider; HTTP only to the configured engine and ComfyUI endpoints (localhost by default). No telemetry, credentials, self-updater or approval bypass.
From its catalog listing.
How to install
- Install it. Hermes asks for any API keys it needs, then asks whether to turn it on.
hermes plugins install smart-model-swap - Said no? Turn it on later:
hermes plugins enable smart-model-swap - Restart Hermes so it loads the plugin.
Some plugins also need a sign-in or settings: see Full setup & settings. Memory plugins ask whether to become your memory provider instead. Steps follow the official plugins guide (opens in a new tab).
- Category
- AI models & providers · Chat apps & voice
- Maintained by
- CarmaOS-Dev (Community)
- Needs
- No API keys declared
- Adds tools
sms_statussms_plansms_swapsms_restore- Adds hooks
pre_llm_callpre_tool_callpre_transcription- Works on
- linux
- Needs Hermes
- 0.23.0 or newer
- Version
- 0.4.0
- In the catalog
- 2026-11-14
Known issues (from its catalog listing)
- Needs an NVIDIA GPU with nvidia-smi and a built llama.cpp llama-server; only llama.cpp is managed (Ollama, vLLM and LM Studio are detection-only).
- On WSL2, GGUFs on a Windows drive (/mnt/<drive>, 9p) load ~10x slower than from the Linux filesystem.
- Keep the same -c on every model: Hermes caches one context length per model id (the editor enforces it).
Using the Hermes desktop app? See where plugins live

Full setup & settings ↗Source code ↗
More in ai models & providers
All ai models & providers plugins
New Hermes Agent updates every day on X. Follow @HermesWatcher