Smart Model Swap

From its catalog listing

Run a big and a small local LLM on one GPU rig and let the agent switch between them by task: the small one for quick or voice chat and to free VRAM for GPU jobs (ComfyUI), the big one for deep work. SmartModelProvider launches and stops llama.cpp's llama-server itself, swaps in the background without cutting replies, checks per-GPU VRAM before loading, restores the big model when idle, and ships a launch-parameter editor in the dashboard and Desktop, plus a bundled routing skill. Ollama, vLLM and LM Studio are detection-only.

Good to know before installing

starts and stops llama-server as a long-running background process (in a transient systemd-run --user scope when available), autostarts the primary model when the gateway starts and swaps back to it if a smaller model was left serving (GPU memory), and keeps a detached runner per swap and a watcher that restores the primary after inactivity and, by default, loads another registered model if the serving one dies; sends SIGINT/SIGKILL only to the engine process it manages or an engine of the same kind holding its port; runs nvidia-smi, ss, systemctl --user show, systemd-run and llama-server --help/--version with a minimal environment (no provider keys); reads /proc of other processes to identify the engine and GPU users, and its guard reads shell scripts a terminal command is about to run; adds one-line hints to turns, can veto terminal/execute_code calls that manage the engine by hand, and can end a turn with an explanation while the LLM is unloaded on purpose; writes its own settings to config.yaml and, only on request, installs a model-provider plugin under plugins/model-providers/smart-model-provider; HTTP only to the configured engine and ComfyUI endpoints (localhost by default). No telemetry, credentials, self-updater or approval bypass.

From its catalog listing.

How to install

  1. Install it. Hermes asks for any API keys it needs, then asks whether to turn it on.
    hermes plugins install smart-model-swap
  2. Said no? Turn it on later:
    hermes plugins enable smart-model-swap
  3. Restart Hermes so it loads the plugin.

Some plugins also need a sign-in or settings: see Full setup & settings. Memory plugins ask whether to become your memory provider instead. Steps follow the official plugins guide (opens in a new tab).

Category
AI models & providers · Chat apps & voice
Maintained by
CarmaOS-Dev (Community)
Needs
No API keys declared
Adds tools
sms_status sms_plan sms_swap sms_restore
Adds hooks
pre_llm_call pre_tool_call pre_transcription
Works on
linux
Needs Hermes
0.23.0 or newer
Version
0.4.0
In the catalog
2026-11-14

Known issues (from its catalog listing)

  • Needs an NVIDIA GPU with nvidia-smi and a built llama.cpp llama-server; only llama.cpp is managed (Ollama, vLLM and LM Studio are detection-only).
  • On WSL2, GGUFs on a Windows drive (/mnt/<drive>, 9p) load ~10x slower than from the Linux filesystem.
  • Keep the same -c on every model: Hermes caches one context length per model id (the editor enforces it).
Using the Hermes desktop app? See where plugins live
Hermes desktop Capabilities page, Plugins tab filtered to Official plugins built by Nous Research, each with an add button.
The official plugin catalog built into the app.Hermes desktop v0.22.5 · Capabilities > Plugins · Windows · captured Nov 1, 2026

Full setup & settings ↗Source code ↗

More in ai models & providers

All ai models & providers plugins

New Hermes Agent updates every day on X. Follow @HermesWatcher

← All plugins