Quick Eval
From its catalog listing
Get a second opinion on any answer. /quick-eval sends your last question to other models, shows the answers side by side, and records which ones you mark good. Runs and picks go to an append-only JSONL file, and a review page (/quick-eval open: the Quick Eval page in Hermes Desktop on Hermes 0.22.3 or newer, or a browser page elsewhere; also a tab in hermes dashboard) shows each comparison and a model leaderboard.
Good to know before installing
sends your last question (only that message) to the models you name through Hermes' configured providers; probes localhost ports 11434, 1234, 8000 and 8080 for local model servers and, the first time you name a model one of them serves, adds that server as a `providers:` entry in config.yaml; asks for the model/provider-override permission on Hermes' consent screen (`/quick-eval allow` writes it into config.yaml on older Hermes); writes ~/.hermes/evals/evals.jsonl; reads ~/.hermes/state.db read-only to show your sessions; `/quick-eval open` outside Hermes Desktop serves the review page, which includes full session transcripts, on 127.0.0.1:8765 (or a free port) until Hermes exits, locked to a random per-launch token in the opened URL.
From its catalog listing.
How to install
- Install it. Hermes asks for any API keys it needs, then asks whether to turn it on.
hermes plugins install quick-eval - Said no? Turn it on later:
hermes plugins enable quick-eval - Restart Hermes so it loads the plugin.
Some plugins also need a sign-in or settings: see Full setup & settings. Memory plugins ask whether to become your memory provider instead. Steps follow the official plugins guide (opens in a new tab).
- Category
- AI models & providers · Web, research & browsing
- Maintained by
- tikiverse (Community)
- Needs
- No API keys declared
- Adds hooks
post_llm_call- Needs Hermes
- 0.19 or newer
- Version
- 0.1.0
- In the catalog
- 2026-11-14
Known issues (from its catalog listing)
- Local thinking models (Qwen3 and similar) can return no answer on a server with a small default token cap: Hermes drops a custom provider's extra_body and max_tokens on plugin calls (hermes-agent#124290, #50132). Turn thinking off on the server.
Using the Hermes desktop app? See where plugins live

Full setup & settings ↗Source code ↗
More in ai models & providers
All ai models & providers plugins
New Hermes Agent updates every day on X. Follow @HermesWatcher