Quick Eval

From its catalog listing

Get a second opinion on any answer. /quick-eval sends your last question to other models, shows the answers side by side, and records which ones you mark good. Runs and picks go to an append-only JSONL file, and a review page (/quick-eval open: the Quick Eval page in Hermes Desktop on Hermes 0.22.3 or newer, or a browser page elsewhere; also a tab in hermes dashboard) shows each comparison and a model leaderboard.

Good to know before installing

sends your last question (only that message) to the models you name through Hermes' configured providers; probes localhost ports 11434, 1234, 8000 and 8080 for local model servers and, the first time you name a model one of them serves, adds that server as a `providers:` entry in config.yaml; asks for the model/provider-override permission on Hermes' consent screen (`/quick-eval allow` writes it into config.yaml on older Hermes); writes ~/.hermes/evals/evals.jsonl; reads ~/.hermes/state.db read-only to show your sessions; `/quick-eval open` outside Hermes Desktop serves the review page, which includes full session transcripts, on 127.0.0.1:8765 (or a free port) until Hermes exits, locked to a random per-launch token in the opened URL.

From its catalog listing.

How to install

  1. Install it. Hermes asks for any API keys it needs, then asks whether to turn it on.
    hermes plugins install quick-eval
  2. Said no? Turn it on later:
    hermes plugins enable quick-eval
  3. Restart Hermes so it loads the plugin.

Some plugins also need a sign-in or settings: see Full setup & settings. Memory plugins ask whether to become your memory provider instead. Steps follow the official plugins guide (opens in a new tab).

Category
AI models & providers · Web, research & browsing
Maintained by
tikiverse (Community)
Needs
No API keys declared
Adds hooks
post_llm_call
Needs Hermes
0.19 or newer
Version
0.1.0
In the catalog
2026-11-14

Known issues (from its catalog listing)

  • Local thinking models (Qwen3 and similar) can return no answer on a server with a small default token cap: Hermes drops a custom provider's extra_body and max_tokens on plugin calls (hermes-agent#124290, #50132). Turn thinking off on the server.
Using the Hermes desktop app? See where plugins live
Hermes desktop Capabilities page, Plugins tab filtered to Official plugins built by Nous Research, each with an add button.
The official plugin catalog built into the app.Hermes desktop v0.22.5 · Capabilities > Plugins · Windows · captured Nov 1, 2026

Full setup & settings ↗Source code ↗

More in ai models & providers

All ai models & providers plugins

New Hermes Agent updates every day on X. Follow @HermesWatcher

← All plugins