Reliability Lab

From its catalog listing

Architecture-aware reliability testing for agent repositories: discover real usage datasets, generate scenarios, review them independently, execute configured targets and inspect evidence in Hermes web and Desktop dashboards. Freeze audited benchmarks and compare regressions across code changes, with baseline references for native Hermes memory.

Good to know before installing

Sends source excerpts from the repository you point it at and sampled public dataset messages to your active Hermes model, queries huggingface.co and zenodo.org (HF_TOKEN only to Hugging Face), clones the supplied HTTPS repositories into profile plugin data, and starts a detached background worker per campaign; execution is off by default, and when enabled and approved through Hermes' prompt it runs generated evaluator tests, model-authored reference actions and the target agent (a cloned Hermes) on this host outside any sandbox, sending the target API key to your configured target base_url, while OpenHands/DeerFlow targets run in a network-disabled Docker image you build.

From its catalog listing.

How to install

  1. Install it. Hermes asks for any API keys it needs (this one needs HF_TOKEN, RELIABILITY_TARGET_API_KEY), then asks whether to turn it on.
    hermes plugins install hermes-reliability
  2. Said no? Turn it on later:
    hermes plugins enable hermes-reliability
  3. Restart Hermes so it loads the plugin.

Some plugins also need a sign-in or settings: see Full setup & settings. Memory plugins ask whether to become your memory provider instead. Steps follow the official plugins guide (opens in a new tab).

Category
Memory & notes
Maintained by
MadBonzz (Community)
Needs
HF_TOKEN RELIABILITY_TARGET_API_KEY API keys
Adds tools
reliability_start reliability_status reliability_stop reliability_evidence reliability_history reliability_freeze reliability_compare reliability_memory
Adds hooks
pre_tool_call
Works on
linux
Needs Hermes
0.21.5 or newer
Version
0.2.0
In the catalog
2026-11-11

Known issues (from its catalog listing)

  • The Hermes target adapter uses version-sensitive runtime interfaces in separate evaluation workers; compatibility warnings and tested revisions are documented.
  • End-to-end workflows are verified with deterministic models. Live Command A target smoke and Muse scenario generation/preflight passed; a complete autonomous live campaign remains unverified.
Using the Hermes desktop app? See where plugins live
Hermes desktop Capabilities page, Plugins tab filtered to Official plugins built by Nous Research, each with an add button.
The official plugin catalog built into the app.Hermes desktop v0.22.5 · Capabilities > Plugins · Windows · captured Nov 1, 2026

Full setup & settings ↗Source code ↗

More in memory & notes

All memory & notes plugins

New Hermes Agent updates every day on X. Follow @HermesWatcher

← All plugins