Sglang (opens in a new tab)
GitHub tapFast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Pow
Community skill. Read the source first. Listed does not mean tested with Hermes.
hermes skills install synthetic-sciences/openscience/backend/cli/skills/ml-inference/sglang