agent-regression-evals
Use when an agent, prompt, model or tool change needs proof it did not make the agent worse — setting up a regression eval suite, deciding whether a pass-rate drop is real or noise, grading tool-call trajectories, finding flaky eval cases, or wiring an eval gate into CI that blocks the merge. Runs repeated trials, reports pass@k and pass^k, compares candidate to baseline with paired statistics, and refuses to pass a dropped case, a critical-case regression, or a forbidden tool call.
Overview
Use when an agent, prompt, model or tool change needs proof it did not make the agent worse — setting up a regression eval suite, deciding whether a pass-rate drop is real or noise, grading tool-call trajectories, finding flaky eval cases, or wiring an eval gate into CI that blocks the merge. Runs repeated trials, reports pass@k and pass^k, compares candidate to baseline with paired statistics, and refuses to pass a dropped case, a critical-case regression, or a forbidden tool call.
Source: https://github.com/alirezarezvani/claude-skills · MIT · © 2025 Alireza Rezvani
System prompt
Preview redactedThe instruction block that defines the agent's role, rules and output format.
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████User prompt template
Fill in the variables and send it to run the skill: {{variables}}
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████
████████████████████████████████████████████████████████████████Variables
| Name | Required | Description | Example |
|---|
Full skill configuration
Redacted previewThe standard .aiskill file in JSON and YAML.
{
"format": "aiskill/v1",
"skill": {
"name": "agent-regression-evals",
"slug": "agent-regression-evals",
"version": "1.0.0",
"description": "Use when an agent, prompt, model or tool change needs proof it did not make the agent worse — setting up a regression eval suite, deciding whether a pass-rate drop is real or noise, grading tool-call trajectories, finding flaky eval cases, or wiring an eval gate into CI that blocks the merge. Runs repeated trials, reports pass@k and pass^k, compares candidate to baseline with paired statistics, and refuses to pass a dropped case, a critical-case regression, or a forbidden tool call.",
"category": "Development",
"tags": [
"ai-agents",
"agents"
],
"platforms": [
"chatgpt",
"claude-code",
"cursor"
],
"tested_environment": "Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and compatible agents",
"license": "MIT"
},
"prompts": {
"system": "████████████████████████████████████████████████████████████████████████████████████████████████",
"user_template": "███████████████████████████████████████████████████████████████████████████████"
},
"variables": [],
"install": {
"steps": [
{
"title": "Download & unzip",
"detail": "Download the .zip package from SkillVault and unzip it. The agent-regression-evals/ folder contains SKILL.md plus any companion scripts, references and assets."
},
{
"title": "Copy into your agent skills directory",
"detail": "Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code), .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or reload the session."
},
{
"title": "Use it",
"detail": "Describe the task in natural language; the agent loads the skill automatically when the request matches the skill description."
}
]
},
"usage": {
"steps": []
},
"metadata": {
"author": "SkillVault",
"source": "https://skillvault.top/skills/agent-regression-evals",
"exported_at": "2026-11-03T06:36:03.680Z"
}
}format: aiskill/v1
skill:
name: agent-regression-evals
slug: agent-regression-evals
version: 1.0.0
description: >-
Use when an agent, prompt, model or tool change needs proof it did not make the agent worse —
setting up a regression eval suite, deciding whether a pass-rate drop is real or noise, grading
tool-call trajectories, finding flaky eval cases, or wiring an eval gate into CI that blocks the
merge. Runs repeated trials, reports pass@k and pass^k, compares candidate to baseline with
paired statistics, and refuses to pass a dropped case, a critical-case regression, or a
forbidden tool call.
category: Development
tags:
- ai-agents
- agents
platforms:
- chatgpt
- claude-code
- cursor
tested_environment: >-
Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and
compatible agents
license: MIT
prompts:
system: ████████████████████████████████████████████████████████████████████████████████████████████████
user_template: ███████████████████████████████████████████████████████████████████████████████
variables: []
install:
steps:
- title: Download & unzip
detail: >-
Download the .zip package from SkillVault and unzip it. The agent-regression-evals/ folder
contains SKILL.md plus any companion scripts, references and assets.
- title: Copy into your agent skills directory
detail: >-
Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code),
.codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or
reload the session.
- title: Use it
detail: >-
Describe the task in natural language; the agent loads the skill automatically when the
request matches the skill description.
usage:
steps: []
metadata:
author: SkillVault
source: https://skillvault.top/skills/agent-regression-evals
exported_at: "2026-11-03T06:36:03.680Z"
Installation guide
- 1
Download & unzip
Download the .zip package from SkillVault and unzip it. The agent-regression-evals/ folder contains SKILL.md plus any companion scripts, references and assets.
- 2
Copy into your agent skills directory
Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code), .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or reload the session.
- 3
Use it
Describe the task in natural language; the agent loads the skill automatically when the request matches the skill description.
How to use
Platform compatibility
ChatGPT GPTs
OpenAI
Paste the system prompt into the GPT "Instructions" field and keep the user template as your starter message.
Official docsClaude Code
Anthropic
Save the exported .aiskill JSON as a skill file in your project or paste the system prompt into CLAUDE.md / a slash-command markdown file.
Official docsCursor
Anysphere
Copy the system prompt into .cursor/rules/*.mdc (or Project Rules in Settings) and reference the user template in your agent prompts.
Official docsRelated skills
agent-designer
Use when the user asks to design a multi-agent system, pick an orchestration pattern (supervisor/swarm/pipeline), generate tool schemas for agents, or evaluate agent execution logs for cost, latency, and failure bottlenecks. Examples: 'design an agent architecture for research automation', 'generate Anthropic tool schemas from these tool descriptions', 'analyze these agent run logs for bottlenecks'. NOT for Claude Code workflow files (use workflow-builder) or single-agent prompt design (use agent-workflow-designer).
agent-harness
Turn any domain folder of skills into a bounded agentic loop: compile a goal into a verifiable task plan, execute tasks with the domain's own tools, verify every task with machine-run checks, retry with caps, escalate to a human when budgets exhaust, and refuse to close until everything is verified or explicitly waived. Use when you want an agent or subagent to pick up a goal and drive it to a verified close across one of this repo's 18 domains ('run this goal through the engineering harness', 'set up an agentic loop for marketing work', 'make the finance domain self-verifying'). NOT for authoring Claude Code Workflow-tool .js scripts (workflow-builder), N-agent tournaments on one task (agenthub), single-file metric optimization (autoresearch-agent), or discovering published loop recipes (loop-library).
agent-memory
Use when a project's CLAUDE.md has grown past what anyone reads and you want the agent to learn durable facts from its own sessions instead — or when asking why the agent keeps re-learning the same correction, why a remembered rule is wrong, or where a memory line came from. Implements a four-tier store (L0 transcripts / L1 candidates / L2 project context / L3 stable persona) where promotion is earned by recurrence across sessions and days, never by one confident statement, and nothing reaches a committed file without a human adopting it.