skill-eval-scaffolder

skill-eval-scaffolder208 wordsMITv1.0.0Repo-owned

skill-eval-scaffolder

Scaffold evals/evals.json manifests for repo skills with baseline cases. Use when adding behavioral eval coverage. NOT for live LLM eval execution.

Quick Start

Install:

npx skills add github:wyattowalsh/agents --skill skill-eval-scaffolder -y -g --agent antigravity --agent claude-code --agent codex --agent crush --agent cursor --agent gemini-cli --agent github-copilot --agent grok --agent opencode

Use: /skill-eval-scaffolder <skill-name> [--apply]

Works with Claude Code, Gemini CLI, OpenCode, and other agentskills.io-compatible agents.

Source & provenance

This page is generated from the repository file skills/skill-eval-scaffolder/SKILL.md by wagents docs generate.

What It Does

Create minimal, schema-valid eval manifests for repository skills.

Modes

$ARGUMENTS	Action
<name>	Preview scaffold for skills/<name>/evals/evals.json
<name> --apply	Write scaffold when file is missing or user approved overwrite
check <name>	Validate an existing eval manifest
Empty	Show usage gallery and manifest schema summary

Critical Rules

1. Never overwrite an existing eval manifest without explicit user approval.
2. Include at least one explicit-invocation case in new scaffolds.
3. Run validate_evals.py before declaring the scaffold complete.
4. Route live eval execution to skill-creator eval mode or maintainer CI workflows.
5. Keep prompts grounded in the skill dispatch table and NOT-for boundaries.

Public Metadata

Field	Value
Source Type	repo-owned
Display Source	github:wyattowalsh/agents
Source Kind	repo
Installability	portable command
Review State	reviewed
Target Agents	antigravity, claude-code, codex, crush, cursor, gemini-cli, github-copilot, grok, opencode

General

Field	Value
Name	skill-eval-scaffolder
License	MIT
Version	1.0.0
Author	wyattowalsh

Claude Code

Field	Value
Argument Hint	[skill-name] [--apply]

View Full SKILL.md

---
name: skill-eval-scaffolder
description: >-
  Scaffold evals/evals.json manifests for repo skills with baseline cases.
  Use when adding behavioral eval coverage. NOT for live LLM eval execution.
user-invocable: true
argument-hint: "<skill-name> [--apply]"
license: MIT
metadata:
  author: wyattowalsh
  version: "1.0.0"
---

# Skill Eval Scaffolder

Create minimal, schema-valid eval manifests for repository skills.

**Scope:** Eval scaffolding only. Do not run live behavioral evals unless the maintainer explicitly approves.

## Dispatch

| $ARGUMENTS | Action |
|------------|--------|
| `<name>` | Preview scaffold for `skills/<name>/evals/evals.json` |
| `<name> --apply` | Write scaffold when file is missing or user approved overwrite |
| `check <name>` | Validate an existing eval manifest |
| Empty | Show usage gallery and manifest schema summary |

## Workflow

1. Confirm `skills/<name>/SKILL.md` exists.
2. Run the bundled scaffold script:

```bash
uv run python skills/skill-eval-scaffolder/scripts/scaffold_evals.py <name>
```

3. Add explicit, implicit, and negative-control cases before claiming adequate coverage.
4. Validate with the bundled check script.

## Manifest Shape

Each `evals/evals.json` uses:

- `skill_name` — kebab-case directory name
- `evals` — non-empty list of cases with `id`, `prompt`, `expected_output`, optional `files`, `assertions`

## Validation Contract

```bash
uv run python skills/skill-eval-scaffolder/scripts/check.py
```

## Critical Rules

1. Never overwrite an existing eval manifest without explicit user approval.
2. Include at least one explicit-invocation case in new scaffolds.
3. Run `validate_evals.py` before declaring the scaffold complete.
4. Route live eval execution to skill-creator `eval` mode or maintainer CI workflows.
5. Keep prompts grounded in the skill dispatch table and NOT-for boundaries.

Download from GitHub

Resources

Skill CatalogBrowse custom and external skills.

CLI ReferenceInstall and manage skills.

agentskills.ioThe open ecosystem for cross-agent skills.

---

View source on GitHub