skill-up
Alibaba's official eval harness — unit tests for your skills
Install
npx skills add https://github.com/alibaba/skill-up/tree/main/skills/skill-upperOverview
A Go CLI from Alibaba for evaluating and evolving Agent Skills (Apache-2.0): declarative suites in eval.yaml + cases/*.yaml run across Qoder CLI, Claude Code, Codex and Qwen Code with rule, script or agent judges, and emit Anthropic-compatible grading.json plus JUnit / HTML reports for CI. The recommended front door is the skill-upper Agent Skill shipped in the repo: ask it to «evaluate this skill» and the agent writes the cases, diagnoses failures, fixes the skill or its evals, and loops until green. Existing Anthropic evals.json import cleanly.
Key features
- Declarative evals: eval.yaml + cases/*.yaml instead of ad hoc run folders
- One suite across engines: Qoder CLI / Claude Code / Codex / Qwen Code
- Three judge types: rule_based, script, agent_judge
- Imports Anthropic evals.json; report formats align with the official eval loop
Use cases
- Proving a skill actually works, not vibes
- Regression runs after a skill revision, so updates can't silently break
- Comparing the same skill's quality across different agents
Pairs with skill-creator: one authors skills, this one proves them. Note: observation capture/review currently needs the Codex observer plugin; evaluation itself is engine-agnostic.
