FreeSkills
Knowledge wiki setup
- 418418 downloads
- 129129 likes
- 161161 saves
Works with
- Claude Code
- Codex
- Cursor
- and 3 more
Evaluating code models (HumanEval, MBPP, MultiPL-E) v1.0.0
Use this resource to evaluate and compare code generation models
New
- 569 downloads
- 176 likes
- 241 saves
Free
Install with the CLI
In a terminal, from the folder you want:
npx --yes palatekit@latest install code-model-benchmark-evaluationWith no option, the skill goes to ~/.agents/skills/; add --agent claude-code (or codex, cursor…) to target an agent.
First time: connect the CLI with
npx --yes palatekit@latest loginThe agent prepares evaluations with the BigCode Evaluation Harness: HumanEval, MBPP or MultiPL-E, with or without execution, and pass@k computation. You get the commands, sampling settings and results to compare.
Mostly picked by
Updated November 2, 2026
Previews
Details
- Install guides
- Generates with
- Your agent's model
- Licence
- MIT
Similar resources
View all- Free
FreeSkills
Wiki health check
- 552552 downloads
- 202202 likes
- 127127 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
Ask your wiki
- 374374 downloads
- 131131 likes
- 214214 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
Wiki source ingestion
- 315315 downloads
- 230230 likes
- 246246 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
Wiki auditor agent
- 556556 downloads
- 205205 likes
- 192192 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
UX research assistant
- 598598 downloads
- 142142 likes
- 236236 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
Wiki ingestion agent
- 437437 downloads
- 162162 likes
- 191191 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
FreeSkills
Wiki librarian agent
- 304304 downloads
- 120120 likes
- 217217 saves
Works with- Claude Code
- Codex
- Cursor
- and 3 more
What you get
- SKILL.md: quick start and standard and multi-language evaluation workflows
- references/benchmarks.md: list of benchmarks
- references/custom-tasks.md and references/issues.md: custom tasks and known issues
Good fit for
- You must publish a model's coding scores
- You want to test a code model across several languages
Not for
Tags
- benchmark
- code
- humaneval
- pass@k
About
You say
“Evaluate StarCoder2 7B on HumanEval with twenty samples per problem and compute pass@1 and pass@10.”
Skill published by Orchestra Research under the MIT license, delivered in its original version.
Show moreShow less
How to use it
- Install the skill in your agent's skills folder
- Ask your agent to evaluate and compare code generation models
- Review and adjust the result
Content preview
Files
- LICENSE1.1 KB
- NOTICE-PALATEKIT.md0.6 KB
- references/benchmarks.md9.3 KB
- references/custom-tasks.md10.5 KB
- references/issues.md8.3 KB
- SKILL.md11.8 KB
Origin and licence
Listed unmodified
- Author
- Orchestra Research
- Source
- github.com/davila7/claude-code-templates/tree/f57964ebda91cdfb31755661ecea9395f30020f1/cli-tool/components/skills/ai-research/evaluation-bigcode-evaluation-harness
- Licence
- MIT
Notice
« evaluation-bigcode-evaluation-harness » (Orchestra Research, MIT) livré dans sa version d'origine : fiche en français et en anglais par PalateKit.
User reviews
–
No ratings yet
Loading reviews…