Every premium and members-only resource for $9.99/month, as a no-commitment subscription. See the plans

PalateKit

Sign in
Menu

Request a resource

Describe the resource you are missing in one sentence. We will email you when it is published.

0 / 280

A page, a video or an image online.

Unlock PalateKit

Unlock everything

Plan

Paying means accepting our terms of sale.

Secure payment by Stripe. Receipts and invoices sent by email.

Welcome back

Sign in to install, download or save this resource. A free account is enough for free resources.

Fine-tuning evaluation subagent v1.0.0

Use this resource to build a baseline evaluation and approve a fine-tuned model

New

  • 436 downloads
  • 122 likes
  • 54 saves

Free

Install with the CLI

In a terminal, from the folder you want:

npx --yes palatekit@latest install llm-eval-gate-subagent

First time: connect the CLI with

npx --yes palatekit@latest login

This subagent builds golden sets and graders, calibrates LLM judges, sets baseline scores and issues promotion verdicts for checkpoints. It stays independent from training and writes no training configuration.

Mostly picked by

Updated November 2, 2026

Card for “Fine-tuning evaluation subagent”

Details

Works in
Install guides
Generates with
Your agent's model
Licence
MIT

Similar resources

View all

What you get

  • The subagent file (in English): evaluation and gating method
  • An eval set, graders and a promotion report

Good fit for

  • A team that must prove a fine-tuned model performs better
  • A project with no reliable measure of answer quality

Not for

    Tags

    • evaluation
    • fine-tuning
    • llm judge
    • quality control

    About

    You say

    “Build the baseline eval set before any training starts.”

    Card for “Fine-tuning evaluation subagent”

    Subagent published by wshobson under the MIT license, delivered in its original version.

    Show more

    How to use it

    1. Copy the file into .claude/agents/ (project) or ~/.claude/agents/ (all your projects)
    2. Ask your agent to build a baseline evaluation and approve a fine-tuned model, or call the subagent by name
    3. Review its work

    Content preview

    Files

    • LICENSE1.1 KB
    • llm-eval-gate-subagent.md9.8 KB
    • NOTICE-PALATEKIT.md0.5 KB

    Origin and licence

    Listed unmodified

    Author
    wshobson
    Source
    github.com/wshobson/agents/blob/0874a27b17a22e2b7c2e2bcd725768e0803cad06/plugins/llm-finetuning/agents/llm-finetuning-eval-engineer.md
    Licence
    MIT

    Notice

    « llm-finetuning-eval-engineer » (wshobson, MIT) livré dans sa version d'origine : fiche en français et en anglais par PalateKit.

    User reviews

    –

    No ratings yet

    Loading reviews…

    Delete your review?

    Your rating and comment will be removed from the page.