Learning desk

MultiAgent EDU StackGather good sources. Teach what matters.
T5TauricResearch/TradingAgentsT5A Man Who Invented Modern AI (Before Everyone Else) – Jürgen Schmidhuber [video]T5GPT-4 finished training four years ago todayT5AI Settles a 25 Year-Old Problem We Left BehindT5What it was like working on LLMs and security at Meta (2022-2026)T5Ask HN: How do you go from writing code to deploying with agents?T5What Happened: OpenAI and HuggingFaceT5Apple says Mac users in China can connect to Alibaba's Qwen AI serviceT5Show HN: Try Benzi – A coding harness/agent beating Claude Code itself on SonnetT5The AI Apocalypse Is HereT3Auto mode is now the default in Claude Code for Pro, Max, and Team plansT5Show HN: Tura – Build agent that uses 80% less token and delivers better resultsT5TauricResearch/TradingAgentsT5A Man Who Invented Modern AI (Before Everyone Else) – Jürgen Schmidhuber [video]T5GPT-4 finished training four years ago todayT5AI Settles a 25 Year-Old Problem We Left BehindT5What it was like working on LLMs and security at Meta (2022-2026)T5Ask HN: How do you go from writing code to deploying with agents?T5What Happened: OpenAI and HuggingFaceT5Apple says Mac users in China can connect to Alibaba's Qwen AI serviceT5Show HN: Try Benzi – A coding harness/agent beating Claude Code itself on SonnetT5The AI Apocalypse Is HereT3Auto mode is now the default in Claude Code for Pro, Max, and Team plansT5Show HN: Tura – Build agent that uses 80% less token and delivers better results
← Labs

Learner can produce a machine-checkable two-round compounding eval plan with T1/T2 splits, retention metric, and a decision rule that refuses to label an overfit Phase-1 winner as compounds.

Lab specification (does not provision infrastructure).

#2

draftedcurriculum unit #2~73% time

published/labs/2026-07-16/agent-optimizer-compounding.md

Lab: Two-round compounding eval for an agent optimizer

Curriculum unit

  • Unit id: 2
  • Spec: curriculum/agent-optimizer-compounding.md
  • Deepens: Project (two-round compounding eval plan) and Exercises A–C

Observable end-state

Learner can produce a machine-checkable eval plan artifact that defines T1/T2 task splits, Phase-1 / transfer / Phase-2 scores, a retention metric, and a compounds / does-not / inconclusive decision rule, with fixture scores demonstrating that a Phase-1-only "winner" can fail transfer.

Time budget (75 min target)

StepKindMinutesNotes
Read lab brief + open worksheetprerequisite5No cloud
Partition 8 fixture tasks into T1/T2target10JSON edit
Fill Phase-1 / transfer / Phase-2 tabletarget15Use provided fake run logs
Define retention + lifelong average in plan.yamltarget10Schema validated
Write decision rule + apply to fixturestarget15Must flag GEPA-like overfitting case
Export out/plan.json + short rationaletarget15
Schema confusion / YAML typosnoise5Validator script gives line hints
Target share~73%

Environment shape

Local only:

  • fixtures/tasks.json: 8 named tasks with tags
  • fixtures/runs/: three optimizer result stubs (baseline, overfit-happy, regression-aware) with per-phase pass vectors
  • schema/plan.schema.json: required keys for the learner plan
  • tools/validate_plan.py: checks schema + that decision rule labels the overfit stub does_not or inconclusive (not compounds)
  • No GPUs, no API keys, no Terminal-Bench download required (fixtures stand in)

Validation (system state)

Pass when:

  1. out/plan.json validates against schema/plan.schema.json.
  2. out/plan.json includes non-empty t1_task_ids, t2_task_ids, retention_metric, and decision_rule.
  3. out/labels.json maps each fixture optimizer id to compounds | does_not | inconclusive.
  4. The overfit-happy fixture is not labeled compounds.
  5. out/rationale.md is ≤400 words and mentions transfer and retention explicitly.

Do not grade by command transcript.

Explicit non-goals

  • Not running real Terminal-Bench or paid agent rollouts
  • Not implementing GEPA/Meta-Harness/RELAI code
  • Not provisioning cloud runners