Learning desk

MultiAgent EDU StackGather good sources. Teach what matters.
T3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay DraftsT3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts
← Labs

Learner can produce a machine-checkable two-round compounding eval plan with T1/T2 splits, retention metric, and a decision rule that refuses to label an overfit Phase-1 winner as compounds.

Lab specification (does not provision infrastructure).

#2

draftedcurriculum unit #2~73% time

published/labs/2026-07-16/agent-optimizer-compounding.md

Lab: Two-round compounding eval for an agent optimizer

Curriculum unit

  • Unit id: 2
  • Spec: curriculum/agent-optimizer-compounding.md
  • Deepens: Project (two-round compounding eval plan) and Exercises A–C

Observable end-state

Learner can produce a machine-checkable eval plan artifact that defines T1/T2 task splits, Phase-1 / transfer / Phase-2 scores, a retention metric, and a compounds / does-not / inconclusive decision rule, with fixture scores demonstrating that a Phase-1-only "winner" can fail transfer.

Time budget (75 min target)

StepKindMinutesNotes
Read lab brief + open worksheetprerequisite5No cloud
Partition 8 fixture tasks into T1/T2target10JSON edit
Fill Phase-1 / transfer / Phase-2 tabletarget15Use provided fake run logs
Define retention + lifelong average in plan.yamltarget10Schema validated
Write decision rule + apply to fixturestarget15Must flag GEPA-like overfitting case
Export out/plan.json + short rationaletarget15
Schema confusion / YAML typosnoise5Validator script gives line hints
Target share~73%

Environment shape

Local only:

  • fixtures/tasks.json: 8 named tasks with tags
  • fixtures/runs/: three optimizer result stubs (baseline, overfit-happy, regression-aware) with per-phase pass vectors
  • schema/plan.schema.json: required keys for the learner plan
  • tools/validate_plan.py: checks schema + that decision rule labels the overfit stub does_not or inconclusive (not compounds)
  • No GPUs, no API keys, no Terminal-Bench download required (fixtures stand in)

Validation (system state)

Pass when:

  1. out/plan.json validates against schema/plan.schema.json.
  2. out/plan.json includes non-empty t1_task_ids, t2_task_ids, retention_metric, and decision_rule.
  3. out/labels.json maps each fixture optimizer id to compounds | does_not | inconclusive.
  4. The overfit-happy fixture is not labeled compounds.
  5. out/rationale.md is ≤400 words and mentions transfer and retention explicitly.

Do not grade by command transcript.

Explicit non-goals

  • Not running real Terminal-Bench or paid agent rollouts
  • Not implementing GEPA/Meta-Harness/RELAI code
  • Not provisioning cloud runners