ADR-0012: Skill Distillation & the Project Learning Flywheel

Amended by ADR-0015: the user/team/project tiers are concretely the three-level .stet topology ($USER/.stet, $ORG/.stet, <repo>/.stet/); skills are one of three primitives resolved across it.

Framed by ADR-0019: distillation is the outer loop of Stet's compounding memory — slow, batched, across many trajectories, producing durable house-style skills. Its fast, in-session counterpart is the inner loop (ADR-0020), whose curated per-session working memory ("style sheet") is high-signal distillation fuel that graduates into skills through this same PR ceremony — never bypassing it. The one-line thesis this loop implements: every other tool's agent starts cold on every task; Stet's agents remember every review your team ever gave.

Status

Proposed / exploring. Captured during ideation on 2026-07-04; skill tiering was decided in review the same day (marked DECIDED below). Depends on ADR-0011 (trajectories). Revisit for promotion to accepted (+ a spec) after the core loop (SPEC-0001..0003) is standing.

Context and Problem Statement

Once trajectories are first-class, versioned artifacts (ADR-0011), the system accumulates a record of how work actually got done on a project — plus the human feedback (ADR-0006), agent self-reflection, and outcomes (merged vs. reverted vs. abandoned). Over time, recurring patterns emerge: the same corrections, the same conventions, the same "here's how this codebase wants it done."

The idea — skill distillation (per Tomasz Tunguz, https://www.linkedin.com/pulse/skill-distillation-tomasz-tunguz-rioic/): review that accumulated experience and distill it into reusable, named skills that are committed back into the project repository and injected into future dispatches, so the project's agents measurably improve over time. Institutional knowledge becomes a versioned artifact instead of tribal memory.

Framing — skills are software artifacts

Like trajectories (ADR-0011), a skill is part of the project's software: an open, versioned, documented file that is committed, reviewed as a PR, diffed, deprecated, and portable across harnesses. It is not hidden model state or a vendor blob.

Sketch — the flywheel

human markup + trajectory + outcome
    → distillation (a meta-dispatch reviewing N trajectories + feedback)
    → proposed skill (a PR, reviewed through the normal stet loop)
    → skill committed to the appropriate tier
    → injected into the next dispatch's sandbox
    → better trajectory → …

• Distillation is a workflow (ADR-0005) run by a harness (ADR-0004): a periodic or triggered meta-dispatch that reads a window of trajectories, feedback, and outcomes and proposes one or more skills.
• Skills are proposed as PRs and reviewed exactly like code (ADR-0002/0006) — the system improves itself using its own core mechanic. Each skill carries provenance back to the trajectories that spawned it (ADR-0011).
• Injection: a harness-agnostic seam projects the relevant skills into whatever each harness expects (AGENTS.md, extensions, MCP, system-prompt context).

Decisions

DECIDED — Three skill tiers: user, team, project

Skills are routed to one of three scopes, most-specific-wins at injection time:

Tier	Scope	Where it lives
project	specific to this codebase	.stet/skills/ committed in the repo
team	shared across a team's repos	a centralized team skill repo
user	the individual's cross-project skills	a centralized personal skill repo

Stet settings let users register their centralized skill repos (personal and team) so team/user skills are pulled in alongside the project's own. Distillation classifies each proposed skill's tier; project-scoped skills land in the repo, broader ones are proposed to the configured team/user repos. (This mirrors how a maintainer already keeps a personal skills library separate from per-project setup.)

Why it fits

• Good, because it turns Stet from a review UI into a system that makes a project's agents better over time — a genuine, compounding moat.
• Good, because it reuses the dispatch/workflow/review machinery; distillation is "just another workflow" and skills are "just another reviewed change."
• Good, because skills-as-versioned-artifacts keep improvement auditable and reversible, across three well-defined tiers.

Open questions

• Quality & guardrails. Distilled skills can encode bad patterns or overfit. Mitigations: PR review, provenance links, and an efficacy measure (A/B dispatches with vs. without a skill) to keep or retire it.
• Drift & staleness. Detecting a skill that no longer matches the codebase; deprecation policy.
• Format & portability. The open skill schema and its mapping to each harness's native skill/extension mechanism.
• Tier conflicts. Precedence and merge behavior when user/team/project skills overlap or contradict (default: most-specific wins — needs detail).
• Trigger cadence & cost. When distillation runs (every N merged PRs? scheduled? on-demand?).
• Relationship to design artifacts (ADRs/specs, SDD graph) — skills are another node in the provenance lineage; how much to unify.

Consequences (tentative)

• Adds a Skill artifact + open schema, a .stet/skills/ convention (project tier) plus configured centralized repos (team/user tiers), a distillation workflow, and a harness-agnostic skill-injection seam.
• Adds Stet settings for registering centralized skill repos.
• Depends entirely on ADR-0011 (no trajectories → nothing to distill).
• Introduces an efficacy/measurement concern (does a skill actually help?).

More Information

Paired with ADR-0011 (trajectories). Term/source: skill distillation, Tomasz Tunguz (https://www.linkedin.com/pulse/skill-distillation-tomasz-tunguz-rioic/). Shared theme: the project learns over time.