01
Model access & selection
Choose model access, model and version, routing and configuration to fit the task. Record those choices alongside outcomes and test alternatives; this does not require operating model serving infrastructure.
Agentic engineering
11factor
Build. Run. Observe. Improve.
11factor is my framework for engineering with AI agents and building reliable agent-based systems. Build, Run, Observe and Improve connect the task to its execution, evidence and next change. Human control, permissions and version lineage apply throughout.
These are engineering recommendations, not runtime guarantees. The learning mechanisms below describe a research direction; I have not demonstrated the full end-to-end learning loop.
A cycle of work and evidence
Build
02
Context assembly
Select and assemble the active context for each attempt. Version policies for choosing sources, truncating and refreshing information, and test whether the selected context helps the task rather than assuming more is better.
03
Memory & retention
Define what persists between attempts, with provenance, scope and retention rules. Make stored memory correctable and deletable; distinguish observations from inferred lessons and evaluate retrieval before expanding reuse.
04
Tools & data access
Give tools and data access protocol-agnostic contracts with explicit inputs, outputs, permissions, failure modes and side effects. Test these boundaries. MCP is one example, not mandatory; the same controls apply through other protocols.
Run
05
Execution state & orchestration
Preserve durable execution state, checkpoints and explicit handoffs. Account for repeated side effects when retrying or resuming work, and make completion and stop conditions inspectable across workers and interruptions.
07
Operational limits
Bound time, cost, concurrency and retries for each task and learning attempt. Measure quality alongside resource use. Reliability is a whole-system outcome, not a property guaranteed by this factor or by a larger budget.
Observe
08
Evaluation & observability
Connect traces to outcomes, versions and evaluation criteria. Traces explain what happened; outcome judgement asks whether the work met its goals. Use independent checks and keep uncertainty visible rather than accepting an agent’s self-score.
Improve
09
Self-learning & recursive improvement
Agent self-learning turns experience into candidate lessons and proposed changes. Recursive improvement proposes changes to the learning process itself. Both require independent comparisons, authorised promotion and later outcome evidence, without guaranteed gains.
Two nested mechanisms inside Improve: a framework and research direction, not implemented software. Reflection and a self-score are not independent evidence. Model weight training is not required. Permissions remain independently governed. There is no guarantee of continuous gains.
Agent self-learning
- 01
Experience
Select relevant outcome evidence, traces and human corrections from bounded real work.
- 02
Candidate lesson
Extract a candidate lesson with scope and uncertainty; it is not automatically trusted knowledge.
- 03
Proposed change
Propose a skill, context selection, memory policy, tool use, workflow or model behaviour change, with a falsifiable expected benefit.
- 04
Independent test
Use independent evidence to compare the candidate with the baseline on criteria fixed for the comparison.
- 05
Authorised promotion
Only an authorised decision may promote a gate-passing candidate to a bounded canary; monitor and rollback when criteria fail.
- 06
Later outcomes
Feed later outcomes back into learning, including evidence that challenges an earlier lesson.
Recursive improvement
Agents may propose changes to how they select experience, extract lessons or generate candidate updates. Compare the revised learning process against the previous learning process on held-out tasks, with criteria fixed for the comparison. Merely repeating reflection is iteration, not evidence of improved learning machinery. A revised process must pass the same promotion gate; later outcomes can challenge both its lessons and the process that produced them.
Across the loop
These controls apply throughout the cycle, including both learning mechanisms.
06
Human control
Provide human approval, intervention and takeover throughout Build, Run, Observe and Improve. Give people the context to decide, and capture corrections as scoped evidence rather than unquestionable rules for every future task.
10
Permissions & safety
Keep authority externally enforced and separate from optimisation. Permissions, approval requirements and safety boundaries must constrain every stage; an agent cannot grant itself access or weaken controls to make a candidate pass.
11
Versioning, audit & rollback
Preserve version lineage from evidence to lesson, candidate, tests and promotion decision. Record enough context to investigate results and maintain a tested rollback path. Auditability does not promise exact stochastic replay.
Proposed maturity sequence
A proposal for discussing how much change a system may make, not a claim that I operate any of these levels. More autonomy is not necessarily the right goal for every task.
- L0
Observe
Capture work and outcomes without changing behaviour.
- L1
Propose
Suggest evidence-linked changes for human review.
- L2
Auto-test
Run candidate tests automatically; keep promotion a separate decision.
- L3
Automated canary
Trial gate-passing changes within approved limits, with monitoring and rollback.
- L4
Bounded autonomous optimisation
Allow changes only within independently governed permissions and defined promotion criteria.
Open questions
- Which operational limits best expose trade-offs between task quality, time and cost?
- When does a memory become a reusable skill?
- How should we detect regressions when tasks and their context keep changing?