EXP-012 · Measurement and Experimentation

Use explicit continue/revise/revert/retire decisions

• Baseline
• IF
• AND
• THEN
• ELSE
• XOR

The decision

• IF
  • a review point is reached
• THEN
  • choose exactly one decision based on primary metric, guardrails, cost, and confidence
• ELSE
  • continue observation only under a predefined uncertainty rule

Primary metric

decision closure rate

Guardrail

zombie experiments

Reason

Makes the decision explicit, measurable, and reversible where possible.

LAKA levels

• Baseline

Operators

• IF
• AND
• THEN
• ELSE
• XOR

Governs gates

• measurement

Source

04-IF-ELSE-AND-OR-RULEBOOK.md — EXP-012 — Use explicit continue/revise/revert/retire decisions

The other eleven rules in Measurement and Experimentation

• EXP-001 Require a mechanism hypothesis IF an intervention has no stated reason it should affect an outcome Baseline
• EXP-002 Capture a baseline IF a change is about to be implemented Baseline
• EXP-003 Use a change ledger IF any SEO-relevant production change occurs Structural
• EXP-004 Choose one primary metric IF an experiment has many possible success metrics Baseline
• EXP-005 Set guardrails before launch IF a change can improve the primary metric while harming quality, accessibility, revenue, or trust Baseline
• EXP-006 Use controls when feasible IF comparable untreated pages, clusters, regions, or time periods exist Baseline
• EXP-007 Account for seasonality and external events IF demand, algorithms, campaigns, news, inventory, or holidays can affect the outcome Baseline
• EXP-008 Represent uncertainty IF sample size is small, variance is high, or exposure is incomplete Baseline
• EXP-009 Allow processing and learning time IF a change has not had enough crawl, index, ranking, or behavioral exposure Baseline
• EXP-010 Distinguish single-variable and package tests IF several material variables change together Minor
• EXP-011 Escalate by diagnosed mechanism IF a lower-level LAKA intervention fails Minor