Continue
IF evidence is positive
AND guardrails pass
AND additional exposure is needed
THEN continue.Learning
Hypotheses and decisions
A change becomes an experiment when it has a mechanism, a primary metric, guardrails and a decision rule agreed before launch rather than after seeing the data.
The hypothesis grammar
IF [INTERVENTION]
is applied to [OBJECT]
under [CONDITIONS],
THEN [PRIMARY METRIC]
will change in [DIRECTION]
because [MECHANISM],
while [GUARDRAILS]
remain within [THRESHOLDS].IF the title proposition for the Calgary web design cost guide
is rewritten to state the year, location, and estimator benefit,
THEN position-adjusted organic CTR will increase
because relevance and expected utility become clearer,
while qualified assessment rate does not decline by more than 10%.IF [intervention]
is applied to [object]
under [conditions],
THEN [primary metric]
changes [direction]
because [mechanism],
while [guardrails] pass.The experiment card
Experiment ID:
Object:
Conditions:
Action:
Tools:
Resources:
Expected outcome:
Feedback source:
Constraints:
Value:
Failure mode:
LAKA level:
Magnitude:
Rate:
Direction:
Scope:
Depth:
Duration:
Frequency:
Acceleration:
Variability:
Detectability:
Reversibility:
Propagation:
Amplification:
Accumulation:
Primary metric:
Baseline:
Treatment scope:
Comparison:
Minimum exposure:
Decision rule:
Guardrails:
Rollback:
Owner:Decision outcomes
Seven outcomes, chosen in advance. "Keep going and see" is not one of them.
Expand
IF positive effect is credible
AND mechanism is transferable
AND capacity permits
THEN expand to a defined cohort.Revise
IF mechanism appears valid
BUT implementation is weak or guardrails fail mildly
THEN revise.Reverse
IF primary outcome declines materially
OR a critical guardrail fails
AND rollback is feasible
THEN reverse.Escalate
IF the diagnosed failure is deeper than the current LAKA level
AND expected value still exceeds cost
THEN escalate.Retire
IF remaining value is low
OR maintenance/risk is too high
OR the asset has no distinct task
THEN consolidate, redirect, noindex, archive, or remove appropriately.Experiment types by LAKA level
Minor element test
Examples:
- title proposition;
- snippet description;
- introductory answer;
- CTA wording;
- anchor text;
- small media change.
Best for high exposure and reversible mechanisms.
Major asset test
Examples:
- full rewrite;
- new comparison;
- original data;
- tool or video;
- new evidence architecture.
Evaluate as an intervention package unless components can be separated.
Structural test
Examples:
- page merge;
- template change;
- internal graph redesign;
- facet control;
- CRM linkage;
- site architecture.
Use cohort or time-series analysis and monitor propagation.
Paradigm test
Examples:
- calculator instead of article;
- benchmark instead of generic guide;
- verified directory instead of location-page set;
- interactive diagnostic instead of lead form;
- data API or agent action instead of static copy.
Primary value may include new demand, citations, brand searches, product adoption, and conversion.