{"number":"05","slug":"measurement-and-experiment-system","filename":"05-MEASUREMENT-AND-EXPERIMENT-SYSTEM.md","title":"Measurement and Experiment System","bytes":12913,"section_count":49,"purpose":"Metrics, formulas, experiments, and dashboards","outline":[{"heading":"05 — Measurement and Experiment System","anchor":"05-measurement-and-experiment-system","depth":1},{"heading":"1. Measurement philosophy","anchor":"1-measurement-philosophy","depth":2},{"heading":"2. Outcome hierarchy","anchor":"2-outcome-hierarchy","depth":2},{"heading":"3. Primary formulas","anchor":"3-primary-formulas","depth":2},{"heading":"4. Data grain","anchor":"4-data-grain","depth":2},{"heading":"5. Event dictionary","anchor":"5-event-dictionary","depth":2},{"heading":"6. Baseline protocol","anchor":"6-baseline-protocol","depth":2},{"heading":"7. Hypothesis grammar","anchor":"7-hypothesis-grammar","depth":2},{"heading":"8. LAKA experiment card","anchor":"8-laka-experiment-card","depth":2},{"heading":"9. Decision outcomes","anchor":"9-decision-outcomes","depth":2},{"heading":"10. Experiment types","anchor":"10-experiment-types","depth":2},{"heading":"11. Comparison designs","anchor":"11-comparison-designs","depth":2},{"heading":"12. Observation windows","anchor":"12-observation-windows","depth":2},{"heading":"13. Dashboard architecture","anchor":"13-dashboard-architecture","depth":2},{"heading":"14. Metric anti-patterns","anchor":"14-metric-anti-patterns","depth":2},{"heading":"15. Monthly learning review","anchor":"15-monthly-learning-review","depth":2}],"sections":[{"heading":"05 — Measurement and Experiment System","depth":1,"anchor":"05-measurement-and-experiment-system","path":["05 — Measurement and Experiment System"],"prose":[],"bullets":[],"blocks":[],"tables":[]},{"heading":"1. Measurement philosophy","depth":2,"anchor":"1-measurement-philosophy","path":["05 — Measurement and Experiment System","1. Measurement philosophy"],"prose":["SEO is a partially observed causal system. Search engines do not expose every internal variable, so the measurement model separates:","Never infer success from a proxy without checking the downstream outcome it is supposed to influence."],"bullets":[],"blocks":[{"lang":"text","code":"BUSINESS OUTCOMES\nDIAGNOSTIC OUTCOMES\nSYSTEM STATES\nINTERVENTIONS\nCONTEXT\nUNCERTAINTY"},{"lang":"text","code":"ranking ↑\ndoes not automatically imply\nqualified value ↑"}],"tables":[]},{"heading":"2. Outcome hierarchy","depth":2,"anchor":"2-outcome-hierarchy","path":["05 — Measurement and Experiment System","2. Outcome hierarchy"],"prose":[],"bullets":[],"blocks":[],"tables":[]},{"heading":"Level 1 — Business outcomes","depth":3,"anchor":"level-1-business-outcomes","path":["05 — Measurement and Experiment System","2. Outcome hierarchy","Level 1 — Business outcomes"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"qualified leads\nsales\nsubscription starts\nretained customers\nexpansion revenue\nsupport value\ncontribution margin\ncustomer lifetime value"}],"tables":[]},{"heading":"Level 2 — Journey outcomes","depth":3,"anchor":"level-2-journey-outcomes","path":["05 — Measurement and Experiment System","2. Outcome hierarchy","Level 2 — Journey outcomes"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"task completion\nnext-step action\nassessment completion\ndemo request\nquote request\ntrial start\nqualified form\nreturn visit\nrenewal action\nreferral/review"}],"tables":[]},{"heading":"Level 3 — Search outcomes","depth":3,"anchor":"level-3-search-outcomes","path":["05 — Measurement and Experiment System","2. Outcome hierarchy","Level 3 — Search outcomes"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"impressions\nsearch appearances\naverage position / rank distribution\nclicks\nCTR\nlanding sessions\nbranded demand\nnon-branded demand"}],"tables":[]},{"heading":"Level 4 — Asset and graph diagnostics","depth":3,"anchor":"level-4-asset-and-graph-diagnostics","path":["05 — Measurement and Experiment System","2. Outcome hierarchy","Level 4 — Asset and graph diagnostics"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"query-cluster coverage\ncanonical ownership\ninternal in-links\nclick depth\norphan state\nqualified referring domains\ncontent/evidence completeness\nfreshness"}],"tables":[]},{"heading":"Level 5 — Technical states","depth":3,"anchor":"level-5-technical-states","path":["05 — Measurement and Experiment System","2. Outcome hierarchy","Level 5 — Technical states"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"discoverable\ncrawlable\nrenderable\ncanonicalized\nindexable\nindexed\nretrievable\nvalid structured data\nfield performance"}],"tables":[]},{"heading":"3. Primary formulas","depth":2,"anchor":"3-primary-formulas","path":["05 — Measurement and Experiment System","3. Primary formulas"],"prose":[],"bullets":[],"blocks":[],"tables":[]},{"heading":"3.1 Actual organic contribution margin","depth":3,"anchor":"3-1-actual-organic-contribution-margin","path":["05 — Measurement and Experiment System","3. Primary formulas","3.1 Actual organic contribution margin"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"ACTUAL ORGANIC CONTRIBUTION\n=\nΣ(\n  attributed organic conversions\n  × contribution margin per conversion\n)\n− attributable SEO operating cost"}],"tables":[]},{"heading":"3.2 Incremental lift","depth":3,"anchor":"3-2-incremental-lift","path":["05 — Measurement and Experiment System","3. Primary formulas","3.2 Incremental lift"],"prose":["When a reasonable comparison group exists:","Apply separately to:","- impressions;","- clicks;","- qualified conversions;","- revenue;","- contribution margin;","- retention."],"bullets":["impressions;","clicks;","qualified conversions;","revenue;","contribution margin;","retention."],"blocks":[{"lang":"text","code":"INCREMENTAL LIFT\n=\n(Post_treatment − Pre_treatment)\n−\n(Post_control − Pre_control)"}],"tables":[]},{"heading":"3.3 Expected value","depth":3,"anchor":"3-3-expected-value","path":["05 — Measurement and Experiment System","3. Primary formulas","3.3 Expected value"],"prose":["Every estimated input must be labeled and later replaced with observed data."],"bullets":[],"blocks":[{"lang":"text","code":"EXPECTED VALUE\n=\nΣ(\n  demand estimate\n  × probability of eligibility\n  × probability of useful visibility\n  × expected CTR\n  × expected qualified conversion rate\n  × contribution margin\n)\n− expected cost"}],"tables":[]},{"heading":"3.4 Value-weighted visibility","depth":3,"anchor":"3-4-value-weighted-visibility","path":["05 — Measurement and Experiment System","3. Primary formulas","3.4 Value-weighted visibility"],"prose":["Visibility weight may be based on impression share, rank distribution, or a calibrated CTR curve."],"bullets":[],"blocks":[{"lang":"text","code":"VALUE-WEIGHTED VISIBILITY\n=\nΣ(\n  query opportunity value\n  × observed visibility weight\n)\n/\nΣ(query opportunity value)"}],"tables":[]},{"heading":"3.5 CTR residual","depth":3,"anchor":"3-5-ctr-residual","path":["05 — Measurement and Experiment System","3. Primary formulas","3.5 CTR residual"],"prose":["This is more useful than a universal “good CTR” threshold."],"bullets":[],"blocks":[{"lang":"text","code":"CTR RESIDUAL\n=\nObserved CTR\n− Expected CTR(\n    position,\n    device,\n    country,\n    brand class,\n    query class,\n    search appearance\n  )"}],"tables":[]},{"heading":"3.6 Qualified conversion value","depth":3,"anchor":"3-6-qualified-conversion-value","path":["05 — Measurement and Experiment System","3. Primary formulas","3.6 Qualified conversion value"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"QUALIFIED ORGANIC VALUE\n=\norganic value events\n× qualification probability\n× contribution margin"}],"tables":[]},{"heading":"3.7 Opportunity score","depth":3,"anchor":"3-7-opportunity-score","path":["05 — Measurement and Experiment System","3. Primary formulas","3.7 Opportunity score"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"OPPORTUNITY SCORE\n=\n(\n  demand confidence\n  × business fit\n  × task value\n  × information advantage\n  × attainable response gap\n  × conversion value\n  × evidence confidence\n)\n/\n(\n  effort\n  × risk\n  × maintenance burden\n  × time to learning\n)"}],"tables":[]},{"heading":"3.8 Eligible canonical coverage","depth":3,"anchor":"3-8-eligible-canonical-coverage","path":["05 — Measurement and Experiment System","3. Primary formulas","3.8 Eligible canonical coverage"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"ELIGIBLE CANONICAL COVERAGE\n=\npriority canonical URLs that are eligible and indexed\n/\nall priority canonical URLs intended for indexing"}],"tables":[]},{"heading":"3.9 Canonical agreement","depth":3,"anchor":"3-9-canonical-agreement","path":["05 — Measurement and Experiment System","3. Primary formulas","3.9 Canonical agreement"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"CANONICAL AGREEMENT\n=\nURLs where intended/declarative canonical agrees with observed selected canonical\n/\nevaluated duplicate or canonical URLs"}],"tables":[]},{"heading":"3.10 Cannibalization indicators","depth":3,"anchor":"3-10-cannibalization-indicators","path":["05 — Measurement and Experiment System","3. Primary formulas","3.10 Cannibalization indicators"],"prose":["High entropy is a diagnostic, not automatic failure. Confirm same-intent duplication, instability, divided links, or value loss."],"bullets":[],"blocks":[{"lang":"text","code":"DOMINANT URL SHARE\n=\nlargest URL impression share for cluster\n/\ntotal cluster impressions"},{"lang":"text","code":"URL ENTROPY\n=\n−Σ(p_url × ln(p_url))"}],"tables":[]},{"heading":"3.11 Internal graph diagnostics","depth":3,"anchor":"3-11-internal-graph-diagnostics","path":["05 — Measurement and Experiment System","3. Primary formulas","3.11 Internal graph diagnostics"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"PRIORITY ORPHAN RATE\n=\npriority pages with no useful crawlable internal in-link\n/\nall priority pages"},{"lang":"text","code":"WEIGHTED CLICK DEPTH\n=\nΣ(page business weight × page click depth)\n/\nΣ(page business weight)"}],"tables":[]},{"heading":"3.12 Authority diagnostics","depth":3,"anchor":"3-12-authority-diagnostics","path":["05 — Measurement and Experiment System","3. Primary formulas","3.12 Authority diagnostics"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"QUALIFIED LINK RATE\n=\nretained relevant editorial referring domains\n/\nall acquired referring domains"}],"tables":[]},{"heading":"4. Data grain","depth":2,"anchor":"4-data-grain","path":["05 — Measurement and Experiment System","4. Data grain"],"prose":["Store search observations at the smallest useful grain:","Attach:","Store technical observations separately:","Store changes:"],"bullets":[],"blocks":[{"lang":"text","code":"date\nsite_id\ncluster_id\nquery\ncanonical_url\ncountry\ndevice\nsearch_type\nsearch_appearance\nbrand_class\nchange_id"},{"lang":"text","code":"impressions\nclicks\nctr\naverage_position\norganic_sessions\ntask_completions\nnext_actions\nqualified_leads\nsales\nrevenue\ncontribution_margin"},{"lang":"text","code":"observation_time\nurl\nstatus_code\nrobots_allowed\nrender_success\ndeclared_canonical\nselected_canonical\nindex_state\nsitemap_state\ninternal_inlinks\nclick_depth\nfield_lcp\nfield_inp\nfield_cls\nstructured_data_state"},{"lang":"text","code":"change_id\nobject_type\nobject_id\nlaka_level\nhypothesis\nmechanism\nowner\nstart_time\nend_time\nrollback_state\nprimary_metric\nguardrails\ndecision_rule"}],"tables":[]},{"heading":"5. Event dictionary","depth":2,"anchor":"5-event-dictionary","path":["05 — Measurement and Experiment System","5. Event dictionary"],"prose":["A measurement plan must define every event.","Avoid meaningless event inflation. An event should correspond to a user state or business state."],"bullets":[],"blocks":[],"tables":[{"headers":["Event","Definition","Required properties"],"rows":[["`organic_landing`","Session begins on an organic search landing page","landing URL, cluster, query class"],["`task_start`","User initiates the primary page task","task ID"],["`task_complete`","User reaches a valid completion condition","task ID, method"],["`next_action`","User takes the intended adjacent step","action ID"],["`qualified_form`","Form meets qualification rules","offer, segment, qualification"],["`sales_accept`","Sales accepts the lead","CRM status"],["`closed_value`","Revenue or margin is confirmed","value, margin, offer"],["`retention_value`","Renewal, expansion, or support outcome","customer, event value"],["`evidence_interaction`","User opens data, sources, demo, comparison, or case proof","evidence ID"],["`internal_path`","User follows a meaningful graph edge","edge type, target"],["`calculator_complete`","Tool returns a valid result","inputs band, result band"],["`download_use`","User downloads or uses a template","asset ID"]]}]},{"heading":"6. Baseline protocol","depth":2,"anchor":"6-baseline-protocol","path":["05 — Measurement and Experiment System","6. Baseline protocol"],"prose":["Before changing a priority object:","If historical data is unavailable, create a prospective baseline and explicitly reduce causal confidence."],"bullets":[],"blocks":[{"lang":"text","code":"1. Verify data collection.\n2. Define treatment object and exposure.\n3. Record at least the available stable pre-period.\n4. Record query mix, device, country, brand class, and seasonality.\n5. Record technical eligibility.\n6. Record current page and graph state.\n7. Record current business value.\n8. Assign change ID.\n9. Predefine primary metric and guardrails.\n10. State minimum exposure and review rule."}],"tables":[]},{"heading":"7. Hypothesis grammar","depth":2,"anchor":"7-hypothesis-grammar","path":["05 — Measurement and Experiment System","7. Hypothesis grammar"],"prose":["Example:"],"bullets":[],"blocks":[{"lang":"text","code":"IF [INTERVENTION]\nis applied to [OBJECT]\nunder [CONDITIONS],\nTHEN [PRIMARY METRIC]\nwill change in [DIRECTION]\nbecause [MECHANISM],\nwhile [GUARDRAILS]\nremain within [THRESHOLDS]."},{"lang":"text","code":"IF the title proposition for the Calgary web design cost guide\nis rewritten to state the year, location, and estimator benefit,\nTHEN position-adjusted organic CTR will increase\nbecause relevance and expected utility become clearer,\nwhile qualified assessment rate does not decline by more than 10%."}],"tables":[]},{"heading":"8. LAKA experiment card","depth":2,"anchor":"8-laka-experiment-card","path":["05 — Measurement and Experiment System","8. LAKA experiment card"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"Experiment ID:\nObject:\nConditions:\nAction:\nTools:\nResources:\nExpected outcome:\nFeedback source:\nConstraints:\nValue:\nFailure mode:\n\nLAKA level:\nMagnitude:\nRate:\nDirection:\nScope:\nDepth:\nDuration:\nFrequency:\nAcceleration:\nVariability:\nDetectability:\nReversibility:\nPropagation:\nAmplification:\nAccumulation:\n\nPrimary metric:\nBaseline:\nTreatment scope:\nComparison:\nMinimum exposure:\nDecision rule:\nGuardrails:\nRollback:\nOwner:"}],"tables":[]},{"heading":"9. Decision outcomes","depth":2,"anchor":"9-decision-outcomes","path":["05 — Measurement and Experiment System","9. Decision outcomes"],"prose":["At every review, choose one:"],"bullets":[],"blocks":[{"lang":"text","code":"CONTINUE\nEXPAND\nREVISE\nREVERSE\nESCALATE\nRETIRE\nMONITOR"}],"tables":[]},{"heading":"Continue","depth":3,"anchor":"continue","path":["05 — Measurement and Experiment System","9. Decision outcomes","Continue"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF evidence is positive\nAND guardrails pass\nAND additional exposure is needed\nTHEN continue."}],"tables":[]},{"heading":"Expand","depth":3,"anchor":"expand","path":["05 — Measurement and Experiment System","9. Decision outcomes","Expand"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF positive effect is credible\nAND mechanism is transferable\nAND capacity permits\nTHEN expand to a defined cohort."}],"tables":[]},{"heading":"Revise","depth":3,"anchor":"revise","path":["05 — Measurement and Experiment System","9. Decision outcomes","Revise"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF mechanism appears valid\nBUT implementation is weak or guardrails fail mildly\nTHEN revise."}],"tables":[]},{"heading":"Reverse","depth":3,"anchor":"reverse","path":["05 — Measurement and Experiment System","9. Decision outcomes","Reverse"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF primary outcome declines materially\nOR a critical guardrail fails\nAND rollback is feasible\nTHEN reverse."}],"tables":[]},{"heading":"Escalate","depth":3,"anchor":"escalate","path":["05 — Measurement and Experiment System","9. Decision outcomes","Escalate"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF the diagnosed failure is deeper than the current LAKA level\nAND expected value still exceeds cost\nTHEN escalate."}],"tables":[]},{"heading":"Retire","depth":3,"anchor":"retire","path":["05 — Measurement and Experiment System","9. Decision outcomes","Retire"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"IF remaining value is low\nOR maintenance/risk is too high\nOR the asset has no distinct task\nTHEN consolidate, redirect, noindex, archive, or remove appropriately."}],"tables":[]},{"heading":"10. Experiment types","depth":2,"anchor":"10-experiment-types","path":["05 — Measurement and Experiment System","10. Experiment types"],"prose":[],"bullets":[],"blocks":[],"tables":[]},{"heading":"10.1 Minor element test","depth":3,"anchor":"10-1-minor-element-test","path":["05 — Measurement and Experiment System","10. Experiment types","10.1 Minor element test"],"prose":["Examples:","- title proposition;","- snippet description;","- introductory answer;","- CTA wording;","- anchor text;","- small media change.","Best for high exposure and reversible mechanisms."],"bullets":["title proposition;","snippet description;","introductory answer;","CTA wording;","anchor text;","small media change."],"blocks":[],"tables":[]},{"heading":"10.2 Major asset test","depth":3,"anchor":"10-2-major-asset-test","path":["05 — Measurement and Experiment System","10. Experiment types","10.2 Major asset test"],"prose":["Examples:","- full rewrite;","- new comparison;","- original data;","- tool or video;","- new evidence architecture.","Evaluate as an intervention package unless components can be separated."],"bullets":["full rewrite;","new comparison;","original data;","tool or video;","new evidence architecture."],"blocks":[],"tables":[]},{"heading":"10.3 Structural test","depth":3,"anchor":"10-3-structural-test","path":["05 — Measurement and Experiment System","10. Experiment types","10.3 Structural test"],"prose":["Examples:","- page merge;","- template change;","- internal graph redesign;","- facet control;","- CRM linkage;","- site architecture.","Use cohort or time-series analysis and monitor propagation."],"bullets":["page merge;","template change;","internal graph redesign;","facet control;","CRM linkage;","site architecture."],"blocks":[],"tables":[]},{"heading":"10.4 Paradigm test","depth":3,"anchor":"10-4-paradigm-test","path":["05 — Measurement and Experiment System","10. Experiment types","10.4 Paradigm test"],"prose":["Examples:","- calculator instead of article;","- benchmark instead of generic guide;","- verified directory instead of location-page set;","- interactive diagnostic instead of lead form;","- data API or agent action instead of static copy.","Primary value may include new demand, citations, brand searches, product adoption, and conversion."],"bullets":["calculator instead of article;","benchmark instead of generic guide;","verified directory instead of location-page set;","interactive diagnostic instead of lead form;","data API or agent action instead of static copy."],"blocks":[],"tables":[]},{"heading":"11. Comparison designs","depth":2,"anchor":"11-comparison-designs","path":["05 — Measurement and Experiment System","11. Comparison designs"],"prose":["Use the strongest feasible design:","SEO often prevents perfect randomization. State the design’s limits rather than implying certainty."],"bullets":[],"blocks":[{"lang":"text","code":"Randomized controlled test\n> matched page/cluster control\n> staggered rollout\n> difference-in-differences\n> interrupted time series\n> before/after with context controls\n> directional observation"}],"tables":[]},{"heading":"12. Observation windows","depth":2,"anchor":"12-observation-windows","path":["05 — Measurement and Experiment System","12. Observation windows"],"prose":["Do not use one fixed calendar window for every intervention. Define exposure criteria:","A review may occur when one or more exposure thresholds are met, with a maximum stop date to prevent endless observation."],"bullets":[],"blocks":[{"lang":"text","code":"minimum crawl confirmation\nminimum index confirmation\nminimum impressions\nminimum qualified sessions\nminimum conversions\nminimum full business cycle"}],"tables":[]},{"heading":"13. Dashboard architecture","depth":2,"anchor":"13-dashboard-architecture","path":["05 — Measurement and Experiment System","13. Dashboard architecture"],"prose":[],"bullets":[],"blocks":[],"tables":[]},{"heading":"Executive dashboard","depth":3,"anchor":"executive-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Executive dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"incremental organic contribution margin\nqualified organic conversions\nvalue-weighted non-brand clicks\nvalue-weighted target-cluster visibility\neligible canonical coverage"}],"tables":[]},{"heading":"Demand dashboard","depth":3,"anchor":"demand-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Demand dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"cluster impressions\nnew query families\nbranded vs non-branded\njourney state\naudience/offer\ncountry/device\nvalue-weighted demand"}],"tables":[]},{"heading":"Asset dashboard","depth":3,"anchor":"asset-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Asset dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"canonical owner\nrank distribution\nCTR residual\ntask completion\nnext-action conversion\nfreshness\ninformation/evidence score"}],"tables":[]},{"heading":"Technical dashboard","depth":3,"anchor":"technical-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Technical dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"status correctness\nrender parity\ncanonical agreement\nindex eligibility\nsitemap coverage\norphan rate\nclick depth\nfield CWV exposure\nstructured data validity"}],"tables":[]},{"heading":"Authority dashboard","depth":3,"anchor":"authority-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Authority dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"qualified new/lost referring domains\nrelevance\ntarget page\nplacement type\nretention\nbrand-demand lift\nassisted value"}],"tables":[]},{"heading":"Experiment dashboard","depth":3,"anchor":"experiment-dashboard","path":["05 — Measurement and Experiment System","13. Dashboard architecture","Experiment dashboard"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"change ID\nLAKA level\nstatus\nprimary metric\neffect estimate\nuncertainty\nguardrails\ndecision\nlearning"}],"tables":[]},{"heading":"14. Metric anti-patterns","depth":2,"anchor":"14-metric-anti-patterns","path":["05 — Measurement and Experiment System","14. Metric anti-patterns"],"prose":[],"bullets":[],"blocks":[{"lang":"text","code":"DO NOT:\ncount indexed pages as success without value;\ntreat average position as a precise rank;\nuse a single sitewide CTR benchmark;\nequate bounce with failure for every task;\nequate tool authority scores with Google internals;\nclaim revenue from a forecast;\nattribute all movement to the most recent change;\nignore branded demand created by other channels;\noptimize event count rather than user state;\ncontinue an experiment without a stop rule."}],"tables":[]},{"heading":"15. Monthly learning review","depth":2,"anchor":"15-monthly-learning-review","path":["05 — Measurement and Experiment System","15. Monthly learning review"],"prose":["Ask:","The output is an updated rule prior, portfolio, and backlog—not merely a report."],"bullets":[],"blocks":[{"lang":"text","code":"Which clusters created the most qualified value?\nWhich assets lost demand or eligibility?\nWhich query families are emerging?\nWhere is the intended canonical not dominant?\nWhich formats improve task completion?\nWhich evidence earns links or branded demand?\nWhich next actions produce quality, not merely volume?\nWhich technical failures block the most value?\nWhich LAKA level produced the best value per effort?\nWhich assumptions were disproved?\nWhat should the generator weight differently next month?"}],"tables":[]}]}