awesomechecklists
llms.txtGitHub ↗
/

20 — Analytics and Product Metrics Checklist#

Verify that product metrics answer meaningful decisions and are built on well-defined events, trustworthy data, privacy-aware collection, and sound experimentation. This template does not certify that a particular analytics vendor's reports or causal claims are correct.

Scope: AI-specific. This file is one part of the 20_Analytics_and_ProductMetrics.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Critical product metrics and metric owners:
  • Analytics platforms / tracking plan version:
  • Consent and data-processing review owner:
  • Reviewer / owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Copy this shared checklist into a project-specific review; leave all repository checkboxes unchecked.
  2. Mark [x] only after verifying an item using approved metric definitions, a tracking plan, validation tests, reconciliation, representative reports, an experiment design, or authorized privacy decisions. Record failed items and valid N/A rationales in project notes/issues keyed by ANL ID rather than adding evidence tables here.
  3. Tailor measurement to actual product goals and lawful data collection. Not every service uses advertising attribution, browser analytics, monetization funnels, experiments, or GA4.
  4. Review ## API Checks and ## Frontend Checks independently when applicable; document N/A and why for missing product surfaces in project records.
  5. Revalidate after event-schema, consent, attribution, instrumentation, pipeline, identity, or metric-definition changes. Complete detailed Security, Legal, Data Governance, and Experimentation specialist reviews separately where needed.

Measurement strategy and metric definitions#

  • ANL-008 Choose meaningful leading, lagging, and guardrail indicators instead of relying on a single aggregate KPI.

Experimentation and decision quality#

  • ANL-087 Choose a primary outcome metric and a limited set of guardrail metrics before observing results.

Release gate#

  • Launch-critical business and user outcome metrics have stable definitions, owners, targets or baselines, and documented measurement limitations.
  • Critical events have an approved tracking plan, unique trigger semantics, appropriate identity and attribution rules, and validated representative client/server payloads.
  • Key conversion, transactional, or financial metrics reconcile with the authoritative system to an agreed tolerance, or material gaps have an accepted remediation plan.
  • Applicable consent, notices, data minimization, privacy controls, recipient access, and retention requirements have been reviewed and approved by the appropriate owner.
  • Analytics ingestion and dashboards are checked for freshness, missing events, duplicates, inappropriate test traffic, and known export or modeling limits.
  • Any launch-critical experiment has a documented hypothesis, decision metrics, randomization/health checks, and approved stop or rollout rules.
  • Responsible API/frontend owners confirm that relevant user journeys are measured correctly without disrupting functionality or collecting prohibited sensitive data.
  • Unresolved defects that could materially mislead product decisions, harm users, or breach applicable data-processing requirements have an owner, escalation path, and explicit authorized disposition.

Domain-specific approval rule: Do not approve measurement as release-ready when critical decisions depend on undefined or demonstrably incorrect metrics, required consent/collection rules are unaddressed, server-side business events systematically double count, or invalid experiment allocation is used to claim causality. Analytics review is not a substitute for legal/privacy compliance review or independent financial audit.

Trusted references and scope#

These checks translate source guidance into testable measurement and decision controls. Vendor-specific technical details apply only when that platform is used; these references are not a universal compliance standard.

Applicability: GA4 event names, API payload limits, and BigQuery export semantics are relevant only to GA4 implementations. UK tracking obligations depend on applicable law and current regulatory guidance, including changes made in 2026; seek qualified legal advice for jurisdiction-specific decisions. Online controlled experiments should be used only when methodologically appropriate. Consent-driven gaps, identity uncertainty, ad blockers, and sampling can affect measurement completeness and must not be ignored.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: event naming, consent, schemas, experiments, and reporting integrity; include model and dataset behaviors where applicable.
  • Cross-stack: GA4 DebugView.
  • Python: Pandera + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: Zod + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: Ajv + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • analytics — Review event data and measurement plans (coreyhaines31/marketingskills, 59,899 repo stars).
  • ab-testing — Assess experiment controls and results (coreyhaines31/marketingskills, 59,899 repo stars).
  • kpi-dashboard-design — Check metric definitions and dashboards (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the AI-specific scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.