← All Guests

Archie Abrams

Shopify

6 debates 6 evidence cards 1 episode
Metrics Should product decisions be driven primarily by data and experimentation, or by taste and intuition?

"Hybrid approach with an unusual twist: core product teams have no KPIs (essentially banned), making decisions on taste and intuition toward a 100-year vision, while the growth team uses long-term holdout experiments that often reveal short-term metrics are misleading."

Design Should the design quality bar be enforced by a small group of taste-keepers at the top, or distributed across every PM, designer, and engineer?

"At Shopify, a small handful of people — Tobi, Glen, and a few deputized group leads — hold the quality and taste bar for the entire core product, and nothing ships without their explicit approval."

Metrics When does A/B testing help vs. hurt product development?

"Core product teams should NOT have KPIs - use taste and intuition instead; experimentation is for growth"

Metrics Are OKRs effective or do they create more dysfunction than alignment?

"Ban KPIs for core product teams - use loose OKRs only for coordination"

Metrics Is retention the only metric that truly matters?

"Optimizing for churn is better than optimizing for retention - lower barriers, accept failure"

Product Should roadmaps have dates or be flexible outcome-oriented documents?

"Roadmap priorities should be driven by a 100-year CEO vision, not quarterly planning"

Shopify

30-40% of experiments showing short-term lift have no GMV lift a year later

Shopify's three product groups: core (100-year vision, no KPIs), merchant services (medium-term), growth (experiments with long-term holdouts)

Shopify

Shopify maintains holdout groups for every experiment and automatically revisits results at 3, 6, 9, and 12 months

30-40% of experiments showing short-term metric lift showed no incremental GMV lift a year later

Shopify

Shopify: 10% of US e-commerce, $235B GMV in 2023 (roughly Finland's GDP), achieved without core team KPIs

Long-term holdout experiments: automatic re-evaluation at 3, 6, 9, 12 months showing 30-40% of 'wins' have no lasting effect

Shopify

Shopify represents 10% of US e-commerce with $235B GMV in 2023 -- roughly the economy of Finland -- built on a model...

Shopify represents 10% of US e-commerce with $235B GMV in 2023 -- roughly the economy of Finland -- built on a model that accepts high churn

Shopify

Shopify's core product group operates with no KPIs or metrics -- they are literally banned -- relying instead on...

Shopify's core product group operates with no KPIs or metrics -- they are literally banned -- relying instead on Tobi's 100-year vision for commerce

Shopify

At Shopify, nothing ships in core product without an 'okay-to' approval from Glen, Carl, or Archie — a three-person gate for a 600-person org

A tiny central group holds the taste bar for every release; Glen Coates reviews every single one in depth before it ships

esc
Loading…
navigate filter openesc close