Mastering A/B Testing: Lessons from Chess.com (200M+ Users) & Phil Carter

Running an A/B test is easy. Running one that delivers insights you can trust and repeat at scale is not. It takes careful planning, sharp execution, and clear rules for interpreting results.

In this session, Phil Carter (Elemental Growth, ex-Faire, Quizlet, Ibotta) and Joe Wilkinson (Chess.com, ex-Meta) will share advanced A/B testing strategies and hard-earned tactics from years of hands-on experimentation. They’ll show how to design tests that truly scale in web2app funnels, where every misstep means wasted spend and lost users.

You’ll learn:

• When to use different types of A/B tests (standard A/B, A/A, “Do No Harm”, etc.)
• How to set success, secondary, and guardrail metrics
• How to calculate sample size and runtime for trustworthy results
• How AI is transforming companies’ approach to A/B testing setups and techniques

Note: If the registration form doesn’t load, try opening the page in Google Chrome