🍪 CRO Bites #14: CUPED Applied - 8 Tech Companies Sharing Insights
The best CRO resources, delivered monthly and powered by Conversion Stash
Hi there 👋
Most of us want to test faster, but random noise often forces us to keep experiments running longer than they should.
Enter CUPED (Controlled-experiment Using Pre-Experiment Data). Coined by Microsoft back in 2013, this powerful variance reduction technique has become an absolute game-changer for experimentation programs.
In a nutshell: CUPED uses historical data from before your experiment starts to clean up the noise in your current test. By filtering out this background noise, it dramatically increases both the speed and accuracy of your experimentation program, allowing you to detect true winning variations much faster.
What is particularly exciting is the wealth of practical knowledge that has been shared over the years. Several tech companies have published excellent papers and articles explaining how they apply CUPED to scale their testing velocity.
We curated the most valuable publications from 8 leading tech companies:
Microsoft (2013) | Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data
Netflix (2016) | Improving the Sensitivity of Online Controlled Experiments: Case Studies at Netflix
Booking.com (2018) | How Booking.com increases the power of online experiments with CUPED
Meta (2020) | Increasing the sensitivity of A/B tests by utilizing the variance estimates of experimental units
DoorDash (2020) | Improving Online Experiment Capacity by 4X with Parallelization and Increased Sensitivity
Airbnb (2021) | Zero to Hero: Exploiting Null Effects to Achieve Variance Reduction in Experiments with One-sided Triggering
Etsy
Google Cloud (2025) | Tune Out the Noise: Use CUPED to Save A/B Tests from Relaunching
Special thanks to all authors of the individual papers and blog articles for sharing your expertise!
Looking for more engineering insights? We curate these and other experimentation blogs directly at Conversion Stash.
🏆 Resource of the Month
Confidence Blog by Spotify
Explore expert articles on experimentation and technical insights from Spotify’s R&D team. Confidence is an experimentation platform by Spotify that helps you run experiments and make data-informed decisions throughout your entire business.
📣 Experimentation Event of the Month
Circus London 2026 (formerly known as CXL)
Join 200+ in-house leaders & innovators to explore and shape the future of how we build, ship, test, and work.
👉 Claim your FREE ticket: The organizers are looking for moderators to help out and are offering a few free tickets in return. Interested? Reach out to Ben Labay via LinkedIn DM (mention Conversion Stash) or email us directly.
📆 Upcoming Experimentation Events in 2026
🇳🇱 MeasureCamp Amsterdam | April 18, 2026 | Amsterdam, Netherlands
🇦🇹 Experimentation Unite Vienna | April 22, 2026 | Vienna, Austria
🇺🇸 Marketing Analytics Summit | April 27-30, 2026 | Santa Barbara, USA
🇩🇪 Berlin Experimentation Meetup #13 | April 28, 2026 | Berlin, Germany
🇬🇧 Space Academy by ExElite | AI That Works | May 6, 2026 | London, UK | 🏷️ Save 12 % with code
CS12🇬🇧 Space Academy by ExElite | Tomorrow’s World Today | May 7, 2026 | London, UK | 🏷️ Save 12 % with code
CS12🇬🇧 Circus (formerly known as EXL) | May 11-12, 2026 | London, UK
🇳🇱 Optimization Meetup Utrecht #2 | May 28, 2026 | Utrecht, Netherlands
💻 Best In Test Awards #2 | June 10, 2026 | Online
💻 The Women in Experimentation Summit | September 14-15, 2026 | Online
🇬🇧 Experimentation Elite Awards | September 24, 2026 | London, United Kingdom
🇳🇱 DDMA Experimentation Heroes #10 | November 5, 2026 | Amsterdam, Netherlands
🇳🇱 Conversion Hotel | November 20-22, 2026 | Texel, Netherlands
👀 Check out the A/B Testing Tool Comparison 2026
Compare the best experimentation tools and find the perfect fit for your organization.
🧩 Featured Assets
50 Experimentation Interview Questions
Hiring for CRO or preparing for your next role? This list of 50 essential questions will help assess candidates’ expertise in experimentation, analytics, and optimization strategy.
CRO Glossary Cheatsheet
Master the language of experimentation with our CRO Glossary Cheatsheet. This quick reference guide covers 100+ key terms, helping you navigate A/B testing, statistical significance, and optimization frameworks like a pro.
💼 Looking for a New Challenge in Experimentation?
Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week’s featured roles:
Medior CRO specialist at De Mandemakers Groep (Waalwijk, Netherlands)
Data-analist (Experimentation & Digital Analytics) at Online Dialogue (Utrecht, Netherlands)
Senior CRO Specialist at sofatutor (Germany & EMEA)
Director, Web Experience and Optimization at Mimecast (Lexington, USA)
Head of Digital Optimisation at Frasers Group (London, United Kingdom)
Are you hiring? Submit here
🙌 Want your Product discovered by our Audience?
We love partnering with brands that resonate with our audience. Drop us a line at info@conversion-stash.com to explore sponsorship options!
Stay optimized,
Niklas & Nils from Conversion Stash
👉 Follow us on LinkedIn (1k+ CROs & Experimenters already do)