Archive by Author

Starting a Growth Team from Scratch

Most high-growth companies now have Growth Teams (e.g. Facebook, Uber). These teams have enormous potential… Creating a Growth Team isn’t as easy as it sounds. You’ll find many roadblocks coming your way and going lean (generating results with small money) is the first step to actually building a standalone team. This article will walk you through the […]

Reach A/B test significance faster with this little known optimization hack!

What if I told you you could run tests up to 2x faster using a standard optimization feature hardly anyone uses? The concept is really not that difficult, but so many people get it wrong that it easily costs marketers and product owners millions of dollars a year in lost revenue. I’ve made the mistake myself, […]

Sample Pollution: The A/B Testing Problem No One is Talking About

Here’s an uncomfortable truth about conversion rate optimization: lots of people are running bad tests without even knowing it. They’re making decisions based on false positives, they’re acting on inconsistent data, they’re avoiding the issue of sample pollution – I could go on. “If you don’t know about sample pollution, stop testing.” The first step […]

Why Sampling Your Analytics Data is Bad

It’s long been the case that the costs associated with using high-quality analytics services force companies into tradeoffs of picking which events or users to track, with sampled data being the most common fate that people resign to. Unfortunately, sampling can be very harmful, leading to a slew of problems including inaccurate test results and a […]

When to Run Multi Armed Bandit Tests Instead of A/B/n Tests

When should you use bandit tests, and when is A/B/n testing best? Though there are some strong proponents (and opponents) of bandit testing, there are certain use cases where bandit testing may be optimal. Question is, when? First, let’s dive into bandit testing and talk a bit about the history of the N-Armed Bandit Problem. […]

Conversion Optimization Tips For Low Traffic Sites

At its core conversion optimization is the idea of testing out different versions of your pages, different copy, different visuals, etc. to make the pages get as much conversion as possible. “I have too little traffic to test; hence, I can’t do conversion optimization.” However, to conduct a scientific test, you need enough visitors to […]