High Tempo Testing for Success

What is High Tempo Testing?

Most growth professionals understand that growth hacking isn’t about a specific tactic, but rather it’s about a process of discovering which tactics will be effective for growing your business. This discovery happens through testing potential growth drivers ranging from new channels to new sharing and engagement features deep within a product. The more tests you run, the more you learn about how to grow your business. So it’s only natural to want to run as many tests per period of time as possible. That’s the idea behind high tempo testing.

My inspiration for high tempo testing comes from American football and the “high tempo offense.” Perhaps the best example of this is the Oregon Ducks’ college football team. From the moment they hit the field they are go, go, go… The opposing teams often end up on their heels and Oregon is able to find weaknesses in their defense. It’s exciting and frankly kind of exhausting to watch. High tempo testing is approached with a similar energy and urgency to quickly uncover growth opportunities to exploit.

Requirements for High Tempo Testing

Generating enough ideas

High tempo testing starts with identifying experiments worth running. If one person is responsible for all ideation, they generally run out of ideas within a few weeks (at least ideas worth testing). Even within a dedicated team, ideation can become ad hoc and stagnate without a process in place that acts as a springwell of new ideas.

To jump start fresh ideas for growing open up the ideation process up to a much broader team. Naturally this includes everyone on the growth team, but we also invited our interns, engineers, salespeople, support people and more. Essentially any employee or contractor in the company that wanted to participate. But we didn’t stop there. We included advisors, friends and even some of the most active community members.

We collected hundreds of ideas from this group. Interestingly, different groups contributed ideas based on insights that were unique to them. Some people gravitated toward ideas to improve retention, while others focused on acquisition or referral. The ideas themselves ranged from ways to improve community engagement to product enhancements.

For example, one of the engineers suggested we should use Oembed to make it easier to embed highly engaging multimedia content. Most of the rest of us didn’t even know that was possible and had suggested a more manual media upload process.

Canvas Ideas
 

After the ideation started to slow down, we decided to add a leaderboard to celebrate the people who were generating the most ideas. The leaderboard re-energized the ideation process, so that we are now adding about 2 – 3X more ideas per week than we can test. Therefore, we are continuing to grow our backlog of hundreds of ideas.

Canvas Leaderboard

Prioritizing Ideas

Eventually we faced the challenge of sorting through all of the ideas. With hundreds of ideas, it was tough to know where to start. We scored every new idea based on its potential impact on growth, our confidence that it will be successful, and how easy it is to implement. In other words, using an ICE score. Once the ideas were scored, it was easy to sort them and find the best ideas in the areas we thought we should prioritize first (like activation).

Canvas list view

Managing Tempo

In the beginning we often got bogged down trying to launch a big idea. Each time our tempo slowed down (missing our goal of launching three tests per week), our growth slowed down.

The solution was a Weekly Growth Master meeting. This meeting is not for ideation. It is for processing the backlog of ideas that we already have, prioritizing and assigning the list of ideas for the next week. Think of it like a sprint kick-off in agile development. The backlog has already been groomed, the user stories are already written, it’s time to build the weekly growth agenda (or sprint in agile terms).

Some growth experiments can be implemented by the marketing team, others by product managers and others require deep engineering skills. Balancing the workload of top priority experiments across different teams makes it much easier to hit our tempo goal. While our goal is to launch at least three tests per week, we generally shoot for five tests per week. That way if we hit roadblocks on some tests, we still hit the tempo goal. Many weeks we are able to get all five tests out the door.

Our weekly growth master meeting is also used to review the tests from previous weeks. Were we able to implement all of the tests planned? Have any tests concluded and if so what were the key lessons learned? This accountability and planning meeting is an essential part of keeping the growth process on track.

Capturing Learning

All of the completed experiments are ultimately stored in a knowledge base so we ensure that we capture this learning and don’t keep repeating the same tests. The knowledge base is also very helpful when adding new members to the team, so they can understand what has been tested, what worked and what didn’t.

The analysis has started to become a bottleneck for us, so we recently added a role that is responsible for analyzing the completed tests and managing the knowledge base. Previously this responsibility was shared by the team.

Canvas knowledgebase

High Tempo Testing Drives Results

High tempo testing isn’t easy, but it is extremely effective. It requires both a rigorous process and a system to manage that process. Much like lean manufacturing it focuses on throughput and small batch sizes. By adopting high tempo testing, we eliminated the bottlenecks at the ideation and experimentation steps to unlock more growth. If your team is truly committed to driving growth,High Tempo Testing is the most predictable way we’ve found to hit aggressive targets.

No comments yet.

Leave a Reply