How to Run Growth Experiments? And Measure Them
Running growth experiments is not something complicated, anyone can do it. Fundamentally, it is just testing out an idea – it can be different titles for your landing pages, hashtags for your social media platforms, or changing the theme of your website.
Growth Marketing and Performance Marketing are very similar in this way, but what makes growth experiments successful is making decisions after measuring their results. That is where the complexity comes, especially when running multiple experiments simultaneously.
Thankfully, there is a structure you can follow to make sure you are measuring your growth experiments accurately. Let's dive into it.
List down your experiments
Start with a table, where you can list down all of your experiments. The way to do it is to note down the hypothesis behind it, and the funnel it will affect. If you have a broad experiment, for example, a design change that is going to affect multiple things, you should list it down for individual funnels.
At nTask, we changed the color of our logo and it affected a lot of things, the theme needed to be updated, and the same color was used in our emails, our social media, and everywhere else. We listed this as 12 different experiments because it would have affected each of those things differently.
Here is an example of how to list down your experiments
Determine factors to run the experiment
Every experiment is different, and prioritizing them is fundamental to your growth marketing efforts. Use this simple framework by determining what it would take to run an experiment and what outcome to expect. Give each experiment a value on a scale of 1-10:
How many working hours would it take to make it happen?
If the effort required needs to come from multiple teams, these should be measured separately.
How much impact would it have on the funnel in question?
What is the probability of success with the experiment?
How much confidence do you have that it would be successful?
You can label them as you want, I like to call them effort, impact, probability, and confidence. Many growth hackers also use more factors to determine the priority of their experiments, and you should do that too if you feel it is important for your business.
For example, you may want to factor in the time in which you will be able to decide whether this experiment was successful or not. Add anything that makes a significant impact on your decision-making, but don't get carried away by making it complex.
Score the factors
The next step is obviously to score the factors you decided on. Give them a score between 1-20 and then create an average score out of all of them to determine the priority.
One thing I would strongly recommend is to calculate the weighted average of these and not treat the factors equally. For example, it can be important for you to put less stress on the dev team and effort might be a bigger challenge for you. In this case, you should put more emphasis on running experiments that require less effort first.
I know it sounds complex and is not always a rule of thumb, but it helps to prioritize. You should always look at the individual score of every factor before finalizing the priority and not just use the average score.
Want a sample Google sheet to measure your experiments?
Group experiments and start experimenting!
A good idea is to group experiments by funnels and start working on running them. This will allow you to scale to success faster because you will always have some tests running on each of your funnels.
This can also be challenging at times when you have a small team – in that case, you can group experiments by teams. Again, a smaller team would mean overlap between multiple people/teams to run one experiment, so assign owners!
An owner should be a person/team that has the most amount of effort going into an experiment and who is directly responsible for that particular function. Grouping by owners will allow you to not burden your teams and have a healthy workload ratio across your growth team.
Measure the outcomes
You should consider three things when measuring the outcome of your experiments.
Result, which is the quantitative and qualitative result of the experiment. Be as accurate as possible about it.
Learning, which tells you the subjective learning you got from running that experiment, even if it was a failure. Failed experiments, make way for new hypotheses and more experiments.
Difficulty to scale, which measures how much more effort would it take to scale a successful experiment. This can vary because sometimes your learnings might tell you that the experiment was a success but it is going to take a lot more effort on average to scale it up.
Sometimes, an experiment gives you multiple outcomes and you should note it that way. For example, changing your hashtags on social media might have given you more engagement, but fewer impressions. That can mean that you need to use the hashtags used earlier as well, to reach more people.
Noting down multiple outcomes will help you determine if your experiments should entirely replace something or combine it with how you were doing things before the experiment.
This is very important because a lot of growth experts sacrifice what is already working to find faster solutions and more often than not, it results in unsustainable growth patterns.
Scale away!
Now that you have solid learnings, and everything by the number, it is time to decide which experiments are promoted to be scaled. Choose wisely, and spend time studying your outcomes and what impact they can have on your overall efforts as a team and as a company.
Being hasty about scaling leads to unsustainable growth. Always prioritize efforts that can give you long-term growth even if these are slow to scale. Speedy growth is good if you need to achieve short-term goals, but for long-term sustainable growth, that might not be a good thing.
Need help with growth experiments? Just ask.
Running regular growth experiments is important to achieve growth. No matter which industry you are in, consistently testing your efforts will allow you to get to success much faster. It can become challenging, and sometimes you need an extra set of brains to help you with it.
If you are a business interested in growth, I do direct mentorship with early-stage startups and small businesses for free. Just leave me an email at email@fahadsheikh.net or fill out this form.
Even if you are not a small business or a startup, I can help you run growth experiments efficiently and save countless hours of work. Reach out if you need consulting on growth, marketing, sales, or SaaS product development.