This post is a part of Made @ HubSpot, an internal thought leadership series through which we extract lessons from experiments conducted by our very own HubSpotters.
My role at HubSpot is the French Marketing Automation Manager — essentially, I’m in charge of the marketing automation for HubSpot leads and free users in our French market. This past January, we noticed that one of our key automation programs — our onboarding and nurturing email program for our free users — was dated and needed a revamp.
When our free users sign up, they receive a series of onboarding emails that helped them get set up and answered any potential questions. Here are some screenshots of the old onboarding emails:
As helpful as these emails were, we decided we could do better by our users and leads.
As we started brainstorming and creating proposals for this program revamp, we came across a core challenge: We didn’t really understand our users’ needs, challenges and goals.
At HubSpot, we do a lot of user research, but much of this research is focused on English users. Also, as marketers, we don’t talk to our customers as much as our sales and customer service teams do. Yet, we can all agree on one thing: The better you know your audience, the better you can serve them and help them.
Phase One: Researching our Free Users
Before working on our email program revamp, we decided to get to know our audience better. We chose to conduct research on our French free users. We did this by releasing one survey and holding several interviews.
The survey was first sent to two audiences: active free users and inactive free users (those who haven’t used their portal for more than 30 days). We were curious if there were differences between the two audiences, but we didn’t discover many.
When analyzing the survey results, we identified contacts with various job titles and different expectations when they signed up. Users that I named “end users” represent about 80% of our free users and signed up only to use the free tools. The other 20% represent what I call “evaluators,” and they signed up to discover HubSpot while thinking of buying a paid version. They are here to evaluate the tool and use the sign up as a trial.
For the interviews, I wanted to get a mix of evaluators and end users and also people from different job fields and departments to better understand their needs. So, I interviewed a wide variety of our free users. Here are a few responses that stood out when I asked “Why HubSpot?”
“[To] find a database, more automated and fluid and adapted than Excel.” — Marketing Manager
“The objective was to bring together [my] various contacts from different bases. [I] then discovered the other features of HubSpot. ” — CEO
The results of the survey and interviews were incredibly insightful. We learned a number of important things about our free users. Here are a few main takeaways:
- Most of our free users are part of small to medium-sized enterprises (SMEs) and therefore, wear multiple hats and do multiple jobs. For example, some folks were CEO and made sales.
- They mainly come to HubSpot for the free CRM. They want and need a CRM, and that’s why they signed up for HubSpot.
- Our free users are not aware of everything that’s included in their free plan, so they are even less aware of what’s included in the paid plans. For example, some users didn’t know they had access to the Form Builder or Email Marketing tools and were using external tools for these functions. Additionally, some users didn’t realize the Landing Page Builder and Workflows features were available with the paid plan. When discussing this during the interviews, they confirmed that they are interested in knowing more about HubSpot paid plans.
What we discovered about our audience and free users changed our thinking quite a bit. For example, before conducting this research, we weren’t promoting our paid plans and features to our free users. That was about to change.
Phase Two: Email A/B Testing
With our new research in mind, we were ready to address our onboarding and nurturing email program.
Before rolling out a new nurturing email sequence to our free users, however, we first wanted to test their reactions to the new content. They’d told us during the market research that they’d be interested in learning more, but how would that translate to reality?
We decided to run an email A/B test to measure how our audience responded to this new approach.
First, we set a hypothesis: Our free users were not aware of the possibilities and features included in their free plan and even fewer of those included in paid plans. If we provided more visibility into what they could do with a paid HubSpot plan and made an effort to educate them about how to try those premium features, they would be pleasantly surprised and sign up for trials — and eventually purchase.
We believed this because our free users mentioned they weren’t fully aware of what was included in the paid plan and why they should subscribe.
So, we developed an experiment to test this new approach. For this our A/B test, I didn’t want to target all of our free users — only the ones who’d been actively using the HubSpot free tools in prior few months. I assumed this audience would be interested in learning more about what they could do and achieve with a paid version since they were already so active in the free version.
We divided these active free users into two segments:
- Our control group didn’t receive any nurturing emails, as this was the case before the experiment.
- Our variant group received a series of two emails promoting key paid features, including Attribution Reporting, Workflows, and Sequences, and offering them a trial to test out those features.
Here are some screenshots of the text emails:
Our Email A/B Test Results
We ran our email experiment over the course of four weeks and sent about 3,000 emails.
At the conclusion of the test, we were pleased with the results as they were highly conclusive and confirmed our original hypothesis. When we compared the control and variant groups after one month, we found that the emails sent to the variant group:
- Generated 71% more qualified leads
- Created 125% more deals
- Generated 12% more trials
- Generated 15% more revenue
Learnings and Takeaways
Both the research and experiment we conducted yielded valuable insights and takeaways. Here are my top four:
First, get to know your audience. As marketers, we don’t often have the opportunity to discuss with our audience.
If you can chat with your target market and customers, I highly recommend it! This can be a gamechanger for your marketing strategies and alert you to insights and trends you otherwise wouldn’t have known.
Second, align with Sales. Sales talks to hundreds of prospects and free users per week, so they know what features and components of your product or service are most important for certain leads.
You probably can’t promote your entire product in one email and have to select some features. For this reason, you should better align with Sales to better understand who your audience is and what their goals, needs, and pain points are.
Third, test your content formats and information to know what your audience prefers.
For example, we’ve seen in France that doing short videos or GIFs of HubSpot tools was highly appreciated. For each feature or functionality of the tool, our emails included a text description, and, next to it, a video or GIF of it. This helped our readers see what the tool looked like in action.
Finally, experiment as much as you can.
Even if your experiment is not conclusive or doesn’t yield the results you were looking for, it’s not a waste of time if you’ve learned something new and valuable about your audience, content, or product.