Here’s a blog post from Baptiste Debever, the co-founder of Feedier – a feedback-gathering SaaS tool:
In an ideal world, we would ship our product, see the users coming in naturally and move onto the next product.
But we are not in such a world, and product management is synonym to a constant loop of innovation.
You have likely heard terms like feedback loop, lean process, MVP, ship fast, A/B tests and all of those fancy words.
They are all synonyms with product improvement through data, or, in other words, running constant experiments on a weekly basis in order or analyze how users interact with our latest change and features, and decide if we can move forward.
And that’s exactly what we want to talk about today.
Although we do like data, I will also help you avoid falling into the trap of being data-driven, and rather adopt a data-informed stance.
Don’t get me wrong, I am not saying that you have to make decisions based on your feelings only, but you, as a company, have to find the right trade-off between using data to make every single decision and listening to your gut and your customer’s voice.
So, without further ado, let’s dive right in.
What will we discuss?
We will first give you a brief overview of the difference between both approaches, then we will discuss the processes needed in order to maintain a data-informed posture by listening to your users, and we will finally talk about the decision making in a data-informed world.
Note that this article will mainly focus on software products, even though we could apply the model to most types of product.
Difference Between Data-Informed and Data-Driven
In a data-driven company, the product team is on the constant lookout for ways to improve the key metrics or KPIs, without actually looking at the external factors that might have impacted the changes.
For example, your churn might be going up simply because a new competitor has just entered the market and it turns out to be very attractive and efficient. Or, it could simply be the fact that the timing is not right for the product, and the customers will come back to you later.
Therefore, in order to improve their key performance indicators, these data-driven teams ship improvements as fast as they can, measure the results, and use the data to make their next shot. Then, they loop again.
On the other hand, a data-informed team will value the voice of the customer, the external context, their experience and customer needs as much as the data.
Data is nowhere near the only factor influencing the decision, it is only one of them. Thus, data-informed teams are able to blend different aspects in order to make the right call.
Data-informed teams are able to fit the data into their global strategy and action plan and move forward by combining them with the data analysis.
Hexter Data Analysis Information
This was nicely put by Alastair Simpson from the Atlassian team:
We shipped an experiment to improve our onboarding experience for new customers. From a data perspective, the result was a huge failure. It resulted in -12% engagement (time spent using the product) and neutral conversion. So did we throw it away and move to something else? No, the team believed in the experiment we had designed and our qualitative research was telling us that we were on the right track. Our conviction turned out to be correct. We took some lessons from the data, applied some insights from our qualitative research, iterated on the experiment, and with tweaks to the design, it became a success at +22% conversion, with engagement staying neutral.
This is being data-informed, not data-driven. We used the data we had and combined it with qualitative feedback and our design intuition to produce an iteration that was ultimately successful. We weren’t binary, we didn’t throw it away completely at the first sign of trouble.
Besides this, the illustration from Josh Porter perfectly depicts the situation and explains the concept of being data-informed.
Is data-informed always the best shot?
Don’t get me wrong, the data-driven has also its use-cases. As mentioned by Andrew Chen in this article, “the first problem with being data-driven is that the data you can collect is often systematically biased in unfixable ways.”
In other words, it’s relevant to collect and refer to the data when you have a lot of data available in a very short window.
The percentage of people opting in or subscribing, for instance, would be a good fit here. You could run A/B test on the signup page and see which one works best.
But, on the other hand, when you rely on a very scarce or costly event, it is anything but relevant to rely on data alone.
Great… But now, you might be wondering what does it take to build a data-informed company?
Well, that is exactly the topic of our next section, how to create a loop in order to feed our insights.
Create a Design Feedback Loop
We have explained the concept of data-informed companies that manage to mix-match the data at hand with their own experience, the context and the voice of their users.
Which leads us to the second point: the user feedback.
As a matter of fact, in order to build up empathy and be able to really understand your users’ needs, you have to listen to their feedback.
Effective feedback, both positive and negative, is very helpful. Feedback is valuable information that will be used to shape important decisions. Leading companies became leading because they consistently search for ways to improve by listening to their customers.
For that reason, it’s important to have a great feedback loop that allows you to see first hand how customers are feeling about your product.
When you’re in the startup phase, gathering feedback is relatively easy during the conversation with your early adopters.
However, as you scale, enabling customer satisfaction teams to evaluate and pass on feedback to the product team is critical. You need to have a properly looped and automated system in place in order to analyze the qualitative feedback, combine it together with the quantitative data and make the right decision.
So, the first step to getting feedback is to scale your app…
The goal here is to put a feedback request whenever it’s relevant to your users, or, in other words, when they are interacting with the very specific thing you would like to get feedback on.
For instance, if you are collecting feedback on a new feature, you could use a feedback tool offering the ability to embed a widget appearing on the web-application when the user interacts with the feature.
But, you could also be interested in collecting feedback in order to understand what drove some users away. In that case, the widget won’t make sense.
Here the better option is to pick up the phone and call the relevant users.
Then, we can use those insights, and automate the process of sending them over to the product team. Using tools such as Zapier, we can automatically push them through the project management tool we are using.
The feedback loop
Using this loop will help you build up empathy and really understand the drives that led your customers to sign up.
The loop is broken when we are not able to scale anymore, whether it’s because our users are not converting or because we didn’t invest enough in improving the UX.
Hundreds of organizations struggle with two very important things; taking action to drive true business impact and following up with the most important shareholder, the customer.
This is where businesses are in need of a closed loop process. Closing the loop on customer feedback means you are listening to the customer, taking action based on their input, and driving positive, transformational change within your organization to improve the overall customer experience.
Hopefully, by now, we have shed a light on a better way of leading a data-informed team.
The rubber starts to hit the road now… We have to make decisions using our new process.
To sum up, our quantitative data helps us understand the WHAT and the qualitative data from feedback and interviews helps us understand the WHY, in the question of what could be done better.
But that’s not all, do not put aside your emotional intelligence and experience. Do you think Facebook was created on the basis of data? Well, no, it all started from intuition, the intuition that people would hang out and connect online exchange value and stick to it.
Data doesn’t replace human intelligence, clear-headed and strong decision-making, it helps you, the human, make a better decision.
I hope you enjoyed this piece. If you have, there is more like this on our blog.
Feel free to hit me up in the comment, and share this article! 👏
With data-informed decision making taking the stage, having a steady stream of high-quality, quantitative data coming in is crucial for keeping up with the feedback-improvement design loop.
Here we are talking about analytical data on the performance of your website that will give you an insight into user intent and behavior.
By using Morningscore’s SEO tool you will be able to track which keywords suit your target group best and bring in the most traffic. Using the best performing keywords will help you appear before the user at an optimal time in their buying process, namely, the consideration and decision phase. This strategy will pull the customers in and give you a higher conversion rate than a traditional push method where we shower the potential customer with ads when they haven’t even begun to show interest.
About the author
Baptiste Debever is a French entrepreneur, co-Founder and Head of Growth at Feedier, an app that helps companies collect better feedback through gamification.