Good vs. Bad Data-driven Business 📊
Good vs. Bad Data-driven Business 📊
🧩 The Wanderer Newsletter #07
Hello there!
I’m Nico, in The Wanderer newsletter I share some learnings & experiences lived along the way of building a tech company and meeting great people. It does not constitute in any manner universal truths, far from it, but more methods, ideas & thought-processes to broaden your own perspectives when approaching day-to-day challenges, help you advance on your goals and live a better life.
I’ve been quiet for a few months but I’m going to start writing again! 👋
If you are not subscribed yet, it's right here 👆.
If you like it, feel free to share on social networks.
Check out previous posts
“Be evidence-based and encourage others to be the same” - in Principles by Ray Dalio
Data is all around us in our business world. Every customer & worker action produces data. Customer data, prospect data, financial data, user data, product data, infrastructure data.
What topics should we use data for and when? How to combine an intuitive and a data-backed approach to make the most informed decision?
At Spendesk we embedded data intelligence and the scientific method in our revenue & product strategies. We always relied on data to come up with hypothesis and expected results, and put a lot of energy in measuring our actions towards our goals.
A good data culture helps the business make more informed decisions, improves the performance of both the product and the teams, driving overall success.
Today we cover:
What a Good vs. Bad data culture looks like
A background on what worked at Spendesk
📐 Good vs. Bad data culture
“In God we trust. The rest, bring me data”.
👉 A bad data culture excludes intuition, overwhelms meetings with too many indicators. A good data culture understands that intuition and reasoning work together. Intuitions are simply hypotheses, or convictions subconsciously created from a thousand small data points. A good data culture seek to complete a data-driven approach with qualitative interviews in the field to avoid false interpretation and jumping into conclusions too fast.
👉 In a good data culture, data is valued, used effectively, and integrated into the decision-making process at all levels of the organization. As data are at the core of the scientific method, which is fundamental to a performing and enduring business and product, good data culture expects every worker to embed data in their everyday decisions.
👉 A bad data culture focuses on lagging indicators. It focuses on describing what happened. This drives stress to the organisation, focusing on the end-point rather than the leading indicators, the intermediate data points that will impact the end result. Customer satisfaction measured in product is lagging, new bugs resolved by developers are leading. New revenue is lagging, new sales opportunities created are leading. Leading indicators help refine our craft.
👉 A good data-driven business knows data can help on a multitude of topics. Not only to help high-level strategic decisions but on small decisions as well. Data drives decisions in every function: Strategy with pricing, Marketing with positioning and campaign arbitrage, Sales with market segmentation & territory planning, People with recruiting the right profiles, Product with experimentation and product prioritization.
👉 A bad data culture thinks it’s about having the right tools only. Rather it’s about combining the right tools, with the right process and more important the right training to the team. Training on how to use data in the job with method: how to read it, visualize it, exploit it. Used like this data can empower every worker to be CEOs of their job.
👉 A good data culture is adjusted to the needs of the business. Moving from descriptive observations to diagnostic analysis to forward-looking scenarii. Ensuring the tools in place drive high quality data, in the right volume, making it accessible for the greatest number of workers: as a principle - a small amount of high quality data easily accessible is better than a high volume of untrustworthy data.
👉 Because data can help on such a broad range of topics, a bad data culture thinks it can solve everything. Beware of the common trap of paralysis by analysis. It happens when looking at data we cannot distinct the signal from the noise. Too much data kills insights & inhibit decision-making. As the uncertainty is the DNA of startups, a good data culture knows when to stop seeking for more and start deciding.
👀 Background - what worked at Spendesk
Our data-driven culture has been essential to Spendesk growth.
Across the ages when we were strong at propagating this data-culture we were seeing performance improve. On the contrary when we failed to train properly new generations of workers or diverged from our principles we were lagging behind. With hindsight below are some actions that paid off:
📊 Investing accordingly in the data function
We hired our first data person when we were just 15 people. At the time in 2017 it was un-common to invest so early in this function. But doing so enabled us to have someone whose mission was to equip the business with the right infrastructure and to hold the ‘voice of data’ during meetings. Alexandre Lenoir helped us propel our data culture forward especially educating the team on the importance of high quality data.
As needs increased we strengthened the team and in parallel educated the rest of the company. Hiring data engineers was critical as we were able to centralize and structure our data early on in a cloud data warehouse (snowflake in our case) to empower the rest of the team with high-quality and accessible data.
💪 Training the team
Our effort to educate our team around the usage of data in every job made a difference. For most of the jobs we:
clarified the lagging vs. leading indicators of performance in the job
set up proper definitions and aligned the team on what KPIs we should look at. We took the habit of reviewing them every 6-9 months as our understanding of our business or job improved and as people discovered new and more effective ways of piloting it.
implemented a routine to know when we should look at these KPIs. For example you can look at production data every week but when it comes to assessing the overall performance of a funnel, the quarterly rhythm is a better fit.
👀 Driving expectations in our culture around “Knowing your numbers”
I learned this mantra from King’s former COO, Stéphane Kurgan, and now partner at Index Ventures one of our investors.
During a working session with Stéphane he taught me the importance of knowing your numbers at all time.
Why? Because “knowing your numbers” enables more real-time collaboration with your peers. In a meeting when facing a problem, everyone around the table can bring precise enough numbers, sufficient to drive the collective intelligence towards a solution with an “orders of magnitude” reasoning approach.
This is also a forcing function for you to be engrained in the details of your operations. Knowing what performance you or your team had last week (being product delivery or sales) vs. this week, knowing if we’re getting better or worse.
You can achieve this by spending time in your dashboards to progressively develop a sense of the trends and the different correlations.
This is how talented business leaders can drive excellence in the day-to-day and be truly able to help their teams.
✅ The “so what” test
This simple question, dropped at the right moment during a conversation helps everyone to move from descriptive observations to diagnostic and forward looking actions.
Let’s take an example:
“Last quarter we saw a decline in our achievement vs. sales forecast from 98% to 80%” → this is descriptive. “Ok but… so what?”.
“We traced the root cause to an increase in the length of sales cycle, not anticipated, by 15 days from 20 days to 35 days in the [50;250] employees SMB segment which represented 40% of our sales forecast beginning of last quarter” → this is the diagnostic. “Ok… so what are we doing now?”.
“We took the following corrective actions: 1) reducing our share of pipeline from 40% to 20% through refocusing sales ressources on other segments 2) while we figure out through an iterative process how to be more efficient on this segment. As a consequence we will see more predictability in our sales forecast and are 90%+ confident to bring it back above 95%”. “It seems under control 💪!”.
👉 I summarize these different levels of analysis maturity below (diagram made on Excalidraw).
🤔 Confirmation bias & ‘paralysis by analysis’ traps
Here are my two favorite traps!
👉 Confirmation bias
Easy to fall into especially for data-savvy people. This happens when you’re looking for and filtering data to confirm your own preconceived thesis.
Ex: “The number of tickets managed by customer support team decreased by 30% this quarter as a testament to our increased code quality globally.” “Ok, but did you notice in parallel that the visits to our self-service help-center has increased by 50%? Don’t you think it’s just an effect of communicating vessels?”.
A way to prevent this is 1) to be aware of this bias and use our judgement to challenge ourselves 2) to use counter-KPIs. Secondary KPIs that we should maintain as we focus on improving the principal KPIs.
👉 Paralysis by analysis
When you start having a solid data infrastructure in place and a trained team you have at disposal numerous data points. This is when we should be wary of focusing on un-significant KPIs. The ones that even improved by 50% won’t move the needle.
In an early-stage startup you’re not looking for incremental improvements but to identify & double down on powerful trends. By looking into too many KPIs we can lose sight of the main ones but more importantly slow decision-making down. Instead of focusing on the north stars that will catapult our business we keep on adding analysis on top of analysis thinking this helps make a more informed decision.
What I do to ensure we keep on moving is:
1) to understand if a decision to take is a reversible or an irreversible decision - indeed often the best decision is the one taken right now
2) to force a time in the future to drive decision-making - this will be the optimal decision as we would decide with all the data available at this moment.
Ex: “let’s decide on this pricing change problem 3 days from now - in the meantime let’s gather intelligence to weigh pros/cons of each scenario”.
Hopefully this article will bring food for thoughts when it comes to your own data culture 👌.
“A good data culture pushes us to collect the right data. To immerse ourselves in data but then to decide with our heart.” - Jeff Bezos.
— Nico