Beware the Cult of Numeracy

As companies grow I see a tendency to focus more and more on data. While data can be a fantastic tool to understand behavior and measure impact, we should be mindful of its limitations. Many important things in life and business can’t be measured. A good related thought experiment:

Imagine living your personal life such that you never took (or were allowed to take) any action unless you could first quantify or measure its impact. Now you understand one of the pathologies affecting the tech industry.

How does something that seems so crazy in a personal context become the norm in the workplace? In short, we love tangible wins. This is especially true for high achievers. As Clay Christianson write in How Will You Measure Your Life:

High achievers want to optimize achievement. So they spend all their energy on near term tangible achievements. However, this can lead to negative long term consequences.

In his latest book he explores some of these long term consequences:

People who are driven to excel have this unconscious propensity to underinvest in their families and over-invest in their careers — even though intimate and loving relationships with their families are the most powerful and enduring source of happiness.

Bringing this back to business, the proliferation of data is putting more and more attention on short term and easy to measure metrics. In the short run this leads to stellar OKR updates. However, in the long run it often frustrates users, depletes trust, and reduces brand loyalty. This is especially true in competitive and fast changing environments where consumers flock towards simple and user friendly experiences.

Running 20 A/B tests to optimize button clicks may seems like a win. However, this hyper focused metric optimization attitude will more often than not degrade your experience over time and prevent you from the types of large scale innovations that lead to step function growth. As Evan Spiegel said in an interview with ReCode:

For me, I think the big red flag for all of us should be when we put more weight on things that can be counted instead of the things that can’t be. Because the things that can’t be counted are the things that make us human, and the things that are the most important to protect.

Data is not bad. In fact, data is a very useful tool to reduce uncertainty, validate hypotheses, and measure impact. However, we need to make sure we recognize its limitations. I love the way Andreas Eldh puts it in Human Driven Decisions:

Because data is sometimes great, we think it’s always great… [but N]umbers don’t show nuance, uncertainty or complexity. They give us an illusion of exactness. Data can tell us what is happening, but not why. Data can describe the problem, but not the solution.

The tail shouldn’t wag the dog. And data, especially near term metrics, shouldn’t be the main driver of our decisions. Rather, it should be one of several inputs alongside values, intuition, and empathy. Squarespace’s Optimize Towards Ideals core value does a good job of articulating this balance [1]:

While metrics are critical for tracking and testing the performance of our business, they are merely a reflection of our ideas and execution in the market. Our values and ideals are our decision making guide.

Consumer expectations are rising. We expect packages to be delivered in days and digital experiences to be fast, simple and easy to use. While we shouldn’t stop using data, we should acknowledge the unintended consequences of overemphasizing it.

So here’s to recognizing the limitations of data and bringing our attention back to the nuances and intangibles that make us human. Less focus on metric optimization. More focus on understanding customer needs and delivering value.

[1] Between writing and publishing this post they have evolved this value. You can check out the updated version here.

Originally published at on February 4, 2021.




product design @seatgeek | i like to build stuff |

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Powerful reporting with JIRA flow companion

Record Matching for Hotel Data

Data compost

5 Keys to a Modern Data Architecture

Dear Companies: Your Data Science Job Descriptions are Awful…

Course Review : Mathematics of Machine Learning Specialization

2 Useful Code Snippets for Aggregated data

1 Line of Python Code for Data Profiling

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adam Waxman

Adam Waxman

product design @seatgeek | i like to build stuff |

More from Medium

Which type of Product Manager do you want to be?

I finally found the product metric I’ve been looking for

As A Customer, See What I DO, Not Just What I Say.

4 Questions on the Adoption & Evolution of Product management