Quotes About Metrics

I call the traditional numbers used to judge startups "vanity metrics," and innovation accounting requires us to avoid the temptation to use them.

companies using the sticky engine of growth track their attrition rate or churn rate very carefully. The churn rate is defined as the fraction of customers in any period who fail to remain engaged with the company's product.

Many of the techniques for doing this—actionable metrics, continuous deployment, and the overall Build-Measure-Learn feedback loop—necessarily cause teams to suboptimize for their individual functions.

In most cases, the answer was no; success was driven by decisions the team had made in the past. None of its current initiatives were having any impact. But this was obscured because the company's gross metrics were all "up and to the right.

This led to some frustrating board meetings at which we could show great product "progress" but not much in the way of business results.

Vanity metrics wreak havoc because they prey on a weakness of the human mind. In my experience, when the numbers go up, people think the improvement was caused by their actions, by whatever they were working on at the time.

There is an antidote to this misuse of data. First, make the reports as simple as possible so that everyone understands them. Remember the saying "Metrics are people, too." The easiest way to make reports comprehensible is to use tangible, concrete units. What is a website hit? Nobody is really sure, but everyone knows what a person visiting the website is: one can practically picture those people sitting at their computers.

Ask most entrepreneurs who have decided to pivot and they will tell you that they wish they had made the decision sooner. I believe there are three reasons why this happens. First, vanity metrics can allow entrepreneurs to form false conclusions and live in their own private reality. This is particularly damaging to the decision to pivot because it robs teams of the belief that it is necessary to change.

Compare two startups. The first company sets out with a clear baseline metric, a hypothesis about what will improve that metric, and a set of experiments designed to test that hypothesis. The second team sits around debating what would improve the product, implements several of those changes at once, and celebrates if there is any positive increase in any of the numbers. Which startup is more likely to be doing effective work and achieving lasting results?

cohort analysis,

Whenever possible, the innovation team should be cross-functional and have a clear team leader, like the Toyota shusa. It should be empowered to build, market, and deploy products or features in the sandbox without prior approval. It should be required to report on the success or failure of those efforts by using standard actionable metrics and innovation accounting.

We tracked the "funnel metrics" behaviors that were critical to our engine of growth: customer registration, the download of our application, trial, repeat usage, and purchase.

The viral coefficient measures how many new customers will use a product as a consequence of each new customer who signs up. Put

This is the kind of storytelling that takes place at most startup board meetings. Most milestones are built the same way: hit a certain product milestone, maybe talk to a few customers, and see if the numbers go up. Unfortunately, this is not a good indicator of whether a startup is making progress. How do we know that the changes we've made are related to the results we're seeing? More important, how do we know that we are drawing the right lessons from those changes?

Innovation accounting

I've come to believe that learning is the essential unit of progress for startups. The effort that is not absolutely necessary for learning what customers want can be eliminated. I call this validated learning because it is always demonstrated by positive improvements in the startup's core metrics.

The bad news was that the hockey stick went up to only about $8,000 per month of revenue. These numbers were so low that we'd often have investors ask us, "What are the units on these charts? Are those numbers in thousands?" We'd have to reply, "No, sir, those are in ones.

Brad explained to me how they hold themselves accountable for their new innovation efforts by measuring two things: the number of customers using products that didn't exist three years ago and the percentage of revenue coming from offerings that did not exist three years ago.

Everybody that has a measurement, whether it's in teaching or whether it's in your job, you're always worried how you will be measured.

I know a few CEOs who delegate the understanding of their financials and their business metrics to the CFO, and then stop worrying about all that 'numbers stuff.' Don't do that. You have to know your numbers inside and out - they are your life blood.

If you think about it, shouldn't we be spending more money against qualified prospective buyers versus shots in the dark at bagging a random stranger? Of course we should. It's a complete no-brainer. (p. 8) ... Our cardinal mistake is to forget that it is these four simple truths (or metrics) that keep us in business: 1. Getting more customers to buy from us; 2. More often; 3. To spend more with us in the process; 4. AND to recommend us to their friends (p. 15)

We all know that Americans love their statistics - in sport, obviously. And in finance too.

We'd all like to be in the business where we don't have to report our numbers, too. You're dealing with a Netflix and an Amazon that don't have to report their viewership. They're not sharing those numbers, so how do you work with a creative entity to renegotiate future seasons when nobody has metrics?

What gets measured gets done.