Quotes About Testing

pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.

For one thing, everyone would insist that assumptions be stated explicitly and tested rigorously not as a stalling tactic or a form of make-work but out of a genuine desire to discover the truth that underlies every project's vision.

Even if the amount of time that each process took was exactly the same, the small batch production approach still would be superior, and for even more counterintuitive reasons. For example, imagine that the letters didn't fit in the envelopes. With the large-batch approach, we wouldn't find that out until nearly the end. With small batches, we'd know almost immediately.

Instead of working in separate departments, engineers and designers would work together side by side on one feature at a time. Whenever that feature was ready to be tested with customers, they immediately would release a new version of the product, which would go live on our website for a relatively small number of people.

A pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.

The first step would be to break down the grand vision into its component parts. The two most important assumptions entrepreneurs make are what I call the value hypothesis and the growth hypothesis. The value hypothesis tests whether a product or service really delivers value to customers once they are using it.

Everything that has been discussed so far is a prelude to a seemingly simple question: are we making sufficient progress to believe that our original strategic hypothesis is correct, or do we need to make a major change? That change is called a pivot: a structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth.

Every business plan begins with a set of assumptions. It lays out a strategy that takes those assumptions as a given and proceeds to show how to achieve the company's vision. Because the assumptions haven't been proved to be true (they are assumptions, after all) and in fact are often erroneous, the goal of a startup's early efforts should be to test them as quickly as possible.

what it does is develop entrepreneurs, because when you have only one test, you don't have entrepreneurs, you have politicians, because you have to sell. Out of a hundred good ideas, you've got to sell your idea. So you build up a society of politicians and salespeople. When you have five hundred tests you're running, then everybody's ideas can run. And then you create entrepreneurs who run and learn and can retest and relearn as opposed to a society of politicians.

Compare two startups. The first company sets out with a clear baseline metric, a hypothesis about what will improve that metric, and a set of experiments designed to test that hypothesis. The second team sits around debating what would improve the product, implements several of those changes at once, and celebrates if there is any positive increase in any of the numbers. Which startup is more likely to be doing effective work and achieving lasting results?

The word pivot sometimes is used incorrectly as a synonym for change. A pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.

The solution to this dilemma is a commitment to iteration. You have to commit to a locked-in agreement—ahead of time—that no matter what comes of testing the MVP, you will not give up hope. Successful entrepreneurs do not give up at the first sign of trouble, nor do they persevere the plane right into the ground. Instead, they possess a unique combination of perseverance and flexibility. The MVP is just the first step on a journey of learning.

In a Wizard of Oz test, customers believe they are interacting with the actual product, but behind the scenes human beings are doing the work.

Split testing often uncovers surprising things. For example, many features that make the product better in the eyes of engineers and designers have no impact on customer behavior.

Fire that customer," I'd say to the person responsible for recruiting for our tests. "Find me someone in our target demographic." If the next customer was more positive, I would take it as confirmation that I was right in my targeting. If not, I'd fire another customer and try again.

When one is choosing among the many assumptions in a business plan, it makes sense to test the riskiest assumptions first. If you can't find a way to mitigate these risks toward the ideal that is required for a sustainable business, there is no point in testing the others.

We had gotten really good at optimizing, tuning, and iterating, but in the process we had lost sight of the purpose of those activities: testing a clear hypothesis in the service of the company's vision. Instead, we were chasing growth, revenue, and profits wherever we could find them.

Strategic planning takes months to complete; these experiments could begin immediately.

There is a flip side to this. In the Unix world, libraries which are delivered as libraries should come with exerciser programs.

retrospective studies are well suited for generating a hypothesis, then the hypothesis can be tested prospectively and supported, especially when independently replicated.

But today it is possible to find out through a blood test whether you are highly susceptible to breast and ovarian cancer, and then take action. . . . Life comes with many challenges. The ones that should not scare us are the ones we can take on and take control of." —ANGELINA JOLIE1

There's a joke about a man who tested his blade using his tongue: sharp blades taste like metal; really sharp blades taste like blood.

It is not when the cable lies coiled up on the deck that you know how strong or how weak it is; it is when it is put to the test.

When you're testing to see how deep water is, never use two feet.