1. Why Did a Team Put Humans Behind the Curtain Instead of Building the Machine?
In the story The Wizard of Oz, the mighty wizard turns out to be an ordinary person hiding behind a curtain. Product designers borrowed the picture. A "Wizard of Oz" test looks like a finished product from the outside, while a person does the work out of sight.
Harvard Business School's Working Knowledge site says Aardvark used a method inspired by this in its early days. The team had humans route users' questions, instead of computers, so it could see what people actually asked before building the technology.[1]
That is the short answer to this article's question. A real system costs time and money. If nobody wants it, all of that is wasted. A stand-in made by hand answers the important question much sooner: will people use this, and what do they want from it? This is one example, though. Some jobs can be done by a hidden person and some cannot, so the trick will not fit everything.
2. Why Is Building the Wrong Thing Such a Costly Mistake?
Tom Eisenmann, a professor at Harvard Business School, puts it this way: in his view, most startups fail not because they cannot build the product they set out to build, but because they build the wrong product, take too long doing it, and waste a lot of money. If the money runs out before the team finds the right product, the startup may fail.[1]
So the hard part is often not "How do we build it well?" but "Is this the right thing to build?" That is where small versions come in. The article describes a "minimum viable product," or MVP, as a product with just enough features to allow useful feedback from early users.[1] Notice what decides its size: not how polished it looks, but how much it can teach you.
The word "lean" in "lean startup" comes from the Toyota Production System, a way of making cars that tries to avoid waste. In a startup, the waste to avoid is time and money.[1] A hand-run service like Aardvark's points the same way, though that link is this site's reading, not something the article states.
3. What Happened When Dropbox Released a Bare-Bones Version?
Dropbox is a service for storing and sharing files online. According to the same article, its team first announced a bare-bones version on Hacker News, a website where people who follow technology share and discuss news, and collected a lot of immediate feedback from readers.[1] The article reports that Dropbox grew from 100,000 users to 4 million in 15 months.[1]
Divide 4,000,000 by 100,000 and you get 40, so the user base became about 40 times larger. Be careful, though. That number says nothing about how fast it grew month by month, and the article does not claim the whole increase came from the early version. What the example does show is that a small first release lets you start listening very early.
Schools have noticed too. In 2010, Harvard Business School announced an MVP Fund for student teams, aiming to give $5,000 to each of ten teams, with no award above $10,000, so students could build early test versions.[2] Trying an idea does not require a huge budget.
4. How Can You Fairly Choose Between Two Versions, A and B?
Suppose a shop's web page has a red button (version A) and a blue button (version B). Which one gets more clicks? Guessing is easy. Checking takes more care. An A/B test builds two versions, shows them to visitors at random, and uses statistics to see which one does better. The UK government's data community describes it this way: you split web traffic between versions and present each user with one version at random.[3]
Before the test, it advises writing a hypothesis, a prediction made before the experiment, in the form "If…, then…, because…". For example, "If we make the button red, then more people will click it, because it stands out." (That example is ours.) Writing the "because" part helps later: if the result surprises you, you can ask which of your assumptions was wrong. The same guide says to split the groups equally and randomly, to decide the main measurement before the test starts, and to run it for at least a full week as a starting point so that differences between days of the week do not fool you.[3]
Why random? The US National Institute of Standards and Technology (NIST) explains in its statistics handbook that in a randomized design the run order is decided by chance. Its example is drawing slips of paper from a box, so that no level is always tested first or last.[4] Chance makes the comparison fairer, because it is harder to accidentally give one version an easier time.
5. What Should a Team Do When Its Prediction Turns Out Wrong?
A wrong prediction is not the end of the story. Eisenmann describes pivoting simply as making a major change of some sort, and adds that doing it well is really hard.[1] A team that learned early can change direction before it has spent everything.
The idea has also moved into university research labs. The US National Science Foundation runs a program called I-Corps. The program lasts seven weeks. Experienced entrepreneurs teach researchers how to test the market through "customer discovery," meaning talking with potential users to learn what they actually need. NSF reported that, since the program began in 2011, more than 5,700 people had taken part and that more than 1,000 startups had launched because of I-Corps, in NSF's words, together raising $760 million or more in later funding.[5] Those figures are totals for everyone involved. They do not show how well any one team did, or how often teams succeed.
6. How Can You Run Your Own Small Experiment With Two Paper Planes?
You can practice the same loop at home: guess, test, look, adjust.
Set up. Fold two paper planes the same way from the same size of paper, but make the wings of plane A larger than those of plane B. Write your hypothesis first: "If the wings are bigger, then the plane will fly farther, because they catch more air." This is only a guess to test; wing size may matter less than you think, and a wrong prediction is useful information.
Make it fair. Use the same thrower and the same spot. Throw each plane five times. Write A or B on slips of paper, mix them up, and draw one before each throw to decide the order, the way the NIST handbook describes.[4] Measure each flight in your own steps.
Read the results. Find the average for A and for B, and also look at how spread out each plane's five throws were. If the averages are close and each plane's throws vary a lot, you cannot say which plane flies farther; ten throws can't separate a real difference from luck. Then change one thing, such as the angle of a fold, and try again.
Stay safe. Throw in a clear outdoor space, away from people, pets, windows, and breakable things, and never aim at anyone. If you ask a friend or a family member to take part, ask them first. A research guideline for working with children defines assent as a child's "affirmative agreement to participate," and says that merely not objecting should not be treated as assent. If a young child says no, respect that.[6]
To go deeper, read the original pages listed below: the UK guide on writing hypotheses, the NIST handbook on randomizing, and the Harvard article on lean startups. A simple first version, followed by a careful look at what happened, is a habit you can use well beyond paper planes.