Reading: Financing and Experimentation

Finally, it is important to emphasize that experimental learning in many areas has not only become cheaper, but also much better. Put differently, the same dollar invested in an experiment reveals more valuable information today than in the past.

For example, online giants such as Amazon can test potential new product features real-time with a selected group of live products and/or customers. Such experiments allow for almost instantaneous market feedback about the customer value added by a specific product or feature. A positive signal from the experiments will therefore greatly improve the success probability of a broader roll-out. In contrast, a negative signal will allow the firm to abandon the idea before wasting too many resources on it.

What makes experiments valuable?

The quality of the experiment is determined by the quality of the information it generates. Going back to the Amazon example above, the information gained from an experiment with real clients who buy real products with their own money presumably reveals much more about the success probability of the proposed feature than a similar experiment that is conducted with a set of students who are asked to buy a product that they would otherwise not consider with hypothetical money they receive for the experiment. 

• An experiment is good if it eliminates much of the initial uncertainty. In the extreme, it provides so much information that, following the experiment, there is no longer any uncertainty as to whether the product is successful or not.
   
• By the same token, a bad experiment reduces little or none of the initial uncertainty. In the extreme, the experiment demonstrates nothing so that the conditional success probability of the venture is the same after the experiment as before the experiment.
   
• Consequently, the value of the experiment can be measured by the difference in firm value when the experiment sends a positive signal and when it sends a negative signal.

Application

Our initial example also allow us to illustrate this important point fairly easily:

• The proposed experiment reduces a great deal of uncertainty: If it sends a positive signal, the conditional success probability increases to 10% compared to an unconditional success probability of 1%. Moreover, with a negative signal from the experiment, we know that the idea will certainly be a failure.
   
• The quality of the experiment is manifested by the relatively large spread between the resulting post-experiment valuations. As a result of the discriminating information generated by the experiment, firm value is 88 million if the experiment sends a positive signal and 0 if it sends a negative signal.

 An even better (worse) experiment would produce more (less) discriminating information and therefore lead to a wider (narrower) spread in the post-experiment valuations. The following two scenarios underline this statement.

Better experiment: More discriminating information

In the one extreme, uncertainty completely disappears after the experiment. After the positive signal, we know with certainty that the drug will be successful whereas with the negative signal, we know with equal certainty that it will be a failure. This (hypothetical) scenario is depicted in the following graph:

Clearly, this experiment generates the widest possible spread between the resulting valuations:

• Positive signal: With a guaranteed payoff of 1 billion, the NPV of the 12 million investment is 988 million (= -12 + 1'000).
   
• Negative signal: As before, firm value is zero.

Worse experiment: Less discriminating information

In the other extreme, the experiment produces no new information so that the conditional success probabilities after the experiment are the same as the unconditional success probability:

This experiment is obviously worthless, as it produces no new information about the success probability of the venture. As a consequence, the resulting post-experiment valuation are the same, namely -2 million.