Reduce Expected Opportunity Loss with Build, Measure, and Learn

Information has economic value that correlates with its ability to reduce expected opportunity loss. Opportunity loss is the cost you will incur when you greenlight a doomed project, or the profit you will miss out on when you decide to pass on an opportunity. When you multiply this opportunity loss by the probability of it happening, you get the expected opportunity loss of a project.

Project cancellation is a high opportunity loss situation: we incur some costs while never receiving any of the benefits. Therefore, information concerning the possibility of project cancellation is the most valuable information for you as a decision maker. Since the opportunity loss is usually higher for lost benefits than estimated costs, the next most valuable information relates to validating the expected benefits. In other words, when it comes to calculating return on investment, we should focus more on the return and less on the investment.

In reality, cost estimation work gets more resources compared to work that focuses on forecasting benefits and the probability of project cancellation. This is partly caused by the fact that estimating costs with detailed technical specifications and spreadsheet software is easier and more straightforward work. But when it comes to estimating the risk of project cancellation and actual benefits, it can be hard for us to know where to start.

You need to get creative with the actual tools and practices for your analysis, but the build-measure-learn feedback loop can be one of the most valuable components of your work. First, you build and ship something small and concrete. After that, you measure the effect of the thing you built. Finally, you adjust your understanding based on what you learned.

Remember that there are two variables that make up the expected opportunity loss: the cost of being wrong and the chance of being wrong. In the context of benefits and expected opportunity loss, market analysis is required for calculating the cost of being wrong. If you're estimating the possible revenue for a new product, you count the number of potential clients and their annual spending in your product category. If your analysis is for an internal automation project, you count the number of possible work hours saved.

But none of this analysis tells you how many of the potential clients will actually buy your product, or how usable and efficient your new internal software actually is in terms of automating some of the manual processes. The chance of being wrong is either the same or reduced only slightly.

Compare this to doing initial sales before building the product or building something small and ugly, but still useful, for a subset of your employees. You won't be much wiser when it comes to estimating the cost of being wrong. But you are able to have an effect on the chances of being wrong.

Since you're building things, it might feel like you're already doing product development. However, it's not product development if the project hasn't been greenlighted. It's research and analysis that yields returns higher than the work your organization uses to calcluate the cost of different initiatives.

Build-measure-learn feedback loop is not the only way to decrease the chance of being wrong. Because of real life work experience, some of us are better at forecasting and our chance of being wrong with a specific opportunity can be lower by default. We might have been observing market behavior in sales (or in providing support for sales) or lived through other, similar projects in the past. There is also design thinking that takes a more strategic approach for reducing the likelihood of being wrong about an opportunity.

We know the build-measure-learn feedback loop mainly because of the lean startup methodology. But the loop is not just for startups or established companies who want to add a Silicon Valley vibe to their office. Build-measure-learn feedback loop is not startup culture. It's rational economic behavior.