FUQed

Here is a term I recently learned and that beautifully describes many of the problems and questions businesses constantly deal with: FUQ. It stands for "fundamentally unidentified question." In their book Mostly Harmless Econometrics, economists Joshua Angrist and Jörn-Steffen Pischke define FUQs as "research questions that cannot be answered by an experiment."

An example of a FUQed research question is "do children who start first grade at age 7 do better in school than children who start at age 6?": a question that has important implications in terms of education policies and parenting advice.

To answer this question, we can select a group of 7 year olds and a group of 6 year olds who both start school at the same time. To assess learning performance, we compare first grade test scores between these two groups.

The problem with this setup is that older kids tend to do better at tests because of maturity; it's likely that a 7 year old will perform better than a 6 year old simply because he or she is older. To deal with this problem, we can test the 7 year olds in first grade and the 6 year olds in second grade so that everyone gets tested at age 7. However, now the other group has spent more time in school. The test scores will tell us much more about the relationship between the number of school years and test scores than the relationship between start age and test scores.

In fact, the only way to answer this research question is to study the link between start age and different types of outcomes in adulthood (such as income). In other words, we need to have access to existing data that is able to provide an answer for us. We cannot use experiments to generate this data ourselves.

While FUQs have originated in the field of economics, they are not purely academic problems. Here are few FUQs that your company is probably dealing with:

• What questions should we ask candidates in job interviews?
• What effect will a marketing campaign have on our brand?
• Will a new incentive program improve our employee performance?

You cannot run a controlled experiment to find out the link between your interview questions and employee performance because you are not able to truly assess employee performance until years later. You can run experiments to test out the effectiveness of your marketing copy, but you can't run an experiment to see what effect marketing copy will have on your brand loyalty.

FUQs are frustrating when you want to figure out the best course of action in a given situation. However, the important thing to remember is that just because you are dealing with a FUQ and can't run an experiment to justify your decisions, it doesn't mean that the FUQ is not critical for your organization.

Additional notes

I first learned about FUQs from Tim Harford's book Adapt.