The Tortoise and the Hare

In his podcast Revisionist History, Malcolm Gladwell does two episodes inspired by William Henderson's paper The Lsat, Law School Exams and Meritocracy: The Surprising and Undertheorized Role of Test-Taking Speed. Those two episodes opened my eyes to how we constantly judge people on false merit and how our society often has a bias toward people with good test-taking speed. It's a great listen; you should check it out.

Henderson and Gladwell focus on the world of law. But in this post I'm going to discuss their idea from the perspective of programming.

Who's the better chess player?

Here is Gladwell's argument in a nutshell: We constantly measure people's reasoning ability with time pressured tests. We use those tests to decide for example who is accepted to high-ranking universities. However, science says that test-taking speed does not correlate with reasoning ability. Time pressured tests measure the speed of our reasoning ability, not the actual quality of it.

Gladwell divides people into two categories based on test-taking speed: tortoises and hares. Tortoises need their time to figure out solutions and answers while hares are fast thinkers. And just like in the fable of the tortoise and the hare, tortoises triumph over hares when the game is not about speed. For example, Henderson found out that law students who don't do so great in time constrained tests, are still able to get solid A's from take-home exams and papers.

You can see this same pattern in the world of chess. Chess games have a time limit. This is so that the games wouldn't go on forever and end up in ties. However, the time limits are not always the same: Classical chess games start from 90 minutes per side for the first 40 moves. In blitz chess, each player gets 10 minutes or less for all their moves.

Chess player Fabiano Caruana is ranked 2nd in the world. His peer Hikaru Nakamura is eleven ranks below him at 13th. However, these rankings are based on classical chess games. In blitz, Caruana wouldn't stand a chance against Nakamura. Nakamura dominates in the world of speed chess. Caruana's strength is his calculation skills while Nakamura's strength is his pattern recognition skills. When you decrease the play time, pattern recognition skills give you a greater advantage as a player.

So who is actually the better chess player? Nakamura or Caruana? We have the official ranking to answer this question. But if the organization in charge of the ranking would decide to decrease play time dramatically, the ranking would look something completely different. Great chess players are great in different ways. The official ranking is just one side of the story.

Why is someone a great programmer?

Think about your colleagues' and your own personal strengths and weaknesses. Think about Nakamura and Caruana. Think about the fact that test-taking speed doesn't correlate with reasoning ability. Are you able to agree with the following statements:

• Great developers have great reasoning ability
• However, your learning speed doesn't correlate with your reasoning ability
• ...your delivery speed doesn't correlate with your reasoning ability
• ...your ability to succeed in whiteboard interviews doesn't correlate with your reasoning ability

Even if you agree with those statements, you probably also make conclusions like these (consciously or unconsciously):

• Because person A learned C++ in a month, she's a better programmer than person B who spent 6 months learning the language
• Because person A delivers features faster than person B, she's the better programmer in our team
• Because person A nailed our whiteboard interview, she's the best candidate and therefore the best programmer we can hire

It's likely that inside your team and your organization, there is a bias toward tortoises or hares. It can be that because of your interview process, you end up hiring more hares or tortoises. Because of the way delivered work gets highlighted and successes celebrated inside your organization, you might be promoting either more hares or tortoises.

It's also possible that there isn't anything wrong with your biases. If you need hares, by all means hire hares. If you need tortoises, hire tortoises.

But what if you haven't thought about whether the work you have set out to do is meant for hares or tortoises. What is the version of the fable people tell themselves inside your organization? In the minds of people, is the winner the hare or the tortoise? And if you look at the facts objectively, does their version of the story match with reality?

My interview story

A little more than a year ago, I was interviewing for a job. I was asked to take a couple of online tests to measure my reasoning abilities. The tests had such strict time constraints that most people weren't able to finish all the questions in time. You had to mark your answers fast and not second-guess anything.

I took the tests and I scored fairly well (not amazing but also not bad). Based on the results, I thought that the tests suggested that I'm smart. But in fact, all I had managed to prove anyone is that I have pretty good test-taking speed. I, and the interviewers, falsely thought that the tests said anything truly meaningful about my reasoning ability.

By the way, just so you don't think I'm trying to do a humblebrag with this story, let me drill my point into you by quoting Henderson's paper: "it is widely acknowledged that test-taking speed and reasoning ability are separate abilities with little or no correlation to each other." Just because you're fast, doesn't mean you're smart, and vice versa.

I actually don't know if I would have been hired. I dropped out of the interview process. I figured that the job required much more contemplation and deep reflection that I had the patience for. It was work meant for a tortoise and I was a hare.

Sadly, the company had forgotten the lesson of the fable. They were betting on the hares of the world.