A lesson in probability and thinking from the casebook of Sherlock Holmes

This behavioural piece talks about mental model: Probability Thinking from the casebook of Sherlock Homes.

A lesson in probability and thinking from the casebook of Sherlock Holmes

There’s something comforting about a good detective story. As an aspiring Baker Street Irregular, there is no greater joy than the existence of something known as the ‘Holmes Moriarty Game’. There’s a lot to be learnt from Arthur Conan Doyle’s Sherlock Holmes, most prominently the ‘science of deduction’. But breaking down Holme’s approach to deducing the background and narratives of his clients and of crime scenes, brings us to a basic tenet of Probability Theory.

“When you have eliminated the impossible, whatever remains, however improbable, must be the truth.”The Holmes Moriarty Game emerges from the fascination of one Oskar Morgenstern (future academic partner of John von Neumann, with whom he would write a seminal book on the founding principles of Game Theory in 1944). As described by Michael Wainwright (2012) “Holme’s mysteries often amount to the logical puzzles of interpersonal relations known as ‘coordination problems’”.

In such a game, the players are making simultaneous decisions and the payoff is generally higher for the players if they make the same choice. However, this is a mixed strategy coordination game, with Holmes not desirous of making the same choice as Moriarty, as we shall see further.The case is taken from “The Final Problem”, one of only three stories of the entire oeuvre where Professor Moriarty makes an appearance. It is also the setting of the series’ denouement - the iconic death scene at the Reichenbach Falls. Upon learning of Moriarty’s murderous intent, Sherlock Holmes attempts to flee England and reach the safer shores of Continental Europe. To this end, he boards an express train from Victoria to Dover, with Canterbury as the only stop along the way.With the knowledge of Holme’s travel known to both Holmes and Moriarty, the question then becomes of which station offers Holmes a higher probability of survival upon alignment. While Watson breathes a sigh of relief upon getting Holmes on the train, believing his safety to be ensured - Holmes is more wary. According to Holmes, Professor Moriarty is “..a man of good birth and excellent education, endowed by nature with a phenomenal mathematical faculty.

At the age of twenty-one he wrote a treatise upon the Binomial Theorem, which has had a European vogue…He is the Napoleon of crime, Watson. He is the organizer of half that is evil and of nearly all that is undetected in this country"There are cognitive processes that also contribute to our inability to accurately assess probabilities. Heuristics are mental shortcuts, or rules of thumb, employed by people to forge the path of least resistance towards making a decision. This path of least resistance is developed as a result of the fact that we are fundamentally energy saving machines, as emphasised by Lisa Feldman Barrett’s work on the brain (Specifically Allostasis).

The most prominent work in heuristics has been done by Nobel Prize winners Daniel Kahneman and Amos Tversky, a lot of which has been covered in Kahneman’s book Thinking Fast and Slow. The book details two broad classifications of thinking employed by people - System 1 and System 2. While the former represents a faster, instinctive and emotional process, the latter speaks of a slow, more deliberate, analytical process. Previous pieces in this series have already highlighted the discrepancy of focus in this cognitive and emotional dichotomy. We plan and articulate a logical stream of thought, not always accounting for the human factor, labelling inexplicable decisions as ‘counterintuitive choices’ in hindsight. While there is a healthy space in Risk theory and literature for stochastic outcomes, it is not always incorporated in risk assessments, especially with respect to decision making in everyday life.The central issue is that of idiosyncratic overestimation or underestimation of risk. Three of the specific heuristics and biases that impair our ability to estimate the probability of risk are described in Judgement under Uncertainty: Heuristics and Biases (Tversky & Kahneman, 1975). A longer list of heuristics and biases can be found here, while a new periodic table method based classification can be found here.

Mental Model: Probabilistic Thinking

Probabilistic Thinking - a potential response to our default mode of energy conservation, is a wilful, concerted effort to move beyond the brain’s primary goal of survival to thriving in an increasingly competitive world. Learning to look at the world probabilistically involves deliberate System 2, data based evaluation of available information. “It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” The Sign of the Four (1890).One of the most critical ways to achieve this is the adoption of Bayesian Thinking in the evaluation of probability and risk with respect to decision making. Bayes Theorem has a wide ranging impact on innovations past (Alan Turing cracking the ENIGMA code) and future (much of Artificial Intelligence and Machine Learning) and fundamentally refers to the updating of one’s estimates and understanding based on new and relevant information, as it becomes available. Thus, the prior information that is routinely used to make estimates becomes more accurate, improving the chances of the updated estimate being accurate.

This article on probabilistic thinking identifies three kinds of people:

  • The first - who can understand how to calculate probability (Many of us fail at this stage itself)
  • The second - who can intuitively think probabilistically (Through practice)
  • The third - who knows and can take action on probabilistic outcomes (Even those who work and achieve a degree of fluency in the first two levels, could find it difficult to elevate to the third)

Even though much of this article has been dedicated to exploring the causes of our inability to assess probabilities and risk inaccurately, it isn’t intended to hamper your efforts to try and achieve the levels of mastery of Probabilistic Thinking detailed above. “There are only two things that determine how your life turns out: luck and the quality of your decisions. You have control over only one of those two things. When you make better-quality decisions, you increase the chances that good things will happen to you.”  Go forth and make better decisions.

This was written by a former junior fellow Divyani of Sanrachna Foundation.