Systems of Human Judgement

John Maindonald - 19 August 2024

Kahneman’s book Thinking Fast and Slow (2013) is a good starting point for thinking about the strengths and limitations of human thinking processes. There is no good substitute for the use of “educating gossip”, as Kahneman describes it, for training in effective judgement and in decision making.

Important themes that Kahneman notes are:

• We have “an excessive confidence in what we think we know”.

• We too readily judge decisions by outcome, rather than by the strength of the arguments that support them.

• We have two selves — an experiencing self and a remembering self

• These do not always have the same interests.

• Automatic memory formation has its own rules

• This is exploited to improve the memory of a bad episode

• “We easily think associatively, … metaphorically, casually, but statistics requires thinking about many things at once”

System 1 and System 2

Humans have been conditioned to respond quickly to immediate risks and challenges, without stopping to consider too carefully whether what we heard was a false alarm. They also have the ability, when the occasion seems to demand it, to stop to ponder. This is the basis for Kahneman’s categorization of human thought processes as of two types — System 1 which jumps rapidly to make a judgement, and System 2 which takes time for careful consideration.

System 1 Features are:

• It may answer an easier question in place of a harder one.

• It responds to irrelevancies — priming, framing, affect, memory illusions, illusions of truth, …

• Priming by one stimulus can affect the response to a second stimulus that occurs shortly afterwards

• The portrayal of logically equivalent alternatives in different ways or ‘frames’ can affect the response

• The emotion or ‘affect’ generated by a question can affect the response

• It has little understanding of logic and statistics

• It cannot be turned off, but it can be trained

• When flummoxed, it calls on System 2

System 2 features are:

• It keeps you polite when angry, alert when driving in a severe rainstorm
• In its world, gorillas do not cross basket-ball courts …
• Problems that put 1 & 2 in conflict may require large mental effort & self-control to overcome the impulses and intuitions of System 1.
• Its effectiveness depends, in important areas, on training.

Both systems are amenable to training. A well-trained System 2 helps greatly in creating a better System 1. Further points are:

• Healthy living is a compromise

• Recognize situations where mistakes are likely

• Aim to avoid significant mistakes when the stakes are high

• Untrained humans are poor intuitive statisticians

• Judgements about statistical issues may require us to think about more than one, even many, things at once.

• Too often, we jump to conclusions, without careful assessment.

• We may not be equipped to make an informed and carefully thought through decision.

The Intuition of Professionals

Effective professional training is designed to ensure that at least some of the results of well-tuned System 2 expert judgement operate at a System 1 level. The professional will, if the training is doing its job, build up a repertoire of System 2 judgements that will later, when the circumstances seem to demand it, be available at a System 1 level.

The situation has provided a cue; this cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition

(Simon 1992, “What is an Explanation of Behavior?”)

Obstacles to effective judgement

Even those who are experts in their field can be similarly prone to judgements that have no foundation in fact. The following comment appeared in a discussion of the response to a U.S. Preventive Services assessment that prostate screening, when used in accordance with then current treatment practices, was doing more harm than good:

Even faced with … evidence … from a ten-year study of around 250,000 men that showed the test didn’t save lives, many activists and medical professionals are clamouring for men to continue receiving their annual PSA test.

New evidence emerges as time proceeds, and there are advances in the approach to treatment. At least part of the problem has been a rush to treatments that themselves risk increasing damage and the risk of death. Note the comment in Brawley (2018) that:

Over the past few years, the benefit‐to‐harm ratio has improved in favour of benefit if the man understands that active surveillance may be a reasonable path if diagnosed.

A demand for discipline & careful thought

• We make judgements based on evidence that is too limited

• We are easily fooled by irrelevancies

• Kahneman has brought together evidence on what & how.

• Even when data are there for the taking, someone has to notice, to collate the data, and to understand its uses and limitations

• Randomised controlled experiments (RCTs) are often the ideal, but require meticulous planning. If effects of interest are small, the numbers required may be very large.

• A limitation is that results apply only to the population from which trial participants were taken. Any wider generalisation has to be justified

• Observational data does not easily, if at all, substitute for the use of RCTs. It is in general impossible to be sure that all sources of bias have been properly accounted for

• Keep in mind Yule-Simpson “paradox”

• The paradox lies in the failure of human intuition to accommodate straightforward arithmetic!

Further examples

The “conjunction fallacy”

This has also, as a result of the example given in Tversky and Kahneman (1983) come to be known as “the Linda problem”. The name ‘Linda’ comes from the question and usual response that are given by way of example.

Linda is a 31-year old philosophy graduate, single, outspoken, and bright. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which of the following is more probable?

• Linda is a bank teller.
• Linda is a bank teller, and active in the feminist movement.

Adding the further descriptor “active in the feminist movement” can only lower the probability, or just possibly leave it unchanged. Instead of assessing the balance of probabilities, we are tempted to ask which description best meshes with what we have already been told about Linda.

“Linda is active in the feminist movement” is the single descriptor that respondents see as best fitting Linda. While that was not what was asked, one has to pay close attention to prevent System 1 from substituting that for the question that was asked. Note that the correct answer will be “a bank teller”, irrespective of the way that Linda was characterised before the question was asked.

In part, the issue is one of use of language. The “correct” answer is asking us to use the word “probable” in a strict technical sense.

Even careful critics sometimes get it wrong

An irony is that Kahneman was, as he has acknowledged, himself fooled into taking at face value papers that claimed to show that verbal concepts could have the effect of altering behaviour. Thus:

• Being asked to write down stories about unethical deeds made people more likely to want to buy soap;
• Subtly drawing attention to money, e.g., leave banknotes lying around, made people feel more self-sufficient, and care less about others;
• Priming people with old age related words leads people to walk more slowly away from the lab as research assistants armed with stopwatches timed their movements.

As Ritchie (2020) notes (p.28), Kahneman was not alone in being fooled — the study about priming with old age related words has been extensively cited in psychology textbooks. None of these claims have stood up in attempts at replication, with larger numbers and with greater care to avoid unconscious sources of bias. Thus, in the replication of the study relating to age-related words, infra-red beams were used to measure time taken to walk between two points in a hallway, rather than research assistants who knew the group to which participants had been assigned.