Fifty years of J. R. Platt's strong inference

In 1964, John R. Platt (1918–1992) published a paper in Science with the intention of stimulating debate about how science works and how we might do it better (Platt, 1964). The paper was so provocative and got so much attention that Science republished it in 1965. Since then, Platt's ‘strong inference’ continues to circulate among science departments and inspire young scientists. When I first encountered it as a graduate student, it had a profound effect on my development as a scientist, and it is one of the few papers that I now give to every new member of my lab.

The article opens with an irresistible line: ‘Scientists these days tend to keep up a polite fiction that all science is equal.’ Platt goes on to argue that all science is not equal, that some scientists do it better than others, and even that some fields (such as molecular biology and high-energy physics) do it better than others. The difference between good science and bad science, argues Platt, has to do with how systematically one employs the hypothetico-deductive method, which he calls simply ‘strong inference’.

Platt's message is inspiring to new scientists because he asserts that doing great science is a skill that can be learned, and, contrary to popular mythology, one that does not require superhuman intellectual gifts. He tells us that we can make profound breakthroughs if we simply cultivate the habits of mind of Watson and Crick, and Jacob and Monod, who systematically applied strong inference thinking and solved some of the most difficult problems of their time.

Platt characterizes strong inference as the repetition of three essential steps: (1) devise alternative hypotheses; (2) devise a crucial experiment that will exclude one or more hypotheses; and (3) perform the experiment and obtain a clean result. Then, (1′) recycle the procedure to refine the possibilities that remain.

When I first read Platt's strong inference paper as a graduate student, I was familiar with the hypothetico-deductive method, but Platt was the first to reveal to me how it really worked, and I suspect this is true for many of his readers. Platt explained that strong inference can only work if we assume that the universe is a rational place, that even the most confounding questions have an answer, and that the answer must be one of only a handful of reasonable explanations. These insights are the key to understanding how the disproof of hypotheses can lead to the construction of meaningful knowledge, which is something I had not fully grasped before I read Platt's paper. When this light bulb turned on for me, I was enthralled with this new vision of the scientist as Sherlock Holmes, who relied on just this kind of bold strong inference approach. In Holmes' words, ‘when you have eliminated the impossible, whatever remains, however improbable, must be the truth’ (Doyle, 1890).

Platt recognized that it was all too easy to make science complicated and to lose focus of its real purpose, which is to find better and better explanations of the world around us. He therefore provided his readers with several practical aids to help them successfully apply strong inference. One of these was the concept of multiple working hypotheses, an idea championed by the geologist T. C. Chamberlin, a predecessor of Platt's at the University of Chicago. Chamberlin argued in an 1897 paper that a scientist working on a problem should strive to entertain several hypotheses at once (Chamberlin, 1897). Such a measure is necessary, according to Chamberlin, because of our tendency to become attached to our ideas, which can lead to science becoming an irrational argument among scientists, rather than a rational competition among ideas. Chamberlin's vision of science, like Platt's, is one grounded in a firm belief in a knowable reality, and one in which good explanations rise out of the ashes of those shown to be false.

Platt also espouses the systematic use of a laboratory notebook. He regales us with tales of Fermi, Faraday, Roentgen and Pasteur, who all wrote obsessively in a ‘diary’ or lab notebook, and who all made revolutionary discoveries. While students are typically encouraged to keep a lab notebook as a form of record keeping, Platt asserted that it should be used for much more – as a staging ground for ideas, a place to record puzzling observations, ask questions, dream up hypotheses, derive predictions and critically analyze. In his book The Excitement of Science, he urges every person (not just scientists!) to spend at least one ‘gamesworth’ of analytical reasoning each day, which he defines as the amount of effort required to complete a game of chess or a challenging crossword puzzle (Platt, 1962). I took Platt's advice as a graduate student and was amazed to see how even difficult problems can yield to the cumulative effects of this approach.

Platt's paper transformed the way that I thought about my own research and science in general, and it eventually changed how I teach. I was eager to pass on the insights I had learned from Platt, but soon realized that students can find his algorithm of three steps to be quite daunting. For this reason, I find it helpful to expand the strong inference process into smaller, more manageable tasks. This expanded strong inference algorithm, along with the cognitive skills required for each step, is presented in Fig. 1. Each step is explained in more detail below.

Observe. All science starts with observation, but only those observations that are puzzling to us have any value, because they are the ones that signal to us that our current explanation might not be correct. Skilled observation requires sensitivity to our environment and an ability to recognize when something is confusing and therefore worth pursuing.

Question. Puzzling observations organically lead us to ask non-trivial questions, which are generally ones that begin with ‘why’ or ‘how’. Many students harbor the misconception that scientific questions must take the form, ‘What is the effect of x (independent variable) on y (dependent variable)?’ In fact, answering these kinds of questions typically does not require the scientific method, because it is immediately obvious how to answer them (vary x, measure y). In contrast, it is rarely evident at first how to go about answering ‘how’ and ‘why’ questions and this is where strong inference can provide us with a roadmap.

Hypothesize. Platt suggests that we enter difficult problems by devising alternative hypotheses, and my students are shocked to learn that coming up with hypotheses requires nothing less than creativity. When students ask me where good hypotheses come from, I tell them, ‘the same place good poetry comes from’, which is of course a maddening way of saying ‘I don't know.’ Because so many students possess the misconception that a hypothesis is simply an ‘educated guess’ about how an experiment will turn out (i.e. a prediction), I find it useful to apply a simple test to check whether a hypothesis is a good one.

The test of a good hypothesis is to ask whether it represents a satisfying answer to the question that has been posed. If it does not, then it is not worth pursuing. A hypothesis may pass this acid test, but get discarded later because we realize it violates a law of physics or is internally inconsistent. With several cycles of creativity followed by criticism, we can whittle down our list of hypotheses to a handful of reasonable explanations. If we assume that one hypothesis is correct and the others are wrong, this is where the real fun begins, because it is at this stage that the whole enterprise starts to feel like a detective story.

Predict. Platt tells us next to devise a crucial test, and students are always eager to do this, but I find it useful to insert one more step before the experiments are designed, and that is to list the predictions that each hypothesis makes. I constantly need to remind students that these are not predictions that you make, these are predictions that each hypothesis makes (Hutto, 2012). When we predict instead of letting the hypothesis predict, we lose the tight connection between hypothesis and experiment, and the logical structure of the entire process can fall apart. Finding predictions requires large doses of imagination, because we must try a hypothesis on for size and conjure up how the world would look if it were true. Once we have a list of predictions from each hypothesis, it is important to confirm that they are critical predictions.

We can evaluate a prediction's utility by asking ourselves whether the hypothesis can survive if the prediction is found to be false. If it can, then it is not a strong prediction, and probably not worth testing. Focusing on tests with the greatest potential to disprove our hypotheses is important, because it is the fastest way to eliminate faulty explanations that might otherwise stand in our way of reaching the truth.

Test. The testing step, sometimes called the experiment step, is when we evaluate whether a prediction is true by comparing it with some aspect of the real world. Much has been written about the ins and outs of experimental design, because there are lots of places where one can go wrong. Platt deliberately says little about this in his paper, because his intention was to illuminate those steps of the scientific method that he felt were being ignored. The test of a good experiment or test is to ask whether the results, whichever way they turn out, will allow you to evaluate how good a given prediction is.

Analyze and conclude. The last step is to analyze and conclude, and if all the other steps have been carried out properly, this should be easy, and we should find ourselves closer to an answer to our question. If we have neglected certain parts, the logical bones of our structure might not be sound, and we are at risk of making an erroneous conclusion. Of course the process is not a linear one, and data collected during the testing stage may (and very often do!) become new puzzling observations of their own, which can lead to interesting questions and entirely new lines of inquiry.

Platt was not a philosopher of science in the academic sense, and his message was not anything that Karl Popper hadn't said in his extensive writings about scientific discovery (Popper, 1959). Yet Platt's small paper has had a large impact on a wide audience. The ISI Web of Science lists 1320 citations for strong inference, and a perusal of these citations reveals influence in an impressively diverse number of fields, with a large number of citations coming from psychology, ecology, and marketing and business science. Some might wonder whether Platt's message is relevant in our current age of ‘big science’. Surely strong inference was not needed to sequence the human genome, nor will it be useful for mapping all the neural connections in the human brain or barcoding and classifying every species of life on Earth. However, once we have these data, puzzling patterns and compelling questions are certain to emerge, and this is where strong inference can help lead us from confusion to satisfying explanations.

Scientists appreciate Platt's paper because of its no-nonsense tone and the practical advice he provides, but also because he gestures toward something important about the psychology required to do good science. Although he doesn't make the point explicitly, Platt suggests that making progress requires walking a razor's edge between audacity and humility, and applying each in the right place at the right time. It is a message that can benefit anyone who is interested in tackling difficult problems – we must be bold enough to assume that one of our ideas is correct, and yet we must have the humility to abandon those ideas that don't stand up to scrutiny.