Reverse inference: a bridge, not a barrier, between academic and commercial neuro research
Last week I had the pleasure of attending the 4th annual Interdisciplinary Symposium on Decision Neuroscience (ISDN), held at Stanford University. The event was jam packed with cutting edge decision research, presented by a mix of leading academics and commercial neuromarketing vendors. One participant, a psychologist from Cal Tech, told me the two-day event contained more commercial content than any conference he had ever attended. I told him it contained more academic content than any conference I had ever attended. We both reflected that we were meeting at our respective borders — without an event like this, our paths might never have crossed. I felt this represented a new kind of interdisciplinary mix, academic and commercial (aka basic and applied), that seemed to be both energizing and educational for all the attendees I talked to.
Reverse inference at the boundary of academic and commercial research
A highlight of the conference was an all-star panel on reverse inference, chaired by Baba Shiv of the Stanford Business School and including Antonio Rangel from Cal Tech, Brian Knutson from Stanford, Hilke Plassmann from INSEAD, and Carl Marci, CEO of neuromarketing firm Innerscope. I’m not going to recount the whole discussion here, other than to say it got rather heated at a couple of points as one academic aired some less than complimentary opinions about inference in commercial neuromarketing. But Carl did a superb job countering the jibes and describing both the unique challenges and some of the successes we have seen on the commercial/applied side. In the end, I felt the academic provocateur listened respectfully and pulled back a bit, acknowledging that maybe EVERYONE in neuromarketing wasn’t a charlatan or a snake oil salesman after all. From such small acorns do mighty oaks of progress grow.
It became clear early on in the discussion that a key difference between the two sides was in how they use inference. Academic neuro research, for the most part, relies on forward inference. For example, it tries to infer if mental process A (say, having a positive emotional response) is associated with physical state B (say, activity in a particular part of the brain). It tests this by inducing mental state A in a subject and then measuring if physical state B is present at the same time. But most applied research relies on reverse inference. It takes a finding like “mental state A is present when physical state B is present” and tries to infer the reverse: “if physical state B is present, then mental state A must be present as well.”
Some examples
For example, if academic research showed that a preference between two products (mental state A) could be predicted from the gaze patterns measured by eye tracking of images of those two products displayed next to each other (physical state B), then a commercial research company might build a business around measuring gaze patterns with eye tracking, and use that measure to predict comparative product preferences.
The problem for this mode of inference is that even if mental process A (a preference between two products) is accompanied by physical measure B (the unique gaze pattern) in every observed case, it does not follow that every time the gaze pattern is present, a preference must also be present. In other words, “if A, then B” does not logically imply “if B, then A,” because even if every A is followed by a B, it does not logically imply that every B is preceded by an A.
When applied research treats reverse inference as if were a valid logical inference, bad things can happen. A famous (or infamous) case in point was an op ed piece the New York Times by (former) neuromarketer Martin Lindstrom, in which he reported that people looking at pictures of an iPhone while being scanned in an fMRI machine showed a high level of activation in a part of the brain called the insula. Lindstrom observed that previous research had shown that when people looked at pictures of loved ones (inducing mental state A, a feeling of love), there was activation in their insula (physical state B). He then inferred, incorrectly, that because he had observed physical state B in his experiment (insula activation) when people looked at those pictures of iPhones, they must have been in love with their iPhones (mental state A). But what he missed was the fact that the insula gets activated by lots of mental states other than love, making his inference ridiculous, as was pointed out in a follow up letter to the New York Times authored by 45 distinguished neuroscientists!
From logical certainties to statistical probabilities
But most commercial neuromarketing vendors do not use reverse inference in this way. As our academic friends learned at the ISDN conference, commercial neuromarketers do not have the luxury of spending their time patiently building logically airtight explanatory theories. They are serving commercial research buyers who need to make decisions, and who want answers now, not later. These buyers want to know
given that I need to make a decision today, which choice is more likely to increase my probability of being right?
In that circumstance, the neuromarketing vendor is not making the incorrect reverse inference, “if B, then always A.” They are actually making a probabilistic reverse inference, “if B in this case, then if other conditions are adequately controlled, probably A preceded it.” And even if the “probably” in this inference is small, buyers will pay for that information, because even a small increase in the likelihood of making a correct decision, say from 50/50 to 55/45, is worth knowing, especially if millions of dollars are at stake.
Reverse inference: bridge or barrier?
So, how is reverse inference a bridge, not a barrier, between academic and commercial neuro research? I believe there are two fundamental ways reverse inference provides a bridge between basic and applied research.
Reverse inference relies on forward inference
First, the reverse inference used in applied research would not exist if corresponding forward inferences had not already been discovered and validated by basic research. Without a solid “if A, then B”, there is no basis for “if B, then probably A”. Some scientists have no interest in real-world applications of their work, but most do. There will always be some resistance to the fact that applying basic research findings to real-world problems can be monetized, often to the benefit of the applied researcher, not the basic researcher. But such resistance is understandable, and probably accounts for the large number of academics who now operate for-profit applied businesses on the side of their academic duties.
This is how reverse inference provides a flow of knowledge and insights from basic to applied research. Academic researchers of course would like to see some assurance that reverse inference is being properly implemented by applied researchers, that inferential howlers like the Lindstrom study are not being imposed on unsuspecting business clients, and that true probabilities are being accurately calculated and presented in results. Commercial researchers can meet these criteria by being open about the construct validity and reliability of their metrics, as well as the predictive accuracy and limits of their findings. Although determining construct validity is often seen as an academic exercise, I think commercial researchers should invest in it as part of their own R&D. This is very important to both the credibility and accuracy of commercial research results.
Reverse inference relies on a repository of prior knowledge
A second way reverse inference can provide a bridge between academic and commercial research has to do with investing in normative databases by commercial research vendors. In this case, the benefits can flow back to the academic side as well as bolster the credibility of the commercial side.
As we noted in Neuromarketing for Dummies (Chapter 20), any science-driven neuromarketing firm should invest in developing a normative database of its findings to improve the probabilistic interpretation of its results. The ability to apply a probability to a result is dependent on a repository of prior knowledge. To give an accurate probabilistic interpretations of “if B, then probably A”, a researcher must know how often, outside the current study, B is in fact associated with A. A normative database gives commercial vendors the ability to measure this statistic, and therefore to do Bayesian inference. It also allows them to present their results in a Bayesian framework, which is important, because the Bayesian model is very congenial to the way business people think about decision making.
Academic research, for the most part, seems to still be stuck in a significance-testing world, but this is slowly changing (see this often-cited article by neuroscientist Russell Poldrack on Bayesian reasoning in academic neuroscience). To the extent that commercial metrics are based on academic formulations, the normative data of commercial vendors can become a valuable resource for academics as well. As somone pointed out at the ISDN discussion of reverse inference, academics might test a metric a couple of times a year in carefully crafted experiments, but commercial researchers might be able to test the same metric 100 times in a month, and build a probabilistic distribution of outcomes that would take academics years to create. This is how reverse inference can provide a bridge back to academic research from commercial research. After all, since applied researchers are gobbling up all those forward inferences and turning them into money, it seems only fair they should give something back.
Other temptations when biting into the fruit of knowledge
Of course, because these two kinds of research operate under very different objectives, there will always be some tension between them. For commercial researchers, financial considerations (not just making a profit, but sometimes just managing to stay in business) make them susceptible to temptations that (we hope!) don’t impact academic researchers.
One temptation is the use of ad hoc hypotheses. Often, a corporate research buyer has a preferred result. If the research doesn’t produce it, the buyer is inclined to replace the researcher, not the preferred result. To avoid this very real possibility, commercial researchers are sorely tempted to pull out ad hoc hypotheses to “explain” negative results. Favorite ad hoc hypotheses include implicating sample composition, sample size, time of year, location, weather, you name it — whatever will “explain away” the result the client doesn’t like. This is bad science, of course. Although it may seem like good business in the short run, those ad hoc hypotheses will start contradicting each other in the long run. Eventually, the client will catch on.
A related temptation on the commercial side is cherry picking. This occurs when most of the study fails to support a preferred outcome, but the researchers find one or two results that do, and then emphasize those results over the larger picture. Again, this is bad science that may give the client what they want in the short run, but will likely fail in the long run.
Wrapping it up
It is exciting to see academic and commercial researchers mingling in relative harmony at events like the ISDN conference. May there be many more. Academic and commercial researchers may have more in common than either side realizes, and reverse inference may be an unrecognized bridge that connects them together. On the commercial side, there is a temptation to abuse reverse inference, but there are ways it can be policed and resisted, often through closer collaboration with those skeptical academics. Other temptations continue to make commercial neuro research a challenge, but there is every reason to believe that these will dissipate over time (as will the vendors who fail to resist them) as good applied science, based on solid findings and inferences, drives out bad.
Photo: “The Thinker” in the Rodin Garden at Stanford University. Source: artline.ro
About the Author: Steve Genco
Steve is a writer, speaker, researcher, and marketing consultant. He is author of Intuitive Marketing (2019), a study of persuasion and influence in marketing theory and practice, and co-author of Neuromarketing for Dummies (2013), a comprehensive overview of neuromarketing science, applications, methodologies, and ethics. He is Managing Partner at Intuitive Consumer Insights, where he focuses on marketing education and consulting. 
