Pierre Jolicoeur, Professor of Psychology, University of Montreal
Interviewed by Glen Drummond
Glen: What is the current set of research tools for measuring experience?
Pierre: The first would be reaction time methods. Present stimuli to people, ask them to make speeded responses, and by clever use of different experimental conditions, we can actually infer quite a bit in terms of the underlying mental processing ability to see things. I would call that classical psychophysics or classical cognizant physiology where the main dependent measures are reaction time and patterns of error rates.
Glen: This is the sort of stuff that’s gone into the interface design for flight equipment and so on to make sure that pilots get the right information at the right time and don’t crash their planes and so on.
Pierre: Exactly. But because a subject may have moved his or her eyes on top of a stimulus doesn’t guarantee he’s actually processed that stimulus very deeply. So, you also probe their memory to determine if they can remember it later.
Then we look at the measurements of their physiological responses, such as Galvanic skin response (GSR), heart rate, pupil diameter, and measuring the strength and speed of various kinds of reflexes. These have been used in the past, and are still in use today.
Techniques I know more about would be used in core cognitive neuroscience today, such as functional magnetic resonance imaging (fMRI), which looks at the differences in blood flow. The basic principle is that, in the areas of the brain where the neurons are working harder, they require more energy from glucose in the bloodstream, and the cells metabolize more and produce changes in blood flow and blood oxygen level. This allows us to tie the activity to local structure in the brain; that’s where we’re both understanding which parts of the brain do what, when. This technique has major limitations. One, it is very expensive, with a scanner costing up to $7 million and an hour of the procedure costing up to $3,000. Two, it requires that the subject be very still for quite a long time, perhaps an hour.
Glen: Now, there’s a body of thinking that has emerged more recently that claims that we respond at an emotional, precognitive, or pre-rational level to stimuli from advertising and marketing and that we rationalize our impulse after the fact. Do you believe that this kind of inference is one that can be made based on the use of this kind of technique?
Pierre: Well, there’s both, I guess, because if I’m watching a TV commercial, and it’s cut in various ways, the information is flying by really quickly. It either grabs me or doesn’t grab me. I may or may not see something or remember something later on based on my current kind of state of readiness to receive that message. That would be one message that I would convey that is actually closely tied to my own research that we process information, sometimes surprisingly conditional, on our current goals and motivation, even at a very microscopic level.
Glen: We absolutely have evidence that says when we do eye tracking followed by an interview, we see that people looked at something that their interview didn’t reveal.
Pierre: That’s correct, but we can’t discount the eye tracker information completely. For example, if you’re looking in a cluttered visual display for a particular piece of information that you know is red and is supposed to be at the center of the screen, then it turns out that other red things will draw your attention involuntarily, in the periphery.
fMRI has one major limitation from a scientific point of view and that is that the response that you measure, the change in blood flow, actually takes quite a while – 4 or 5 seconds – to take place. When your neurons start to work, their activity will ramp up very quickly, with about a tenth of a second, and we can do a very simple task like judge the pitch of a tone or the color of a visual stimulus in well under a second. But this hemodynamic blood flow response will take another 4 or 5 seconds before it peaks and another 5, 6, 7 seconds before it comes back down to baseline. So it is very difficult to determine the order in which the critical cognitive and perceptual events have taken place.
Therefore, you need to use a variety of techniques. One which is gaining in popularity is EEG, in which we put electrodes on the surface of the scalp to measure the electrical activity of the brain underneath. This technique has existed since 1929 or so, but really emerged in the ’60s and ’70s when people started to discover brain responses and manifestations of that in revoked responses. Now, with faster computers, we can do much more high-quality, high-density recordings of EEG. The big advantage of EEG is that the electrical response occurs at the scalp essentially instantaneously, relative to when the cells start to respond, because electrical signals travel at the speed of light. It’s an immediate measure of what’s happening now.
Glen: Are you able with EEG to locate what area of the brain is making that activity as well as when it’s happening?
Pierre: No, not very well. That’s the drawback of the EEG. I can tell roughly, but with only some degree of precision. Localization of EEG requires a kind of mathematical model, some inverse solution, which is controversial at this point.
Glen: So with our current set of technical apparatus for studying what’s going on in the brain, we have a kind of uncertainty principal. We can find out with fMRI where it’s happening, and we can find out with EEG when it’s happening; the problem is we can’t find out where it’s happening when.
Pierre: We can try, and that brings me to a third technology called MEG, which is magnetoencephalography where we measure magnetic fields produced by the brain. That helps a little bit over EEG because one of the difficulties with EEG is that the signals are distorted by the skull itself, the bone, as a really poor conductor; and the electrical fields get distorted. It turns out bone is essentially transparent for magnetic fields, so they just pass through it without distortion and so that helps. We still have this inverse problem issue.
So yes, your intuition is exactly right that now what we’re going to do is we’re going to combine all these methods and sometimes even use them simultaneously. You can’t use MEG at the same time as fMRI, but we can do EEG at the same time as fMRI and get the initial electrical response and then follow that up with the blood flow response and try to put it all together.
Glen: So, from a marketing discipline standpoint, we ask: Did they see it or not? Did they process it or not? There’s another question: How meaningful was that, how involving, how deeply moving was that particular idea or experience? Do the quantitative sciences bring anything to this?
Pierre: I think they can, but there’s a catch. For both fMRI and EEG, we need to repeat the stimuli over and over because the signal that we’re measuring is relatively small, relative to the noise. So we need to repeat the basic stimulation sequence fifty times, a hundred times, and then we average over those presentations to get a reliable signal. We can modulate it, so we can make it easier or harder to process that stimulus; and then our brain wave response is going to follow, and it will track nicely. We can even tell you whether or not you were successful in coding the semantic content of something. The way we do that is it turns out there’s a special brain wave that appears to be generated only when there’s a mismatch between a semantic context and a currently processed stimulus. That actually gives us a powerful tool to measure when a person has been able to extract a semantic content of the stimuli, but the caveat is that I can’t do that with one trial. I can’t show you one ad and measure this brain wave; I have to show you the stimulus thirty, forty, fifty times.
Glen: As a closing question, what are the natural ranges of the way brains work? Do people process information the same way, or are there really quite significant differences from individual to individual in the way they respond?
Pierre: There are two different layers of response. When we look at early brain responses to auditory, tactile, and visual stimuli, it is uncanny how closely aligned our brain responses are. Then, when I look at brain waves from two people, they actually look pretty similar, as well, so I can average them across people. Every now and then we find an oddball person that has different brain wave; we have some good hypotheses about why, but the basic response looks really similar from person to person.
But the intentions of the observer modulate early brain responses – going back to the red target scenario. You are expecting a red target at the center, and I flash a big green, bright green spot of light, just an inch and a half to the right or left of where you are looking, and it doesn’t do anything. You’ll get a brain response to it, but it won’t capture your attention. If I present the same stimulus, but in red, bang, your attention is drawn over. And if you weren’t paying attention to it, you’re not going to remember it. You’re not going to process it in the same way. If there was an emotional response attached to it, you’re not going to get it or you will, depending on your current state. So I think yes, I think we rapidly diverge; and our intentions and emotional states do modulate what we perceive and remember and process.
Glen: Which, in some respect, brings us from the quantitative back over to the qualitative side of things. If you understand the motivations of the people who are going to be exposed to your content, you can begin to predict the full trajectory of their attention to that. If you don’t understand those intentions and those motivations, you’re likely to miss.
Pierre: Exactly.
Pierre Jolicoeur
Professor of Psychology, University of MontrealCanada Research Chair in Experimental Cognitive Science
Email: pierre.jolicoeur@umontreal.ca
Pierre is a Professor of Psychology at the University of Montreal, as well as an adjunct Professor at the University of Waterloo. As the Canada Research Chair in Experimental Cognitive Science, Pierre’s research involves the study of human attention (the process of selecting one source of stimulation from among many) and its relation to perception and thinking, seeking to determine how, why, and when attention succeeds or fails. Pierre has pursued research in this field for over 25 years after obtaining his PhD in experimental psychology (Harvard, 1982).
Comments
Leave a Comment