SocraticGadfly: FBI criminal profiling – little more than psychics’ “cold reading”

November 17, 2007

FBI criminal profiling – little more than psychics’ “cold reading”

Why the FBI needs even more reform than just being dragged into the Computer Era

Cold reading is what pseudo-telepathic frauds like James van Praagh and John Edward use to make gullible people believe that they can actually read their minds. It’s obviously unscientific. A practitioner makes vague, open-ended statements to fish for information. With the exception of fishing for information, your newspaper horoscope is the same thing, of course.

Well, Skeptic’s Dictionary author Bob Carroll, following up on a New Yorker article by psychosocial insight guru Malcolm Gladwell and an actual statistical survey (PDF) by the University of Liverpool, argues that FBI criminal profiling is little more than bogus cold reading.

Carroll notes about the Liverpool study:
First, the psychologists argue that profiling won't work the way the F.B.I. does it. (F.B.I. profiling assumes a stable relationship between configurations of offense behaviors and background characteristics, which is not supported by the research evidence.) Second, they note that the F.B.I. claims a high degree of accuracy for the method that supposedly shouldn't work. Then, they explain the illusion of accuracy as due to subjective validation.

And then, about the actual FBI profiling study Liverpool analyzed:
It also turns out that it shouldn't be surprising that the profile is bogus. It wasn't based on a representative sample. According to Gladwell, the F.B.I. profilers who came up with the serial killer profile, John Douglas and Robert Ressler, chatted only with convicts who were in prison in California. Furthermore, they had no standardized protocol for interviewing their subjects.

The FBI had been operating under the premise that serial killers fall into two types. Those who preplan their individual killings, based on victim age, race, sex, etc., for some particular psychological fix, and those who kill at random. They then assumed that each type of serial killer had a profile based on a different personality type.

Well, profiles were somewhat off in many cases, and egregiously off in many others. Gladwell says that in Britain, the Home Office studies 184 criminal cases which had profilers involved, and the success rate was 2.7 percent.

More below the fold (pretty long):

The problem is even worse than that, Gladwell points out. Ultimately, the FBI method of developing details that are supposed to belong to a certain type of profile, such as one type of serial killer versus the other, is unscientific:
(FBI agents) Douglas and Ressler didn’t interview a representative sample of serial killers to come up with their typology. They talked to whoever happened to be in the neighborhood. Nor did they interview their subjects according to a standardized protocol. They just sat down and chatted, which isn’t a particularly firm foundation for a psychological system. So you might wonder whether serial killers can really be categorized by their level of organization.

The Liverpool study went back and analyzed a number of specific killings committed by serial killers. They started with the idea that traits that fit in the profile or organized killer, or disorganized killer, would “interlock” with one another.

Not true. Most the crimes had specific factors that were a mix of both profile types.

And, here’s exactly how it’s like cold reading:
A few years ago, Laurence Alison, one of the leaders of the Liverpool group and the author of “The Forensic Psychologist’s Casebook,” went back to the case of the teacher who was murdered on the roof of her building in the Bronx. He wanted to know why, if the F.B.I.’s approach to criminal profiling was based on such simplistic psychology, it continues to have such a sterling reputation. The answer, he suspected, lay in the way the profiles were written, and, sure enough, when he broke down the rooftop-killer analysis, sentence by sentence, he found that it was so full of unverifiable and contradictory and ambiguous language that it could support virtually any interpretation.

Gladwell begins his article by noting that profiling has many of its roots in Freudian psychiatry, which means that, beyond being cold reading fishing expeditions, actual profiling work-ups are also often wrong in the same way that Freudian psychiatry is.

We of course have seen FBI profiling go tragically wrong three notable times in recent years, first in falsely implicating Richard Jewell as the Atlanta Olympics bomber. the failure to consider blacks as sniper-type serial killers, as was disproved by John Allan Muhammad and Lee Boyd Malvo, and finally, getting the Wichita, Kan. BTK serial killer incredibly misprofiled while he remained at large for decades. Note this FBI profiling of BTK, vs. the reality, from 1984:
The best minds in the F.B.I. had given the Wichita detectives a blueprint for their investigation. Look for an American male with a possible connection to the military. His I.Q. will be above 105. He will like to masturbate, and will be aloof and selfish in bed. He will drive a decent car. He will be a “now” person. He won’t be comfortable with women. But he may have women friends. He will be a lone wolf. But he will be able to function in social settings. He won’t be unmemorable. But he will be unknowable. He will be either never married, divorced, or married, and if he was or is married his wife will be younger or older. He may or may not live in a rental, and might be lower class, upper lower class, lower middle class or middle class. And he will be crazy like a fox, as opposed to being mental. If you’re keeping score, that’s a Jacques Statement, two Barnum Statements, four Rainbow Ruses, a Good Chance Guess, two predictions that aren’t really predictions because they could never be verified—and nothing even close to the salient fact that BTK was a pillar of his community, the president of his church and the married father of two.

Now that we have strong academic reasons for saying FBI profiling (not to mention movies based on it like “Silence of the Lambs”) is pretty much full of shit, we need to get the Attorney General in the next administration to get the FBI’s badly needed technological updates to be done in a way to push seat of the pants, cold-reading “criminal profiling” to the fringes of the Bureau.

Let’s put this in the “War on Terror” context. We could have false profiles of terror bombers (witness Jewell for a past sample of that). Or note that Transportation Security “watch lists” are based n about the same level of scientific credibility.

Shoe leather detective work is one thing. But seat-of-the-pants hunches and guesswork gussied up as “profiling” is another thing altogether.


Anonymous said...

Movies are full of shit when it comes to reality? Oh, my - say it ain't so. Now that's a really grand piece of investigative reporting. What would we have done without you and your searing insights into this?

Having done a lot of work and research on profiling I know the "reality" of the system the FBI uses. I've sat with Robert Ressler and discussed this all with him. I've read all the material, including academic pieces on profiling. Ressler at least(if not Douglas) freely admits that his profiles never captured a killer - the cops on the ground did. He said that in his first book "Whoever Fights Monsters" and he said it to me as recently as September 2006.

His and Douglas' profiles and the men and women of BSU were more suggestions based on the crime scene evidence. They helped narrow the suspect list and yes, sometimes they were off.

At the time that Ressler developed his interview process it was revolutionary. Interviewing killers in jail was 1) the only access anyone had - there was no literature on these men and their crimes, 2) not being considered as a law enforcement tool by any crime enforcement entity. It was/is also today, an ongoing work in progress. And no, they didn't just interview inmates in CA - Ressler has talked to hundreds of men from every part of the country so on the basis of it, the facts your purporting aren't correct.

Also, they used a quite complex set of *written* criteria for the interview process. They didn't "cold read" as is purported by Gladwell and your blog. They may have suggested a direction since most of these men aren't all that articulate and also didn't want to volunteer information but they allowed the men themselves to create the information.

Stop watching TV and getting all your information from movies about these men and women and their groundbreaking work. Try reading a few books - see if that helps.

Mark Sevi

Gadfly said...

Oh, I'm just cut to the bone by your two-bit attempt at sarcasm.

And, I don't care what expertise you claim, especially and precisely because you've got a vested interest in it.

It's clearly unscientific and has been demonstrated so.

I don't know if Ressler has interviewed anybody elsewhere, but his California study followed no scientific protocols, as demonstrated.

Uhh, nothing in my post was from any TV or movies, either. How can you suggest I read some books when obviously you can can't even read my blog post correctly?

Go profile yourself.