I also posted this on Less Wrong.
This post follows my question Can Social Dynamics Explain Conjunction Fallacy Experimental Results? The results of the question were that no one provided any research contradicting the social dynamics hypothesis.
There is research on social dynamics. Asch’s conformity experiments indicate that wanting to fit in with a group is a very powerful factor that affects how people answer simple, factual questions like “Which of these lines is longer?” People will knowingly give wrong answers for social reasons. (Unknowingly giving wrong answers, e.g. carelessly, is easier.)
Conformity and other social dynamics can explain the conjunction fallacy experimental data. This post will focus on conformity, the dynamic studied in the Asch experiments.
This post assumes you’re already familiar with the basics of both the Asch and Conjunction Fallacy research. You can use the links if you need reminders.
First I’ll talk about whether conformity applies in the Conjunction Fallacy research setting, then I’ll talk about how conformity could cause the observed results.
Conformity in Groups
The Asch Conformity Experiments have people publicly share answers in a group setting. This was designed to elicit conformist behavior. Should we also expect conformist behavior in a different setting like the Conjunction Fallacy experiments setting? I suspect the different setting is a major reason people don’t connect the Asch and Conjunction Fallacy results.
I haven’t seen specific details of the Conjunction Fallacy research settings (in the text I read, details weren’t given) but I think basically people were given questionnaires to fill out, or something close enough to that. The setting is a bit like taking a test at school or submitting homework to a teacher. Roughly: Someone (who is not a trusted friend) will look over and judge your answers in some manner. In some cases, people were interviewed afterwards about their answers and asked to explain themselves.
Is there an incentive to conformity in this kind of situation? Yes. Even if there was no peer-to-peer interaction (not a safe assumption IMO), it’s possible to annoy the authorities. (Even if there were no real danger, how would people know that? They’d still have a reasonable concern.)
What could you do to elicit a negative reaction from the researchers? You could take the meta position that your answers won’t impact your life and choose the first option on every question. Effort expended on figuring out good answers to stuff should relate to its impact on your life, right? This approach would save time but the researchers might throw out your data, refuse to pay you, ask to speak with you, tell your professors about your (alleged) misbehavior (even if you didn’t violate any written rule or explicit request), or similar. You are supposed to abide by unwritten, unstated social rules when answering conjunction fallacy related questions. I think this is plenty to trigger conformity behaviors. It’s (like most of life) a situation where most people will try to get along with others and act in a way that is acceptable to others.
Most people don’t even need conformity behavior triggers. Their conformity is so automatic and habitual that it’s just how they deal with life. They are the blue pill normies, who aren’t very literal minded, and try to interpret everything in terms of its consequences for social status hierarchies. They don’t think like scientists.
What about the red pill autists who can read things literally, isolate scenarios from cultural context, think like a scientist or rationalist, and so on? Most of them try to imitate normies most of the time to avoid trouble. They try to fit in because they’ve been punished repeatedly for nonconformity.
(Note: Most people are some sort of hybrid. There’s a spectrum, not two distinct groups.)
When attending school people learn not to take questions (like those posed by the conjunction fallacy research) hyper literally. That’s punished. Test and homework questions are routinely ambiguous or flawed. What happens if you notice and complain? Generally you confuse and annoy your teacher. You can get away with noticing a few times, but if you complain about many questions on everything you’re just going to be hated and punished. (If people doubt this, we could analyze some public test questions and I can point out ambiguities and flaws.)
If you’re the kind of person who would start doing math when you aren’t in math class, you’ve gotten negative reactions in the past for your nonconformity. Normal people broadly dislike and avoid math. Saying “Hmm, I think we could use math to get a better answer to this.” is a discouraged attitude in our culture.
The Conjunction Fallacy research doesn’t say “We’re trying to test your math skills. Please do your best to use math correctly.” Even if it did, people routinely give misleading information about how literal/logical/mathematical they want things. You can get in trouble for using too much math, too advanced math, too complicated math, etc., even after being asked to use math. You can very easily get in trouble for being too literal after being asked to be literal, precise and rigorous.
So people see the questions and know that they generally aren’t supposed to sweat the details when answering questions, and they know that trying to apply math to stuff is weird, and most of them would need a large incentive to attempt math anyway, and the more rationalist types often don’t want to ruin the psychology study by overthinking it and acting weird.
I conclude that standard social behavior would apply in the Conjunction Fallacy research setting, including conformity behaviors like giving careless, non-mathematical answers, especially when stakes are low.
How Does Conformity Cause Bad Math?
Given that people are doing conformity behavior when answering Conjunction Fallacy research questions, what results should we expect?
People will avoid math, avoid being hyper literal, avoid being pedantic, not look very closely at the question wording, make normal contextual assumptions, and broadly give the same sorta answers they would if their buddy asked them a similar question in a group setting. Most people avoid developing those skills (literalism, math, ambiguity detection, consciously controlling the context that statements are evaluated in, etc.) in the first place, and people with those skills commonly suppress them, at least in social situations if not throughout life.
People will, as usual, broadly avoid the kinds of behaviors that annoy parents, teachers or childhood peers. They won’t try to be exact or worry about details like making probability math add up correctly. They’ll try to guess what people want from them and what other people will like, so they can fit in. They’ll try to take things in a “reasonable” (socially normal) way which uses a bunch of standard background assumptions and cultural defaults. That can mean e.g. viewing “Linda is a bank teller” as information a person chose to tell you, not as more like an out-of-context factoid chosen randomly by a computer, as I proposed previously.
Conformity routinely requires making a bunch of socially normal assumptions about how to read things, how to interpret instructions, how to take questions, etc. This includes test questions and similar, and most people (past early childhood) have past experiences with this. So many people won’t take conjunction fallacy questions literally.
People like the college students used in the research have taken dozens of ambiguous tests and had to figure out how to deal with it. Either they make socially normal assumptions (contrary to literalism and logic) without realizing they’re doing anything, or they noticed a bunch of errors and ambiguities but figured out a way to cope with tests anyway (or a mix like only noticing a few of the problems).
Conclusions
Conformity isn’t a straight error or bias. It’s strategic. It has upsides. There are incentives to do it and continue doing it (as well as major costs to transitioning to a different strategy).
If this analysis is correct, then the takeaway from the Conjunction Fallacy shouldn’t be along the lines of “People are bad at thinking.” It should instead be more like “People operate in an environment with complex and counter-intuitive incentives, including social dynamics.”
Social status hierarchies and the related social behaviors and social rules are one of the most important features of the world we live in. We should be looking to understand them better and apply our social knowledge more widely. It’s causally connected to many things, especially when there are interactions between people like interpretations of communications and requests from others, as is present in Conjunction Fallacy research.
Messages (3)
Good Afternoon,
This was a very interesting read and it reminded me of something, that may or may not be related. Are you familiar with mTurk and "Turking?"
If not I'll try to summarize it: Amazon will pay you (there are a ton of these not just Amazon, Amazon just happens to be the one I did) to fill out these research surveys. Personally, I must have done thousands of these surveys. You get paid anywhere from 10 cents to 75 cents per survey.
The incentive here is honestly to get through them as fast as possible. Many of these surveys have questions that test whether you're paying attention. The question might start like a normal but then it adds something like, "This is a test to make sure you're paying attention, please select D for the answer"
After a while you get pretty good at skimming through and finding these questions. So essentially you can blast through the survey, not care about the questions or answers except the attention ones and then get paid, move on to the next survey.
I am not claiming everyone does this, nor am I claiming that it is the right thing to do. However, it does make me extremely skeptical of any research done via surveys. These are done by thousands and thousands of people around the world, who are just doing it to get money a few cents at a time.
Btw, The few times I did pay attention to the questions, I found them quite ambiguous. I don't pretend to know statistics to know how they account for chance answers, mindless answers, or bad questions. It all seems rather suspect.
#16985 I broadly agree. But a lot of the psych research uses college students as subjects and deals with them in person (some of this research is from before the internet, too!). Lots of them are psych students. Sometimes they are participating in their own professor's research. I think the rate of good faith participation is a lot higher in that kinda scenario compared to mturk.
Also a lot of the college students participating in the research have certain cultural traits. They tend to be highly westernized and act middle class or better. They're generally well behaved. They're used to operating in high trust social circles and protecting their reputations with good behavior. They're pretty obedient to authorities like the researchers. Stuff like this results in a lot more good faith question answering.
#16986 It's not just any college students, either. This sorta research more often is done at research oriented schools like UBC (where some Conjunction Fallacy research was done) with very selective admissions, not at community colleges or party schools.