[Previous] Programming Discussion | Home | [Next] IKEA's Bad Grammar, Capitalism & Learning to Code

Twin Studies Are Frauds

Analytic approaches to twin data using structural equation models by Fruhling V. Rijsdijk and Pak C. Sham in 2002:

The classical twin study is the most popular design in behavioural genetics.

After saying how widely used they are, the paper talks about how twin studies work. It's pretty up front about why they don't work. The reasons they don't work are well known (and basically ignored anyway):

Assumptions of the twin method

They know they are making some assumptions. That’s not controversial. That’s interesting because many people who discuss this with me, and favor the power of genes, try to deny those assumptions exist. They will debate that because they are ignorant and the field hasn’t highlighted these assumptions enough in their public-facing material (which is no accident).

• Gene–environment correlations and interactions are minimal for the trait.

This assumption is, broadly, false for interesting or complex traits. Gene-environment interactions are everywhere. That's a key point that ruins ~all the twin studies. (More on this below.) It's not the only big problem though.

• Matings in the population occur at random (no assortment).

This assumption stood out to me because it’s so blatantly false. Mating isn't even random for animals. However, you may be able to get approximate answers anyway, so I’m not going to focus on this.

Gene–environment interaction (or genetic control of sensitivity to the environment) refers to different genotypes responding differently to the same environment or some genotypes being more sensitive to changes in environment than others.

E.g. suppose hypothetically that genes have some control over math ability. That results in the (school) environment responding differently to those genes by e.g. giving more praise and higher test scores for people with the genes that cause better mathematical ability. So if genes were involved in math ability, there would be major gene-environment interactions.

It's the same story with ~everything else. Does height help you win at basketball? Sure. But there are plenty of gene-environment interactions, like coaches who see that you're tall, or see that you hit more shots, and thus encourage you to pursue basketball more than they do for a short person who makes fewer shots. So the environment responds to you differently according to your height genes.

Any times genes have an effect that people notice, then people will respond to it. So the "environment" (which includes other people) is responding differently based on genes. So gene-environment interactions are basically only avoided when genes don't cause any variation that anyone notices. (Non-variation would be a group of people who all have one head. Genes caused them to have one head rather than zero or two. But because there is no variation in the trait, the environment can’t respond in varied ways to that trait.)

And you can't just look at gene-environment interactions which are directly on-topic. E.g. a math anxiety paper can't only look at math and anxiety stuff. You also need to consider e.g. childhood and parenting behaviors. A gene for infant smiling would be noticed by parents and result in different treatment, which could lead to better or worse results in general later on, including regarding math. Or it could be more complicated, e.g. maybe less infant smiling could result in more alienation from the parent which could tend to result in being better at math. Maybe people who have better relationships with their parents tend to end up more social, have more friends, and do, on average, more social climbing and less intellectual stuff.

The methods used by twin studies would claim that infant smiling gene as indicating partial genetic control over mathematical ability, even though it has nothing directly to do with math, and it could have dramatically different consequences, or no consequences, in a different culture. They would then publish about how genes partially control our lives, which they have proven yet again (using the same methods as all the previous studies with the same systematic weakness shared by those studies).

Here is an example of one twin study, of many, which is false and should be retracted because of the gene-environment interaction problem:

Genetic factors underlie the association between anxiety, attitudes and performance in mathematics

This paper is notable for citing a hostile satire (cite 30) as if it were serious research that they got some of their claims from. By “hostile” I mean that the satire article suggests that math anxiety research is a joke which does not merit funding. The paper also uses (as is typical) very-low-quality data (including getting some of their data years apart), e.g. badly designed surveys (even well designed surveys are highly problematic!) and proxies that don’t make sense (e.g. their idea of “number sense” is dot-quantity-estimation accuracy in 0.4 second flashes done 150 times). The paper also has carelessness and imprecision throughout. I discuss that paper, and its many flaws, at length in this video.

The paper authors are aware of the gene-environment problem. Near the end they say:

The current investigation presents some limitations. As well as relying on the methodological assumptions of twin design (see Rijsdijk & Sham, 2002 for a detailed description) (47), the models employed in the current investigation do not specifically account for gene–environment interplay. One possibility is that the observed genetic association between MA, attitudes and performance may operate via environmental effects that are correlated or interact with genetic predisposition. For example, children with a genetic predisposition towards experiencing difficulties with mathematics may develop a greater vulnerability to negative social influences in the context of mathematics, such as negative feedback received from teachers or parents on their effort and performance, which in turn may lead to greater feelings of anxiety towards mathematics (56). This has the potential to generate a negative feedback loop (7) between performance, motivation and anxiety - that is potentially a product of interacting inherited and environmental factors. The present investigation, including one time point for each measure of mathematics anxiety, attitudes and performance does not allow us to establish the direction of causality between the observed associations. Longitudinal genetically informative studies, integrating multiple measures of mathematics attitudes, anxiety and performance are therefore needed.

They know perfectly well that their research is inadequate to reach the conclusions they reached. They published anyway. The whole field acts like that in general. So they conclude:

Our findings of a shared, likely domain-specific, etiology between these mathematics-related traits provide a seminal step for future genetic research aimed at identifying the specific genes implicated in variation in the cognitive and non-cognitive factors of mathematics.

Instead of carefully thinking about the gene-environment interaction problem, and what to do about it, they simply ignore it and call their paper “seminal” anyway. They have no solution to the problem but they want to be scientists who publish papers with important conclusions, so they are dishonestly evading reality and lying to the public.

The field in general is like this. There are no sophisticated analyses of why gene-environment interactions would be minimal nor any counter-arguments to my fairly simple reasoning about why they’d be ever-present. They’re just making big, false claims without serious regard for what’s true. That’s the “social sciences”.


Elliot Temple on August 4, 2019

Comments (15)

> Or it could be more complicated, e.g. maybe less infant smiling could result in more alienation from the parent which could tend to result in being better at math.

That’s interesting. My mom said that she didn’t see me smile until 1 month old, which is when she learned that I have a dimple. And I always liked and excelled in math.


Anonymous at 2:38 PM on August 4, 2019 | #13206 | reply | quote

Twin studies don't understand causes and don't address what would happen in a different environment (like a different society and culture). But our society and culture change on a daily basis, they aren't constant, so twin studies are always wrong. The only way to deal with the constant change is to understand causal mechanism which help you figure out which changes are relevant and which aren't. This is the same kind of issue as with induction: there is ongoing change, and you need to figure out *which* patterns are relevant and important and will matter in the future, *not* just assume "patterns continue" (that's no help because there are infinitely many contradictory patterns).


Dagny at 3:18 PM on August 4, 2019 | #13207 | reply | quote

https://www.journals.uchicago.edu/doi/pdfplus/10.1086/705415

> We show that genetic endowments linked to educational attainment strongly and robustly predict wealth at retirement.

This assumes a static culture. It doesn’t show that those genes would be predictive in a different culture. Culture changes every day. To know which cultural changes are relevant requires conceptual understanding and analysis of causal mechanisms, *not* statistics. The paper, by ignoring causes (because they are harder than statistics) doesn’t even try to work on the important issues. This is typical of the field.

Found the paper via the author defending it on twitter. Sadly he indicates he's tired of wasting time replying to ignorant idiots who are just flaming – and indeed he has quoted someone who had nothing substantive to say – so I expect to be ignored. https://twitter.com/NWPapageorge/status/1157306414332878848


curi at 3:29 PM on August 4, 2019 | #13208 | reply | quote

Bad critics cause signal/noise problems. I guess he'd have more success getting worthwhile criticism on a discussion forum instead of on Twitter. Not that I think he actually wants it. Bad critics also provide a smokescreen to 1) pretend you addressed criticism 2) show you're better than your critics 3) give you an excuse for ignoring criticism b/c of its low quality.


curi at 3:31 PM on August 4, 2019 | #13209 | reply | quote

If X correlates with Y in situation Z, you cannot know from that whether X will correlate with Y in situation W. You have *zero information* about that. (Even if you call the new situation Z2 and point out that it has many big similarities to Z).

You must try to explain what is causing X and Y to correlate so that you can determine first if it's a coincidence or not, and second you can know what kind of changes to the situation would affect it.


curi at 3:34 PM on August 4, 2019 | #13210 | reply | quote

https://twitter.com/NWPapageorge/status/1157306414332878848

> It's clear most tweets are from ppl who did not read our paper.

And he makes recurring complaints related to this like:

> Publicly available genetic data linked to large data sets means: we need to be extremely careful. We try to be. (Read our paper to see how!)

He also calls reading the "darn" paper a "low bar!".

This is disingenuous because *the paper is paywalled*. When you charge $20, you can't expect most of Twitter to buy it. You ought to expect comments from non-purchasers instead of suggesting those are unreasonable.

The paywall page *does not say how many pages the paper is*, and otherwise does a shitty job of providing pre-sales information. I'm guessing it's long enough (and written in a difficult style) that reading it is not a "low bar".


Anonymous at 3:40 PM on August 4, 2019 | #13211 | reply | quote

#13211 It's 95 pages. Not a "low bar"!


Anonymous at 3:45 PM on August 4, 2019 | #13212 | reply | quote

What does the paper have to say about causes? From the conclusions section:

> genetic measures are likely endogenous to family environment, so one must be careful before assigning a causal interpretation to the gene-outcome gradients that we observe.

Earlier:

> 2.4 Interpretational Issues

> Several caveats apply to the interpretation of variation in polygenic scores, and correlations between polygenic scores and outcomes. First, it is difficult to assign a causal interpretation to the estimated relationship between the score and outcomes. In particular, variation in the polygenic score may reflect differences in environments or parental investments rather than differences in genetic factors across individuals.

They know what they are doing. They know their research doesn't work. They know what's wrong with it. They publish this stuff anyway and try to mislead the public. They are liars and frauds *on purpose*.

Why don't they take appropriate measures to deal with this problem? Because they can't. They have no idea how to. It's too hard. But rather than give up and do something they can actually do, they prefer to pretend they can do this.

---

The word "cause" is only in the paper once, in a title in a citation, and the word "causal" is only present twice (both quoted above).


curi at 3:52 PM on August 4, 2019 | #13213 | reply | quote

re #13206 - not about science

I'm the Anon who wrote #13206.

I remember my mom told me that when I was like 4 years old I was trying to cut my lashes because my teacher said she liked my lashes or something like that.

I explained this to my daughter and she said that it means I didn't want the attention.

Maybe I thought that long lashes meant I look like a girl and I didn't want to look like a girl.


Anonymous at 6:19 PM on August 4, 2019 | #13214 | reply | quote

> You must try to explain what is causing X and Y to correlate so that you can determine first if it's a coincidence or not, and second you can know what kind of changes to the situation would affect it.

What are some approaches to do this if statistics are excluded?


Anonymous at 2:24 AM on August 5, 2019 | #13219 | reply | quote

#13219 Stats are not *excluded*, they just aren't primary or sufficient.

The approach needed is *critical rationalism*. It is an *epistemology* (philosophy of knowledge) which explains *how learning works*. In particular, we learn by *conjectures and refutations* (brainstorming guesses and and using criticism to find and correct mistakes and otherwise improve the ideas we've created).

To understand causes, one must brainstorm ideas about what the causes could be and use critical thinking, including critical discussion, to rule out bad guesses.


curi at 10:53 AM on August 5, 2019 | #13222 | reply | quote

> I'm the Anon who wrote #13206.

> I remember my mom told me that when I was like 4 years old I was trying to cut my lashes because my teacher said she liked my lashes or something like that.

> I explained this to my daughter and she said that it means I didn't want the attention.

> Maybe I thought that long lashes meant I look like a girl and I didn't want to look like a girl.

Most people wind up seeking approval of others pretty early, but you were going the other way. That was a good sign!


Anonymous at 10:57 AM on August 5, 2019 | #13223 | reply | quote

>>I remember my mom told me that when I was like 4 years old I was trying to cut my lashes because my teacher said she liked my lashes or something like that.

>Most people wind up seeking approval of others pretty early, but you were going the other way. *That was a good sign!*

I don't agree. Not destroying something that others find pretty is not the same as seeking approval. Destroying the pretty is sometimes just the approval seeking from others. Just look at some fairly recent trends - how lady Gaga dresses or many feminists.


Anonymous at 2:48 AM on August 7, 2019 | #13246 | reply | quote

reply to #13246

> I don't agree. Not destroying something that others find pretty is not the same as seeking approval. Destroying the pretty is sometimes just the approval seeking from others. Just look at some fairly recent trends - how lady Gaga dresses or many feminists.

sometimes sure, but I think Anon was only speaking to the particular situation described in #13214. so I think you're disagreeing with something that wasn't said.


Anonymous at 6:04 AM on August 10, 2019 | #13271 | reply | quote

#13271

>> I don't agree. Not destroying something that others find pretty is not the same as seeking approval. Destroying the pretty is sometimes just the approval seeking from others. Just look at some fairly recent trends - how lady Gaga dresses or many feminists.

>sometimes sure, but I think Anon was only speaking to the particular situation described in #13214. so I think you're disagreeing with something that wasn't said.

I don't know how you (?) arrived to this being a good sign (the wanting to cut the lashes).

I think the right move of not seeking approval is just saying "thanks" and not bother with it one way or another. Wanting to cut the lashes stems from second-handedness, the praiser's opinion, not one's own.


N at 3:44 AM on August 12, 2019 | #13289 | reply | quote

(This is an unmoderated discussion forum. Discussion info.)