I’ve discovered that David Deutsch (DD) is an unreliable quoter. His book The Beginning of Infinity (BoI) contains many serious quotation errors, and he has misquoted elsewhere too.

For context, DD and I were close associates for a decade. I helped with BoI for 7 years and wrote over 200 pages of comments, suggestions and edits on drafts of the book. I learned a lot from him but I trusted his scholarship too much. I promoted his books. I was wrong about him and his books in multiple ways. My mistake. I retract my previous endorsements and recommendations of DD’s books. That doesn’t mean the books are awful or shouldn’t be read, but I no longer want to promote them myself. There are good ideas mixed in, but be wary of major problems.

Misquotes in The Beginning of Infinity

Block quotes are from BoI unless otherwise stated.

I think there will certainly not be novelty, say for a thousand years. This thing cannot keep going on so that we are always going to discover more and more new laws. If we do, it will become boring that there are so many levels one underneath the other . . . We are very lucky to live in an age in which we are still making discoveries. It is like the discovery of America – you only discover it once.
The Character of Physical Law (1965)

That’s different than what Feynman wrote. DD changed the words “perpetual novelty” to “novelty”. DD also changed “keep on going” to “keep going on”. (More details.)

Behind it all is surely an idea so simple, so beautiful, that when we grasp it – in a decade, a century, or a millennium – we will all say to each other, how could it have been otherwise?
John Archibald Wheeler, Annals of the New York Academy of Sciences, 480 (1986)

What Wheeler actually wrote was "Behind it all is surely an idea so simple, so beautiful, so compelling that when–in a decade, a century, or a millennium–we grasp it, we will all say to each other, how could it have been otherwise?”. DD deleted “so compelling” and moved “we grasp it” to before the dashed part. (More details.)

As the physicist Richard Feynman said, ‘Science is what we have learned about how to keep from fooling ourselves.’

Feynman didn’t say that. It’s not even a documented quote with some changes. It seems made up with no original source or evidence.

I checked the web and some Feynman books and speeches. It’s likely that the misquote started in 2000 in the article Magical Thinking (my thanks to Justin Mallone for finding that article), which paraphrased Feynman that way without using quote marks or giving a source. Unfortunately, the wording made it sound like it was an actual quote, so I think people started spreading it as a quote. Then DD probably got the misquote from an unreliable webpage and put it in his book without trying to find a primary source or telling his readers which unreliable secondary source he used. There are now two books which give this quote and cite it to as quoted in BoI. There’s also a book which gives the quote with a footnote saying that the author was unable to find a source for the quote (then don’t put it in your book!).

Feynman said some similar ideas in Cargo Cult Science, but the wordings are different. DD didn’t take the quote from there and add one or two errors (like he did with some other quotes, where you can tell that he’s quoting a specific thing incorrectly). It’s too different to have come from that speech.

Popper wrote:

The inductivist or Lamarckian approach operates with the idea of instruction from without, or from the environment. But the critical or Darwinian approach only allows instruction from within – from within the structure itself . . .

I contend that there is no such thing as instruction from without the structure. We do not discover new facts or new effects by copying them, or by inferring them inductively from observation, or by any other method of instruction by the environment. We use, rather, the method of trial and the elimination of error. As Ernst Gombrich says, ‘making comes before matching’: the active production of a new trial structure comes before its exposure to eliminating tests.
The Myth of the Framework

DD ends the first sentence of the second paragraph with “without the structure” and then a period. Instead of a period, Popper had a comma there and continued the sentence. Then, the rest of that paragraph that DD quotes is actually from a different section of the book. DD combined sentences from different places in the book and presented them as one paragraph with no ellipsis or square brackets to indicate a modification.

And DD left out the words “In fact,” before “I contend”. DD also put an ellipsis at the end of the first paragraph when that should be a period. There are no omitted words there. The paragraph ends there and DD continues without skipping a paragraph. DD also left out Popper’s italics.

A similar misquote also appeared on the Taking Children Seriously (TCS) website (mirror). DD co-founded TCS with Sarah Fitz-Claridge and she’s my best guess at the author of that misquote, though it could have been DD. Either way, he has responsibility for what it says on the official website of the movement he co-founded (particularly for pages, like this one, with no author specified).

Judging by the similarities, the misquote in BoI was likely based on the TCS website misquote. Even when a secondary source is accurate, it’s problematic to take a secondary source quote and then edit it without checking the original. When you do that, you’re making edits without knowing the original context and wording, so you aren’t in a good enough position to judge what edits are OK.

(More details about the TCS website version of the misquote.)

Thanks to Dec for telling me that this misquote is also in BoI after I wrote about the version from the TCS website.

As the physicist Stephen Hawking put it, humans are ‘just a chemical scum on the surface of a typical planet that’s in orbit round a typical star on the outskirts of a typical galaxy’.

This quote seems to be made up based on a similar Hawking quote about “chemical scum” from the 1995 TV program (Reality on the Rocks: Beyond Our Ken by Ken Campbell (IMBD, trailer)). Interestingly, DD had quoted it correctly in The Fabric of Reality as “The human race is just a chemical scum on a moderate-sized planet, orbiting round a very average star in the outer suburb of one among a hundred billion galaxies.” I didn’t find the original video, but it’s quoted that way in various places online that aren’t based on DD’s writing. (Some sources have “around” instead of “round”, which is an understandable difference given how similar those words can sound when spoken out loud.)

It seems that DD made up this misquote for his 2005 TED talk and then based the quote in BoI on his talk. Alan Forrester checked the books The large scale structure of space-time, A Brief History of Time, The Grand Design, The Nature of Space and Time and The Universe in a Nutshell, but found that none contain the word “scum”. And I can’t find any online sources for Hawking ever saying the BoI version of the quote (whereas with the The Fabric of Reality version, I easily found other online sources).

This misquote doesn’t seem fully accidental. DD changed the quote to be more elegant and catchy by repeating “typical” three times. I’ve noticed that many of DD’s misquotes involve changing text to sound nicer.

(More details.)

[Horgan believed] that science has the ability to ‘resolve questions’ objectively […]

Horgan actually wrote “Scientists have the ability to pose questions and resolve them in a way that critics, philosophers, historians cannot.” DD changed Horgan’s words “resolve them” to “resolve questions”, which is wrong without using square brackets to indicate an edit. (More details.)

The issue of what exactly needs to be explained in an ‘appearance of design’ was first addressed by the clergyman William Paley, the finest exponent of the argument from design. In 1802, before Darwin was born, he published the following thought experiment in his book Natural Theology.

It’s unclear what, if anything, “appearance of design” is a quote from, but it’d be understandable if a reader believed it was a quote of Paley in Natural Theology. But it’s not in that book.

[Paley wrote:]

The inference we think is inevitable, that the watch must have had a maker . . . There cannot be design without a designer; contrivance without a contriver; order without choice; arrangement without anything capable of arranging; subserviency and relation to a purpose without that which could intend a purpose; means suitable to an end . . . without the end ever having been contemplated or the means accommodated to it. Arrangement, disposition of parts, subserviency of means to an end, relation of instruments to a use imply the presence of intelligence and mind.

DD changed the words “any thing” to the word “anything” and changed “an use” to “a use”. DD also quoted from both chapters 1 and 2, but presented it as single paragraph with ellipses. DD also removed a comma near the end before “imply the presence of intelligence and mind” which helped the reader understand the text. (DD edited other punctuation too, but this punctuation edit stood out to me because it’s significantly worse than the original.) (More details.)

As Hawking once put it, ‘Television sets could come out [of a naked singularity].’

Thanks to Alan Forrester for looking into this quote at my request. He was unable to find Hawking saying this. He searched the web and the following books: The large scale structure of space-time, A Brief History of Time, The Grand Design, The Nature of Space and Time and The Universe in a Nutshell.

As Hofstadter remarked, ‘In retrospect, I am quite amazed at how much genuine intelligence I was willing to accept as somehow having been implanted in the program . . . It is clear that I was willing to accept a huge amount of fluidity as achievable in this day and age simply by putting together a large bag of isolated tricks, kludges and hacks.’

Hofstadter’s actual paragraph ends with “a large bag of isolated tricks-kludges and hacks, as they say.” DD’s punctation edits changed the meaning. Hofstadter said “isolated tricks” and then gave “kludges and hacks” as a rewording of “isolated tricks”. DD changed it to a list of three things, “tricks, kludges and hacks” and made it sound like the modifier “isolated” applies to all three, whereas in the original it applied only to “tricks”. As a list, it means that all three things were put together. In the original, it says they put together tricks, and then provides the additional information that the tricks could be characterized as kludges and hacks as people (informally) say.

Thanks to Dec for bringing up this misquote.

Representative Roger Q. Mills of Texas complained in 1882, ‘I thought . . . that mathematics was a divine science. I thought that mathematics was the only science that spoke to inspiration and was infallible in its utterances [but] here is a new system of mathematics that demonstrates the truth to be false.’

This text is available in the Congressional Record. DD changed the words "by inspiration" to "to inspiration". It’s also misleading that where DD wrote “[but]”, with no ellipsis, he skipped multiple sentences and continued with text from a different paragraph.

DD likely copied this misquote from Fair Representation without telling his readers that he was trusting a secondary source without fact checking it, and without letting readers know which secondary source he was using.

Thanks to Justin Mallone for looking this up at my request.

Before Blackmore and others realized the significance of memes in human evolution, all sorts of root causes had been suggested [...] [T]here is the ‘Machiavellian hypothesis’ that human intelligence evolved in order to predict the behaviour of others, and to fool them. […] Blackmore’s ‘meme machine’ idea, that human brains evolved in order to replicate memes, must be true.

At first I read “Machiavellian hypothesis” as a quote of Blackmore from her book The Meme Machine that DD mentioned earlier and included in his bibliography. If so, it's a misquote. That phrase isn’t in her book.

But maybe “Machiavellian hypothesis” is merely meant to be the name of a hypothesis. If so, it’s the wrong name. The correct name is "Machiavellian Intelligence”, as one can find out from Blackmore’s book or Wikipedia. Blackmore has an index entry for “Machiavellian Intelligence” and cites two books with “Machiavellian Intelligence” in their title. She also writes “An influential version of social theory is the ‘Machiavellian Intelligence’ hypothesis (Byrne and Whiten 1988; Whiten and Byrne 1997).”. It appears that DD read her book, misremembered the name of the hypothesis, didn’t check it, and put quote marks around it. (More details.)

The more important fundamental laws and facts of physical science have all been discovered, and these are now so firmly established that the possibility of their ever being supplanted in consequence of new discoveries is exceedingly remote . . . Our future discoveries must be looked for in the sixth place of decimals.
Albert Michelson, address at the opening of the Ryerson Physical Laboratory, University of Chicago, 1894

The source for this, which DD didn’t specify, is the book Light Waves and Their Uses (1903) by Albert Michelson. The speaker wrote down what he said in his own book. Michelson wrote:

Many other instances might be cited, but these will suffice to justify the statement that "our future discoveries must be looked for in the sixth place of decimals."

DD incorrectly quoted this as Michelson saying “[o]ur future discoveries must be looked for in the sixth place of decimals” himself, when Michelson actually had it in quote marks and talked about that statement. Deleting quote marks within a quote is misquoting. (More details.)

For example, as I wrote in The Fabric of Reality:

Consider one particular copper atom at the tip of the nose of the statue of Sir Winston Churchill that stands in Parliament Square in London. Let me try to explain why that copper atom is there. It is because Churchill served as prime minister in the House of Commons nearby; and because his ideas and leadership contributed to the Allied victory in the Second World War; and because it is customary to honour such people by putting up statues of them; and because bronze, a traditional material for such statues, contains copper, and so on. Thus we explain a low-level physical observation – the presence of a copper atom at a particular location – through extremely high-level theories about emergent phenomena such as ideas, leadership, war and tradition.

There is no reason why there should exist, even in principle, any lower-level explanation of the presence of that copper atom than the one I have just given. Presumably a reductive ‘theory of everything’ would in principle make a low-level prediction of the probability that such a statue will exist, given the condition of (say) the solar system at some earlier date. It would also in principle describe how the statue probably got there. But such descriptions and predictions (wildly infeasible, of course) would explain nothing. They would merely describe the trajectory that each copper atom followed from the copper mine, through the smelter and the sculptor’s studio and so on . . . In fact such a prediction would have to refer to atoms all over the planet, engaged in the complex motion we call the Second World War, among other things. But even if you had the superhuman capacity to follow such lengthy predictions of the copper atom’s being there, you would still not be able to say ‘Ah yes, now I understand why they are there’. [You] would have to inquire into what it was about that configuration of atoms, and those trajectories, that gave them the propensity to deposit a copper atom at this location. Pursuing that inquiry would be a creative task, as discovering new explanations always is. You would have to discover that certain atomic configurations support emergent phenomena such as leadership and war, which are related to one another by high-level explanatory theories. Only when you knew those theories could you understand why that copper atom is where it is.

In addition to checking this using ebooks, I also compared hardback copies of both books. It’s FoR pp. 22-23 and BoI pp. 109-110.

DD quotes “understand why they are there” but the original reads “understand why it is there”. DD changed the words “it is” to “they are”.

DD quotes "Pursuing that inquiry”, but FoR says “this inquiry”. DD changed the word “this” to “that”.

DD quotes “understand why that copper atom is where it is”. DD omitted the word “fully”. The original said “understand fully why”.

DD wrote “[You]” in BoI, which is an incorrect use of square brackets. He skipped two sentences and should have used an ellipsis. And it says “You” in the original, so he shouldn’t put it in square brackets since it isn’t modified. Square brackets can only replace an ellipsis when the text in square brackets replaces/summarizes/paraphrases all the skipped text, but the word “You” doesn’t replace the skipped sentences.

The italics “why” and “what it was” are not italicized in FoR.

The ellipsis DD used in “studio and so on . . . In fact” is incorrect because the original had a period after “so on”. There should be four dots there (one for the period, and three for the ellipsis), not three dots.

DD doesn’t even quote himself accurately.

Other Misquotes

In The Fabric of Reality, DD wrote:

Mystery is part of the very concept of time that we grow up with. St Augustine, for example, said:

What then is time? If no one asks me, I know; if I wish to explain it to one who asks, I know not. (Confessions)

This quote has some word changes compared to the edition of Confessions that I checked. However, there are other English translations, so it could be an accurate quote of one of those. DD didn’t say which translation he used, which is more problematic than usual when quoting a particular translation rather than quoting something with a single, unambiguous wording that could be looked up.

DD wrote in Not Merely the Finest TV Documentary Series Ever Made:

As Karl Popper put it, we humans can “let our ideas die in our place.”

I found Popper saying something similar three times, but he didn’t use that wording. I think DD relies on his memory for this quote, instead of checking a source. He’s quoted it different ways in different places (e.g. with “theories” instead of “ideas” in Why It’s Good To Be Wrong and BoI). DD should get quotes from sources instead of putting quote marks around what he believes he remembers someone writing.

Popper said similar things in The Myth of the Framework (“By criticizing our theories we can let our theories die in our stead.”), In Search of a Better World (“Now we can let our theories die in our place.”), and Natural Selection and the Emergence of Mind (“Let our conjectures, our theories, die in our stead!”).

DD’s associate Chiara Marletto also misquoted Popper as saying "let our ideas die in our place.”.

DD’s associate Sarah Fitz-Claridge misquoted William Godwin and intentionally sanitized a quote about slavery. She gives “The condition of a … slave in the West-Indies, is in many respects preferable to that of the youthful son of a free-born European. The slave is purchased upon a view of mercantile speculation; and, when he has finished his daily portion of labour, his master concerns himself no further about him. But the watchful care of the parent is endless. The youth is never free from the danger of grating interference.”. She misquoted by changing the words "of its grating" to “of grating”. And she sanitized the 1797 quote by changing "negro-slave" into "... slave”. (I think that’s an incorrect use of an ellipsis, too.) Also, the quote is horrible because it downplays how bad slavery was, so it’s disturbing that Fitz-Claridge liked the quote enough to highlight it.

Fitz-Claridge also misquoted The Myth of the Framework. It’s some of the same Popper material misquoted in BoI and on the TCS website, but misquoted differently. This time, Fitz-Claridge changed “that” to “which” and got the page numbers wrong. (More details in the second update.)

Smaller Issues

DD frequently doesn’t give sources for quotes which makes it harder to check their accuracy. By leaving out sources, he’s asking his reader to trust him. But he made many quoting errors, so that trust would be misplaced.

DD repeatedly writes “X … Y” when X and Y are from different paragraphs or even different sections of a book. This is misleading. He also does it when X is a complete sentence, which makes it look like X is not a complete sentence.

DD frequently changes capitalization and punctuation without square brackets to indicate the change. DD capitalizes stuff to make it look like the start of a sentence when it isn’t (both at the start of a quote or after an ellipsis). DD also puts periods inside the quote marks after quoting a partial sentence, which makes it look like that was the end of the sentence when it wasn’t. DD also repeatedly puts a space then an ellipsis after a sentence ends, which should be a period then ellipsis but he changed the period to a space. So he makes stuff look like the start or end of a sentence when it isn’t, and then other times he makes stuff look like it’s not the end of a sentence when it is.

DD doesn’t appear to have a consistent policy for periods going inside or outside of quotes. E.g. I searched an electronic copy of BoI and found 188 instances of a single close quote followed by a period, and 88 instances of a period followed by a single close quote (and 3 instances of period, single close quote, and period again, which all involved a number, ellipsis, close quote, then period). DD often ends quotes with a period inside the quotation marks when the original sentence doesn’t end there, but other times he puts the period outside the quotation marks, and I don’t know why. DD is also inconsistent about italicizing quotes the same way they are in the original.

There are standard guidelines for how to do quotations, which DD violates, in addition to the larger misquote issues I presented above. For example, Working with Quotations, from Suny Empire State College, says:

Remember that when you do choose to use direct quotations, you need to retain the exact wording, spelling, capitalization, and punctuation of the original source.

DD went to both Oxford and Cambridge. I don’t think they teach lower standards than state colleges, and in any case he hasn’t followed their guidelines. For example, this Guide for authors and editors from the Oxford University Press says:

Quoted matter must reflect the original source exactly in spelling, punctuation, and capitalization. Please double-check all quotations against the sources from which you have taken them to ensure that you have copied accurately.

It must be possible for the reader to identify the work from which a quotation has been taken.

The University Of Oxford Style Guide says “Place any punctuation which does not belong to the quote outside the quotation marks (except closing punctuation if the end of the quote is also the end of the sentence).”

The Cambridge Editorial Style Guide says “A full stop is used outside the quotation mark if the quote is only part of a sentence.” and “Always source quotations”.

Taking Credit from Karl Popper

There’s another scholarship issue in DD’s books. I was horrified to discover that Karl Popper’s name is only in The Fabric of Reality (FoR) chapter 3 two times. That chapter is focused on sharing Popper’s ideas. But it doesn’t give adequate credit. In particular, the diagrams (3.1, 3.2, 3.3) are clearly based on Popper’s diagrams in Objective Knowledge, but DD never tells the reader that they’re modified from Popper. (More details.)

DD has presented himself as very humble and modest. He’s claimed multiple times to be only footnotes to Popper (which comes off as exaggerated modesty, rather than convincing his fans that it’s actually true). But DD has simultaneously misled readers to believe he accomplished much more than he did, e.g. in FoR ch. 3. This is a pattern. For example, in a 2016 paper, The logic of experimental tests, particularly of Everettian quantum theory, DD wrote:

An important consequence of this explanatory conception of science is that experimental results consistent with a theory T do not constitute support for T. That is because they are merely explicanda. A new explicandum may make a theory more problematic, but it can never solve existing problems involving a theory (except by making rival theories problematic – see Section 3). The asymmetry between refutation (tentative) and support (non-existent) in scientific methodology is better understood in this way, by regarding theories as explanations, than through Popper's (op. cit.) own argument from the logic of predictions, appealing to what has been called the ‘arrow of modus ponens’. Scientific theories are only approximately modelled as propositions, but they are precisely explanations.

This passage misleads readers into believing that DD improved on Popper by making a better argument focused on explanations instead of on the logic of predictions. Most readers would be surprised to discover that Popper made both arguments. Popper did make the logic of predictions argument (which is less important but was worth making too) but also made the other argument that DD is implying is his own original work. DD made some original contributions to epistemology, but not this one.

You can search Popper’s Conjectures and Refutations (C&R) for words like "tentative", "explanation", and "support" to see that DD is less original than he implies. Popper also covers these issues in other books. I’ll give one example from C&R:

For a scientific theory—an explanatory theory—is, if anything, an attempt to solve a scientific problem, that is to say, a problem concerned or connected with the discovery of an explanation.[6]

This clearly shows that Popper viewed scientific theories as explanatory theories. DD didn’t come up with the idea that scientific theories are explanations. The footnote at the end of that quote refers readers to more of Popper’s writing. Popper talked about explanation often. Popper also came up with the asymmetry between refutation (tentative) and support (non-existent). Popper emphasized refutation, said it was only tentative, and is also the person who challenged thousands of years of philosophical tradition by arguing that support is non-existent. Popper drew multiple major distinctions between negative and positive approaches to epistemology.

DD also gets other people to praise him while he presents himself as humble. For example, his TCS co-founder Sarah Fitz-Claridge (SFC) wrote that Popper invented a philosophy of science and that David Deutsch and his TCS philosophy had extended Popper’s epistemology to apply outside of science. (SFC probably wrote that. It’s on an official TCS page, but doesn’t specify the author, so it could have been written by DD.) That’s a major misrepresentation. Popper was seeking a general theory of knowledge, and said so, and applied it outside of science. (More details.)

DD has gotten his associates to praise him as e.g. having “the greatest mind ever”. SFC believes that DD made major improvements on Popper, and so do many of DD’s fans. I don’t believe it’s an accident that many people overestimate DD in ways similar to his co-founder who publicly promotes him. It looks like a strategy where DD plays humble while having other people say things that would sound arrogant coming from him. (More details.)

In 2012, SFC wrote to the official Fabric of Reality discussion group (archive of FoR posts) (my italics):

In my view it would be much more accurate to say that David has the greatest mind ever to have existed. His thinking is breathtakingly logical and brilliant. His ideas have changed the world and will do so even more profoundly in the future. I have never met anyone more pure, more truth-seeking and more open to criticism than David.

In 2000, SFC wrote to TCS list (my italics):

So really, people should not speak of Popper, but of Deutsch, because it was David who came up with the link between Popper's ideas and educational theory.

Note that DD has a history of secretly ghostwriting stuff which SFC then claims to be the author of. (Source: DD’s friend. He or she was friends with DD before DD turned 18 and they’re still friends now, over 50 years later. He or she had many discussions about TCS, Popper and more with DD and SFC. DD introduced me to the friend and I had some discussions with him or her.) Full disclosure: In the past, I’ve posted a few things that DD wrote, but under my own name, with his permission and approval. I did this when (as best I remember) I wanted to share something good that he told me, which I thought would benefit the world, but he didn’t want to share it himself and wouldn’t let me post it and attribute it to him or to an anonymous person, but he would let me post it under my name without attributing it to anyone. Here’s an example that I remember (I think it was the most significant, memorable instance). More often, I wrote stuff myself that was based on things DD told me, and he didn’t want credit but was happy for me to say it. At other times, DD helped edit my writing and a sentence or two of his ended up in the final version without credit (he didn’t want credit). DD had substantial influence over some of my early writing, and he also has had substantial influence over some of SFC’s writing that he didn’t fully ghostwrite.

Conclusion

I was mistaken about how good DD’s books are. They’re worse than I thought. I still think there is significant value in those books, but you can’t trust DD’s scholarship. Besides distrusting direct quotes given by DD, you should also distrust paraphrases or summaries of what other people said or thought. You have to check things yourself if you actually want to know. DD is too unreliable. And don’t use DD as a secondary source. Don’t spread quotes that DD quoted; quote directly from the original source or don’t use it. Some people are spreading his misquotes (they’ve even been repeated in books).

FYI, I don’t think DD is especially bad at quoting compared to others. Lots of books and academic articles have major errors related to quotes, paraphrases, cites or facts. But for a book to be considered great, it should do better. In the world today, you shouldn’t trust authors with quotes or facts by default. You should be suspicious by default unless an author earns more trust. Many people believe DD has earned a lot of trust (including me in the past), but they’re mistaken.

I apologize for encouraging people to respect and trust DD more than he merits. I know I played a role in that.

DD’s misquote problem also helps contextualize his recent mistreatment of me. How could a super rational, great person act like that? The answer is that he he’s actually a deeply flawed person with some good traits mixed in as special exceptions.

DD once got very upset with me for questioning a Godwin quote he sent me in a private discussion. He’d sent it without a specific source and I couldn’t find the quote by searching the book. It turned out that he’d quoted an obscure first edition but I was searching the third edition. He should have praised me for looking for errors instead of getting defensive and lashing out at me. Good scholars don’t expect to be trusted and don’t mind being questioned or challenged. Even though he didn’t misquote in that instance, DD’s irrational attitude was a warning sign that DD might be a misquoter. I failed to recognize the full problem and I didn’t go fact check his books at that time (2011).

BoI has an errata page (mirror) which documents a bunch of errors in the book. They are mostly factual errors, and the number and severity should concern readers. Despite all the misquotes in the book, there are currently no misquotes on the errata page, which says something about how little fact checking the book has been getting (there are probably a bunch of other errors that no one has found yet).

I will edit my book recommendation articles to warn people about DD’s misquotes. I will also take down the beginningofinfinity.com website that promotes BoI and put a warning there about DD’s misquotes.

---

Update, 2021-07-12: In a 1985 physics paper in a prestigious, peer-reviewed journal, David Deustch misquoted Alan Turing.

Update, 2021-07-13: I made two videos related to DD misquotes:

Video about the Feynman misquote: "Science is what we have learned about how to keep from fooling ourselves.". This video goes into more depth.

Video about this blog post about DD's misquotes. This video is more of a broad overview.

I thought that even though David Deutsch (DD) and his fans were harassing me, his books were still good. But I hadn’t reread them for years. On review, The Beginning of Infinity contains lots of misquotes. DD’s books are a lot worse than I realized. I was horrified to discover how frequently and severely DD misquotes. I trusted DD’s ability to quote accurately and handle details reliably and correctly, but I was wrong.

Also, DD explains too little in his books. They’re too hard to learn effectively from because he doesn’t give enough depth or detail. I had trouble seeing this in the past because I had many conversations with DD which filled in the gaps for me. But even when DD’s books say something important, he often doesn’t provide enough information for a reasonable, smart person to understand it well.

DD’s books have some good parts mixed in, but, due to the serious flaws, I retract my recommendation of them. I no longer want to actively promote them.

I’m sorry. I should have caught the misquotes earlier. I was capable of finding those errors years ago. I found and wrote about other similar errors.

I was giving DD space after he left the community. I guessed (I think accurately) that he wanted to be left alone by me and I was trying to respect his wishes. I mostly stayed away from him and his work after he left. I thought continuing to recommend his books was safe, but I was wrong about that.

I only started my video series about BoI after I gave up on DD leaving me alone. I caught the Feynman misquote in chapter 1 when I first reread it for the videos. BoI misquotes Feynman:

As the physicist Richard Feynman said, ‘Science is what we have learned about how to keep from fooling ourselves.’

Then, due to the harassment, I was reviewing old information and found a Popper misquote on an old TCS webpage. Someone (“Dec”) saw my post and told me that the same Popper quote was in BoI too. That made two misquotes in BoI, which was a possible pattern. That led to checking more quotes, which led to discovering that there are tons of misquotes in BoI.

It seems that no other readers of BoI have noticed the misquoting problem yet (the errata page has factual errors but no misquotes), which I think is important information about the world. Regardless, I should have done better.

Read about the misquotes.

I completed my 12 part video series about The Beginning of Infinity by David Deutsch. I helped him with the book for 7 years.

Parts 1-9 and 11 explain ideas from chapter 1. Part 10 is about a misquote in chapter 1. And part 12 is about my blog post about many misquotes throughout the book.

I will no longer be promoting Deutsch's books.

David Deutsch (DD) wrote in Quantum theory, the Church-Turing principle and the universal quantum computer (1985), p. 3:

Church (1936) and Turing (1936) conjectured ... This is called the ‘Church-Turing hypothesis’; according to Turing,

Every ‘function which would naturally be regarded as computable’ can be computed by the universal Turing machine. (1.1)

And from Deutsch's references (p. 19):

Turing, A. M. 1936 Proc. Lond. math. Soc. Ser. 2, 442, 230.

Now we'll compare with Turing's paper: On Computable Numbers, With An Application To The Entscheidungsproblem (1936), p. 230:

the computable numbers include all numbers which could naturally be regarded as computable.

Turing wrote "numbers", but DD misquoted that as "function". Turing also wrote "could" which DD misquoted as "would".

I double checked using two other copies of Turing's paper. (One and two.)

There's also a problem because Deutsch uses what appears to be an italicized block quote. You'd expect the whole block quote to be a quote of Turing, but instead it's a paraphrase. Inside the paraphrase are quotation marks surrounding the misquote of Turing that I criticized.

DD's citation is also incorrect. DD cites Turing's paper to volume 442 of the Proceedings of the London Mathematical Society, but it was actually in volume 42 not 442.

To determine what's correct, we can check how Turing himself cites it. In a correction to his paper, Turing cited himself:

Proc. London Math. Soc. (2), 42 (1936-7), 230-265.

You can also get the correct cite, with volume 42, from the Stanford Encyclopedia of Philosophy or from Wikipedia.

You can also see that the latest volume of the journal, published in 2021, is volume 122. Volume 442 is unlikely to exist for over 100 more years. And the journal's website has archives showing that the Turing article was in volume 42.

Tangentially, I hope this lowers your opinion of academic peer review. DD's paper was published in Proceedings of the Royal Society of London, a prestigious and peer-reviewed journal that started in around 1830. It has published work from many famous scientists.

---

Thanks to Dec for finding this misquote.

Note that DD has published a lot of misquotes.

---

Update 2021-07-15: Dec pointed out that a similar Turing misquote is in DD's book The Fabric of Reality:

He [Turing] conjectured that this repertoire consisted precisely of ‘every function that would naturally be regarded as computable’.

No, Turing wrote "all numbers which could" not "every function that would".

It appears that DD got this misquote from his own paper, and also modified it. There's a recurring pattern where every time DD touches a quote, there's a significant chance that he changes something. Here, he took the word "every" which was outside of quote marks in his paper and moved it inside quote marks for his book.

---

Update 2021-09-14: I contacted the academic publisher (proceedings of the royal society). They looked into the matter and said:

Apologies for the delay in getting back to you on this. A board member has had a look at the paper and does not think the misquote affects the outcome of the research presented in the paper. Although the error in the refences is unfortunate, we do not believe it will prevent readers from finding the correct article. Given the age of the paper we therefore do not think any further action is necessary.

I have several criticisms of this response.

They agree with me that DD misquoted and miscited.

Why won't they put up errata on their website? Is that too hard for them (they are bad at websites?) or do they actually not want to?

Errata serves several purposes. Academics working in the field could find out about the issue. People debating the issue could also refer to it – it would e.g. let a student whose professor repeated the error borrow the journal's authority to correct the professor. It's risky to correct your professor in general, but much easier with an official errata to point him to.

Is correcting professors a real issue? I think so because professors have been teaching Deutsch's error (there are some examples posted in the comments below). And they've been doing it out of context. In other words, even if the error did not affect the conclusion of Deutsch's paper, it still can affect other conclusions about other issues. So spreading the error matters, and it has in fact been taught in schools. Also, any reader of the paper may remember the Turing quote and use it for something else, and it may negatively affect the conclusion of their usage, even if it didn't affect the conclusion of Deutsch's paper. (Admittedly, some of the professors don't cite a source and might have been getting the error from Deutsch's book The Fabric of Reality where he repeated a similar error. But the fact that Deutsch put roughly the same error in his book is, IMO, an additional reason to errata it and at least do a little bit to stop the spread of the error.)

If they published an errata or other note about the error, they could also state their reasons for why they believe the paper's conclusion is unaffected. Other people could consider that reasoning and potentially disagree. This could be an area for critical thinking and truth seeking rather than an unaccountable authority pronouncing judgment for secret reasons. Even if it's no big deal in this case, their general attitude is concerning. How many other judgments do they make with no transparency? What is the nature of those judgments? Are any of those judgments mistaken? Do they gloss over many errors in papers they published? Could they be doing that partly out of bias and not wanting to draw attention to their own involvement in errors?

People expect academic science journals with peer review to have high standards and to be really picky about errors. They are not living up to this reputation. So much for their unlimited interest in truth for the sake of truth or whatever they were supposed to be doing.

They are still sharing the paper electronically and could update it there. Deutsch is still alive and available and could actually write or approve a small update, or they could do an update which is labelled as written by a journal editor not Deutsch.

How did this error happen? How did every step of the publishing process miss it? Did anyone intentionally cause or allow the error? Were any biases involved? They did no post mortem, no root cause analysis, no investigation into their peer review and editorial process, etc.

There are major causes for concern here. This errors calls into question how effective their reviewers and editors are. It also calls into question Deutsch's integrity. Maybe it was an accident but they have given no account of how it could have happened accidentally nor asked him to give one.

Do peer reviewers or editors not check quotes or cites? Should they? How widespread a problem is misquoting? How many other misquote reports do they receive, validate as correct criticism, and then bury? Might they be hiding a pattern revealing that many papers contain misquotes? Instead of hiding misquotes should they be doing something different like e.g. paying people enough money for misquote reports to make finding the misquotes worth the time and effort? If they actually wanted to find out about misquotes, and find out how big a problem it is, wouldn't they do something more like that? They could have responded to me by offering me money to find more misquotes since I've proven I can do it. That seems reasonable if they were better and more interesting in correcting errors.

Deutsch had an argument with a referree which was related to the text Deutsch misquoted:

http://www.daviddeutsch.org.uk/wp-content/uploads/2018/03/MathematiciansMisconception.pdf

But I soon found out that not everyone saw it that way. I also had referee problems. The referee of the paper in which I presented that proof insisted that Turing’s phrase “would naturally be regarded as computable” referred to mathematical naturalness – mathematical intuition – not nature.

(BTW, as a first impression, without reading Turing's paper or investigating the issue, I agree with the referree. When talking about naturally regarding something, that sounds like it's talking about what is natural or intuitive to people and their opinions, not about nature, due to what the key word "regard" means.)

Could Deutsch have intentionally misquoted in order to help win a specific logical point he was arguing about with the reviewer? Could the horrible, misleading presentation of the quote (as a block quote with an internal quote – which btw has tricked some people into thinking the whole thing is a quote) have been some kinda compromise worked out between Deutsch and the peer reviewer? Was the misquote in earlier drafts of the paper? Do they have records of what changes were made to the paper during peer review? In any case, there is some possible motive here for Deutsch falsifying the quote on purpose or just being biased and more careless in his own favor. Deutsch has a history of repeated misquotes throughout his career and most of them favor him in some way and I don't recall any that were bad for him, so it seems like whatever's going on involves bias if not actual deliberate, fully-conscious misquoting.

Seriously, how do wording errors in quotes happen accidentally? I understand typoing a letter or two when typing a quote in from a paper book or journal. But how do you just change the word? That seems more like Deutsch quotes stuff from memory – and his memory is biased in his favor (or there's selection bias – if he likes the version he remembers then he uses it, but if it's not ideal then he looks up the exact wording). Quoting from memory in your books and papers (and scripted speeches) is a serious scholarship violation that should lead to repercussions and major reputational damage. That's totally unacceptable. Another possibility, which there have also been potential indicators for, is that Deutsch changes quotes during his editing process without double checking the original. I suspect Deutsch thinks certain minor changes to quotes are OK, and maybe this somehow escalates to more major wording changes after multiple editing passes. Deutsch's editing could be like the game "telephone" where you whisper something to the guy next to you, who whispers it to the next guy, and so on. The goal is to repeat exactly what you heard. After something has been whispered a dozen times, often all the words are different and the meaning is totally changed.

In my experience, people are often willing to view things as "an accident" or "a mistake" without thinking about how exactly it happened. Some mistakes are simple like a one letter typo happening because you pressed the wrong keyboard key by accident because your finger dexterity is good but imperfect so occasionally you hit the wrong key (and then you usually notice and fix the typo, but not always). But many errors don't have such simple explanations and merit actual analysis. Changing the word "numbers" to "function" is not a typo due to flawed finger dexterity. That's bias, misremembering (while incorrectly believing quoting from memory is OK), intentionally falsifying the quote, or perhaps a horribly unreasonable editing processes that edits words within quotes similarly to how it edits words that are not within quotes. Or there are other possibilities like maybe a peer reviewer or editor caused the error and Deutsch didn't have full control over the final wording of his paper.

And how did the journal miss the error? Was it anyone's job to catch the error? Would the journal like to catch such errors in the future? And how did the error remain unnoticed in the archives for decades? Do they have a tiny readership? Do their readers not care about errors? Do their readers fail to report errors? Do their readers report errors but nothing is done? Would it make sense to hire people to review the archives for errors or should they focus on catching more errors before publication or should they just continue to not even post errata about errors and pretend nothing happened?

For more info, see my reply email to the journal:

https://curi.us/2477-academic-journals-are-unreasonable

In The Ayn Rand Lexicon (book not website), Harry Binswanger wrote in the Honesty section:

Self-esteem is reliance on one’s power to think. It cannot be replaced by one’s power to deceive. The self-confidence of a scientist and the self-confidence of a con man are not interchangeable states, and do not come from the same psychological universe. The success of a man who deals with reality augments his self-confidence. The success of a con man augments his panic.

The intellectual con man has only one defense against panic: the momentary relief he finds by succeeding at further and further frauds.
[“The Comprachicos,” NL, 181.]  

The words, book and page number are correct, but the quote is from "The Age of Envy" not "The Comprachicos".

In the self-esteem section, Binswanger gives the correct cite for part of the same quote:

Self-esteem is reliance on one’s power to think. It cannot be replaced by one’s power to deceive. The self-confidence of a scientist and the self-confidence of a con man are not interchangeable states, and do not come from the same psychological universe. The success of a man who deals with reality augments his self-confidence. The success of a con man augments his panic.
[“The Age of Envy,” NL, 181.]

Justin Mallone found this error and I checked it myself too. I asked him to look into Lexicon quoting accuracy after I found multiple citation errors on the Lexicon website that weren't in the book. This is the only error he found in the book. He did find citation and formatting errors on the website. None of the errors, even on the website, are wording errors. (Note there's a second website for the Lexicon. I compared the "Automatization" page and the only difference I found was whether there were spaces around dashes or not.)

I checked 4 quotes originally and Justin checked 16 more. So the book had 1 partial citation error in 20 quotes, but the website had 5 errors in 20 quotes (counting at most one error per quote). The wordings seem to be reliable, unlike in The Beginning of Infinity, and the Lexicon book seems to be pretty reliable. It seems like a serious effort went into getting details right for the book, but the process of creating the website was sloppier and introduced many small errors.

Even the Lexicon website is much better than David Deutsch's use of quotations in The Beginning of Infinity. Deutsch frequently doesn't give sources, makes frequent changes to wordings (with no indicator of any change), changes punctuation too, and uses ellipses and square brackets incorrectly. Even worse, several of the quotes appear to be made up.

The Stanford Encyclopedia of Philosophy article on Epistemology says:

others regard credences as metaphysically reducible to beliefs about probabilities (see Byrne in Brewer & Byrne 2005),

You would expect the cited source to discuss credences, metaphysics, reduction, or probabilities. It does not.

As the title, introduction and ending all make clear, Perception and Conceptual Content is about perception.

Even if it briefly mentioned the topic SEP cited it for somewhere (I didn't read all the words), the cite would still be unreasonable because SEP would be citing it for just one small part but didn't specify a particular page, quote or section. In that scenario, there would be no reasonable way to find or determine what the cite refers to.

This large error is revealing about the scholarship standards not only at the SEP but in academia in general.

---

Update 2021-08-21:

I emailed the authors of the article about this error when I posted this criticism, and I quickly received this response from Ram Neta:

Thanks!

I don’t know how that citation was introduced into the article, since Byrne’s paper was just published this year. Let me see if the SEP editors will let me fix this.

Sent from my iPhone

Errors are sometimes introduced by other people besides the author, but that doesn't stop them from being published or widespread :/ I'm not sure that that's what's going on here, though. See below.

And he's not even sure if the SEP editors will allow the error to be fixed! What is wrong with their publishing process!?

So, I see that the SEP article says:

First published Wed Dec 14, 2005; substantive revision Sat Apr 11, 2020

So it was revised last year, but not this year. Was Byrne's paper just published this year as claimed? That would be unexpected given the cite says it was published in 2005:

(see Byrne in Brewer & Byrne 2005)

And the bibliography has:

Brewer, Bill and Alex Byrne, 2005, “Does Perceptual Experience Have Conceptual Content?”, CDE-1: 217–250 (chapter 8). Includes:
Brewer, Bill, “Perceptual Experience Has Conceptual Content”, CDE-1: 217–230.
Byrne, Alex, “Perception and Conceptual Content”, CDE-1: 231–250.

I see that the Byrne paper has a bunch of cites, but none are from later than 2004.

Looking more, I found the book it was published in, by reading the note at the start of the SEP article bibliography:

The abbreviations CDE-1 and CDE-2 refer to Steup & Sosa 2005 and Steup, Turri, & Sosa 2013, respectively.

So the book is Contemporary Debates in Epistemology 1st Edition, which was published in 2005. And one of the authors of CDE, Steup, is also an author of the SEP article. Using "Look Inside" on the hardcover version on Amazon, I can see the table of contents and confirm that the Byrne article is in the book.

I also found that the Byrne article, in CDE-1, was in the bibliography of the SEP article in 2007:

https://web.archive.org/web/20070609171028/https://plato.stanford.edu/entries/epistemology/index.html

However, in that version of the SEP article, Byrne only comes up in the bibliography, not the text. Looking at more archived versions, I see that "(see Byrne in Brewer & Byrne 2005)" was there in May 2020 but not in Dec 2019. In Dec 2019, the word "credence" wasn't present at all in the SEP article and Neta Ram was not yet a co-author and wasn't cited at all. Steup was the only author listed then. Then in 2020, when a major revision happened, "credence" was added to the page 21 times and "Neta" was added 11 times. It seems like Steup was probably the author in 2005 and cited himself a lot. Then Neta probably did the update, added a bunch of stuff about credences, and added a bunch of cites to himself.

The full sentence with the cite error is:

The latter dispute is especially active in recent years, with some epistemologists regarding beliefs as metaphysically reducible to high credences,[5] while others regard credences as metaphysically reducible to beliefs about probabilities (see Byrne in Brewer & Byrne 2005), and still others regard beliefs and credences as related but distinct phenomena (see Kaplan 1996, Neta 2008).

You can see a new neta cite was added here in addition to the mistaken Byrne cite which goes to a source that had already been in the bibliography for 15 years and was not published this year as claimed. Maybe Byrne came out with a different paper in 2021 about credences and Neta cited the wrong paper because it was already in the bibliography? You might think that doesn't work because how would Neta have been trying to cite a 2021 paper back in 2020, as Neta pointed out in his email to me. But I found that Byrne did have a 2021 paper, but according to Google Scholar it was available online in 2019. That's not unusual. Academics often publish papers online before in print.

So it looks to me like the error was probably Neta's fault despite his attempt to deflect blame. Especially considering he was careless enough that he didn't seem to read my whole email, which was quite short, but did contain the quote "Byrne 2005" and somehow he writes back to tell me Byrne's paper was from 2021 (ignoring the 2005 cite) and that therefore he couldn't have even tried to cite it so he doesn't know what happenend, suggesting he was never trying to cite Byrne there. But he did intend to cite Byrne and was too quick to disown that while carelessly forgetting that papers get prepublished and not trying to investigate what actually happened as I did above.

Link for Byrne's paper being published in 2021: https://onlinelibrary.wiley.com/doi/10.1111/phpr.12768

And the online version I found with Google Scholar which says it's from 2019: https://philarchive.org/archive/BYRPAP-2v1

It seems like Neta rushed to reply to my email and deflect blame, and to move on without any real post mortem or investigation, and made careless statements to me, while under no actual time pressure to reply immediately. I hadn't even told him my negative blog post existed. For reference, here is the full email I wrote to Neta (also sent to Steup):

Subject: Error in SEP Epistemology article

You wrote:

https://plato.stanford.edu/entries/epistemology/index.html

others regard credences as metaphysically reducible to beliefs about probabilities (see Byrne in Brewer & Byrne 2005),

The Byrne text you cite is here:

https://web.mit.edu/abyrne/www/percepandconcepcontent.pdf

It doesn’t contain the strings “credence”, “credal”, “meta”, or “reduc”. The two instances of “proba” are not discussions of probabilities. I hope you’ll appreciate being informed about the error.

Anyway, given the careless email reply to me, you can imagine how careless citation errors get into his work. In this case, it seems like he wanted to cite a Byrne paper and then used the cite already in the bibliography even though the year was over a decade off and the title was totally different. So I can figure out what he did and what happened, but he can't or won't? You may then wonder how and why SEP chose this guy. I wonder that too. I suspect it'd be very hard to get a transparent answer from SEP and that, on a related note, the answer would be damning.

Oh and it gets way worse. The 2021 Byrne paper is relevant but the cite says:

others regard credences as metaphysically reducible to beliefs about probabilities (see Byrne in Brewer & Byrne 2005),

But Byrne doesn't think credences reduce to beliefs. He writes e.g.:

the solution—to adapt a phrase from Quine and Goodman—is to renounce credences altogether.

Those are the last words of the introduction.

A reduction in the other direction, of credence to belief, seems hopeless from the start: as was pointed out, to have credence .6 in p is not to believe anything.

and

Granted that neither credence nor belief can be reduced to the other, there is an immediate problem

and

That leaves belief monism, the thesis of this paper: “there are no such things as credences”

So in addition to citing the wrong paper because he's careless and probably shouldn't be an academic ... and answering my email incorrectly because he's careless and probably shouldn't be an academic ... the paper he intended to cite blatantly contradicts his paraphrase of it.

---

Update 2, 2021-08-21:

I replied by email to Neta, left Steup CCed, and added Byrne to the CC list. I again used a factual, understated style and tone. Neta replied, keeping them CCed, to say

Helpful, thanks!

Sent from my iPhone

So on the one hand it could be worse. On the other hand, he's fully failing to acknowledge how much he screwed up, that it has any significant meaning, or that I did anything special that goes beyond minor help from a random guy to an expert. He's acting kinda like I reported a typo. And that's after I find layers of error in his writing and also his initial email to me was wrong.

I don't intend to reply to Neta's response. Here's a copy what I emailed:

Ram Neta, based on your comments, I looked into it more. I think what happened is this:

Steup put the 2005 Byrne article, Perception and Conceptual Content, in the bibliography, but it wasn’t cited in the text.

In 2020, you wanted to cite Byrne’s 2021 article, Perception and Probability, which you have indicated familiarity with. That was possible because it had been available online since 2019 at https://philarchive.org/archive/BYRPAP-2v1

You accidentally cited the Byrne article that was already in the bibliography instead of the new one.

The new article is on the right topic but contradicts your statements about it. You characterize Byrne’s position like this:

regard credences as metaphysically reducible to beliefs about probabilities

But what Byrne actually says is:

the solution ... is to renounce credences altogether.

and

A reduction in the other direction, of credence to belief, seems hopeless from the start: as was pointed out, to have credence .6 in p is not to believe anything.

and

Granted that neither credence nor belief can be reduced to the other, there is an immediate problem

and

That leaves belief monism, the thesis of this paper: “there are no such things as credences”

So Byrne does not believe that credences reduce to probability beliefs.

I wonder if it could be defamation to publish, in the SEP, lies about what philosophical positions a rival philosopher from another school of thought believes and published... Imagine publishing in the SEP that Ayn Rand was a Marxist!

BTW, speaking of carelessness, Neta's CV says "ENTERIES IN REFERENCE WORKS". That isn't how you spell "entries", so maybe he's a well-suited person to have any of those.

I think a lot of people don't read what they cite, but do they not even skim it, keyword search it, read the introduction/abstract, or glance at the conclusion?

---

Update 3, 2021-08-21:

Alex Byrne replied:

Ram, I cannot cast the first stone -- I am sure I have made many more mistakes of this sort myself. Possibly you had not realized how implausible my view is. Elliot, thanks very much for pointing this out. (The paper appeared in PPR, by the way -- I should update the philpapers entry.)

best
Alex

---

Update 4, 2021-08-22:

Ram Neta wrote:

Thanks Alex: I actually never read your paper, I only recall the version you delivered at Rutgers, and that we talked about on the train afterwards. I’ll read the paper now, since obviously my memory is not to be trusted!

Sent from my iPhone

I guess that answers the thing I was wondering about yesterday: can't read or won't read? This was more of a won't/don't read – he didn't actually read the paper and then egregiously misunderstand it. I'm also not seeing signs that he misuses an assistant to create errors. I think reading something and then citing it months or years later without rereading happens too and leads to errors. I think a lot of these people are bad at skimming and text search, so they rely on memory too much. Like Neta could have web searched to find the paper he wanted to cite, and glanced at it, instead of relying on his inaccurate memories of an IRL talk. But he didn't and just thought citing to a paper based on memories of a talk is acceptable scholarly practice. And that kinda standard is OK with his university, the journals that publish him, and with the SEP.

Regarding sharing these emails: I'm just a random stranger to them, I did nothing to schmooze, establish rapport, or act friendly. All I did was tell him he was badly wrong, twice, without flaming or volunteering my opinion of him, what I think he should do about the error, or what I think the error says about him, SEP and academia. They also (I presume) made the choice not to look me up – my signature had a link and I'm easy to find with web search too. That the author of the emails I sent would have a blog – and even would blog about the errors – should not be very surprising.

I think the world should know what academia is like. It's not really hidden but it's not exactly shared either. It's kinda an open secret for people in the know, but a lot of laymen don't realize it.

They are social climbers. Neta got to write a SEP article and added tons of cites to himself, and also added cites for people he likes or wants to network with. He doesn't remember Byrne's philosophical positions but does remember meeting him IRL and sharing a train ride and a chat. Byrne has an MIT job. So he wanted to give Byrne a cite to strengthen the social connection. Cites are favors used for social networking. Not always but often.

I know Neta is admitting too much because he's not on the defensive. This is common with social climbers. They're two-faced. They try to recognize safe situations to be candid, and other situations to be guarded. They say different things on different occassions. But they're often pretty careless about it and bad at it. Too bad. It reminds me of what people will admit about how bad their romantic relationships are when they aren't defensive or guarded – but if it's a debate their tune and tone change, and they become biased and dishonest, and try to say stuff to benefit their side instead of seeking the truth objectively.

I wrote the below email to the Proceedings of the Royal Society (academic journal) as a followup to the issue of Deutsch misquoting Turing. They agreed that Deutsch's quote and citation were both inaccurate, but didn't want to do anything, even post an errata, on the basis that the errors didn't affect the paper's conclusion.

---

Thanks for getting back to me. I have a few remaining concerns.

The quote in question was related to a disagreement when the paper was first published. Deutsch said:

http://www.daviddeutsch.org.uk/wp-content/uploads/2018/03/MathematiciansMisconception.pdf

I also had referee problems. The referee of the paper in which I presented that proof insisted that Turing’s phrase “would naturally be regarded as computable” referred to mathematical naturalness – mathematical intuition – not nature. And so what I had proved wasn’t Turing’s conjecture.

I wonder what processes were in place – from both Deutsch and referees – that could still miss that it’s a misquote, with an incorrect cite, while actively debating what that exact phrase means. That specific part of the paper got particular attention and the error was somehow missed anyway. Or perhaps the debate over that quote caused edits which introduced the error (I wonder if there are still records of what changes were made during the review process?). I suspect there’s a systems, processes and policies problem somewhere that could be improved.

Turing’s actual words being significantly different (Deutsch changed “numbers” to “function” but those are different concepts) has a meaningful chance to matter to the debate they had over what Turing meant. And Deutsch seems to agree with the referee that that debate matters to what Deutsch had and hadn’t proved, to his conclusion.

I don’t think a wording change like that can easily be explained as a random error, like a typo. I think a root cause analysis would be worthwhile, including e.g. asking Deutsch how he thinks the error happened. There could have been quoting from memory, changing quotes during editing passes, intentionally changing it to better address the referee’s objections, a change made by the referee himself (I don’t know if they are able to change any words), or something else. It’s hard to speculate but could be investigated since there are no obvious answers that make what happened reasonable. I think the results of looking into this would be relevant to many other papers at your journal and others. I’ve found that misquotes are widespread throughout the academic (and non-academic) worlds.

Also, even if the conclusion of this paper is unchanged, I think an errata would be appropriate because people have been spreading the error and using the misquote for other purposes. It's been taught to students in university courses[1]. In general, people read trusted sources like your journal, remember some parts, and then reuse stuff for other purposes. An error that doesn’t matter in one context often does matter in another context. Posting an errata on your website would help with this ongoing problem.

I also think it’d be reasonable to, along with the errata, publicly share the reasoning that the error doesn’t matter to Deutsch’s conclusion so that other people can judge for themselves.

[1] Here is an example of a Stanford course spreading the error: https://cs269q.stanford.edu/lectures/lecture1.pdf

I proposed a game on the Effective Altruism forum where people submit texts to me and I find three errors. I wrote this for that game. This is a duplicate of my EA post.

Introduction

I'm no fan of university nor academia, so I do partly agree with The Case Against Education by Bryan Caplan. I do think social climbing is a major aspect of university. (It's not just status signalling. There's also e.g. social networking.)

I'm assuming you can electronically search the book to read additional context for quotes if you want to.

Error One

For a single individual, education pays.

You only need to find one job. Spending even a year on a difficult job search, convincing one employer to give you a chance, can easily beat spending four years at university and paying tuition. If you do well at that job and get a few years of work experience, getting another job in the same industry is usually much easier.

So I disagree that education pays, under the signalling model, for a single individual. I think a difficult job search is typically more efficient than university.

This works in some industries, like software, better than others. Caplan made a universal claim so there's no need to debate how many industries this is viable in.

Another option is starting a company. That's a lot of work, but it can still easily be a better option than going to university just so you can get hired.

Suppose, as a simple model, that 99% of jobs hire based on signalling and 1% don't. If lots of people stop going to university, there's a big problem. But if you individually don't go, you can get one of the 1% of non-signalling jobs. Whereas if 3% of the population skipped university and competed for 1% of the jobs, a lot of those people would have a rough time. (McDonalds doesn't hire cashiers based on signalling – or at least not the same kind of signalling – so imagine we're only considering good jobs in certain industries so the 1% non-signalling jobs model becomes more realistic.)

When they calculate the selfish (or “private”) return to education, they focus on one benefit—the education premium—and two costs—tuition and foregone earnings.[4]

I've been reading chapter 5 trying to figure out if Caplan ever considers alternatives to university besides just entering the job market in the standard way. This is a hint that he doesn't.

Foregone earnings are not a cost of going to university. They are a benefit that should be added on to some, but not all, alternatives to university. Then univeristy should be compared to alternatives for how much benefit it gives. When doing that comparison, you should not subtract income available in some alternatives from the benefit of university. Doing that subtraction only makes sense and works out OK if you're only considering two options: university or get a job earlier. When there are only two options, taking a benfit from one and instead subtracting it from the other as an opportunity cost doesn't change the mathematical result.

See also Capitalism: A Treatise on Economics by George Reisman (one of the students of Ludwig von Mises) which criticizes opportunity costs:

Contemporary economics, in contrast, continually ignores the vital connection of income and cost with the receipt and outlay of money. It does so insofar as it propounds the doctrines of “imputed income” and “opportunity cost.”[26] The doctrine of imputed income openly and systematically avows that the absence of a cost constitutes income. The doctrine of opportunity cost, on the other hand, holds that the absence of an income constitutes a cost. Contemporary economics thus deals in nonexistent incomes and costs, which it treats as though they existed. Its formula is that money not spent is money earned, and that money not earned is money spent.

That's from the section "Critique of the Concept of Imputed Income" which is followed by the section "Critique of the Opportunity-Cost Doctrine". The book explains its point in more detail than this quote. I highly recommend Reisman's whole book to anyone who cares about economics.

Risk: I looked for discussion of alternatives besides university or entering the job market early, such as a higher effort job search or starting a business. I didn't find it, but I haven't read most of the book so I could have missed it. I primarily looked in chapter 5.

Error Two

The answer would tilt, naturally, if you had to sing Mary Poppins on a full-price Disney cruise. Unless you already planned to take this vacation, you presumably value the cruise less than the fare. Say you value the $2,000 cruise at only $800. Now, to capture the 0.1% premium, you have to fork over three hours of your time plus the $1,200 difference between the cost of the cruise and the value of the vacation.

(Bold added to quote.)

The full cost of the cruise is not just the fare. It's also the time cost of going on the cruise. It's very easy to value the cruise experience at more than the ticket price, but still not go, because you'd rather vacation somewhere else or stay home and write your book.

BTW, Caplan is certainly familiar with time costs in general (see e.g. the last sentence quoted).

Error Three

Laymen cringe when economists use a single metric—rate of return—to evaluate bonds, home insulation, and college. Hasn’t anyone ever told them money isn’t everything! The superficial response: Economists are by no means the only folks who picture education as an investment. Look at students. The Higher Education Research Institute has questioned college freshmen about their goals since the 1970s. The vast majority is openly careerist and materialist. In 2012, almost 90% called “being able to get a better job” a “very important” or “essential” reason to go to college. Being “very well-off financially” (over 80%) and “making more money” (about 75%) are almost as popular. Less than half say the same about “developing a meaningful philosophy of life.”[2] These results are especially striking because humans exaggerate their idealism and downplay their selfishness.[3] Students probably prize worldly success even more than they admit.

(Bold added.)

First, minor point, some economists have that kind of perspective about rate of return. Not all of them.

And I sympathize with the laymen. You should consider whether you want to go to university. Will you enjoy your time there? Future income isn't all that matters. Money is nice but it doesn't really buy happiness. People should think about what they want to do with their lives, in realistic ways that take money into account, but which don't focus exclusively on money. In the final quoted sentence he mentions that students (on average) probably "prize worldly success even more than they admit". I agree, but I think some of those students are making a mistake and will end up unhappy as a result. Lots of people focus their goals too much on money and never figure out how to be happy (also they end up unhappy if they don't get a bunch of money, which is a risk).

But here's the more concrete error: The survey does not actually show that students view education in terms of economic returns only. It doesn't show that students agree with Caplan.

The issue, highlighted in the first sentence, is "economists use a single metric—rate of return". Do students agree with that? In other words, do students use a single metric? A survey where e.g. 90% of them care about that metric does not mean they use it exclusively. They care about many metrics, not a single one. Caplan immediately admits that so I don't even have to look the study up. He says 'Less than half [of students surveyed] say the same [very important or essential reason to go to university] about “developing a meaningful philosophy of life.”' Let's assume less than half means a third. Caplan tries to present this like the study is backing him up and showing how students agree with him. But a third disagreeing with him on a single metric is a ton of disaagreement. If they surveyed 50 things, and 40 aren't about money, and just 10% of students thought each of those 40 mattered, then maybe around zero students would agree with Caplan about only the single metric being important (the answers aren't independent so you can't just use math to estimate this scenario btw).

Bonus Error

Self-help gurus tend to take the selfish point of view for granted. Policy wonks tend to take the social point of view for granted. Which viewpoint—selfish or social—is “correct”? Tough question. Instead of taking sides, the next two chapters sift through the evidence from both perspectives—and let the reader pick the right balance between looking out for number one and making the world a better place.

This neglects to consider the classical liberal view (which I believe, and which an economist ought to be familiar with) of the harmony of (rational) interests of society and the individual. There is no necessary conflict or tradeoff here. (I searched the whole book for "conflict", "harmony", "interests" and "classical" but didn't find this covered elsewhere.)

I do think errors of omission are important but I still didn't want to count this as one of my three errors. I was trying to find somewhat more concrete errors than just not talking about something important and relevant.

Bonus Error Two

The deeper response to laymen’s critique, though, is that economists are well aware money isn’t everything—and have an official solution. Namely: count everything people care about. The trick: For every benefit, ponder, “How much would I pay to obtain it?”

This doesn't work because lots of things people care about are incommensurable. They're in different dimensions that you can't convert between. I wrote about the general issue of taking into account multiple dimensions at once at https://forum.effectivealtruism.org/posts/K8Jvw7xjRxQz8jKgE/multi-factor-decision-making-math

A different way to look at it is that the value of X in money is wildly variable by context, not a stable number. Also how much people would pay to obtain something is wildly variable by how much money they have, not a stable number.

Potential Error

If university education correlates with higher income, that doesn't mean it causes higher income. Maybe people who are likely to get high incomes are more likely to go to university. There are also some other correlation isn't causation counter-arguments that could be made. Is this addressed in the book? I didn't find it, but I didn't look nearly enough to know whether it's covered. Actually I barely read anything about his claims that university results in higher income, which I assume are at least partly based on correlation data, but I didn't really check. So I don't know if there's an error here but I wanted to mention it. If I were to read the book more, this is something I'd look into.

Screen Recording

Want to see me look through the book and write this post? I recorded my process with sporadic verbal commentary:

https://www.youtube.com/watch?v=BQ70qzRG61Y

I proposed a game on the Effective Altruism forum where people submit texts to me and I find three errors. I wrote this for that game. This is a duplicate of my EA post. I criticize Minimalist extended very repugnant conclusions are the least repugnant by Teo Ajantaival.

Error One

Archimedean views (“Quantity can always substitute for quality”)

Let us look at comparable XVRCs for Archimedean views. (Archimedean views roughly say that “quantity can always substitute for quality”, such that, for example, a sufficient number of minor pains can always be added up to be worse than a single instance of extreme pain.)

It's ambiguous/confusing about whether by "quality" you mean different quantity sizes, as in your example (substitution between small pains and a big pain), or you actually mean qualitatively different things (e.g. substitution between pain and the thrill of skydiving).

Is the claim that 3 1lb steaks can always substitute for 1 3lb steak, or that 3 1lb pork chops can always substitute for 1 ~3lb steak? (Maybe more or less if pork is valued less or more than steak.)

The point appears to be about whether multiple things can be added together for a total value or not – can a ton of small wins ever make up for a big win? In that case, don't use the word "quality" to refer to a big win, because it invokes concepts like a qualitative difference rather than a quantitative difference.

I thought it was probably about whether a group of small things could substitute for a bigger thing but then later I read:

Lexical views deny that “quantity can always substitute for quality”; instead, they assign categorical priority to some qualities relative to others.

This seems to be about qualitative differences: some types/kinds/categories have priority over others. Pork is not the same thing as steak. Maybe steak has priority and having no steak can't be made up for with a million pork chops. This is a different issue. Whether qualitative differences exist and matter and are strict is one issue, and whether many small quantities can add together to equal a large quantity is a separate issue (though the issues are related in some ways). So I think there's some confusion or lack of clarity about this.

I didn't read linked material to try to clarify matters, except to notice that this linked paper abstract doesn't use the word "quality". I think, for this issue, the article should stand on its own OK rather than rely on supplemental literature to clarify this.

Actually, I looked again while editing, and I've now noticed that in the full paper (as linked to and hosted by PhilPapers, the same site as before), the abstract text is totally different and does use the word "quality". What is going on!? PhilPapers is broken? Also this paper, despite using the word "quality" in the abstract once (and twice in the references), does not use that word in the body, so I guess it doesn't clarify the ambiguity I was bringing up, at least not directly.

Error Two

This is a strong point in favor of minimalist views over offsetting views in population axiology, regardless of one’s theory of aggregation.

I suspect you're using an offsetting view in epistemology when making this statement concluding against offsetting views in axiology. My guess is you don't know you're doing this or see the connection between the issues.

I take a "strong point in favor" to refer to the following basic model:

We have a bunch of ideas to evaluate, compare, choose between, etc.

Each idea has points in favor and points against.

We weight and sum the points for each idea.

We look at which idea has the highest overall score and favor that.

This is an offsetting model where points in favor of an idea can offset points against that same idea. Also, in some sense, points in favor of an idea offset points in favor of rival ideas.

I think offsetting views are wrong, in both epistemology and axiology, and there's overlap in the reasons for why they're wrong, so it's problematic (though not necessarily wrong) to favor them in one field while rejecting them in another field.

Error Three

The article jumps into details without enough framing about why this matters. This is understandable for a part 4, but on the other hand you chose to link me to this rather than to part 1 and you wrote:

Every part of this series builds on the previous parts, but can also be read independently.

Since the article is supposed to be readable independently, then the article should have explained why this matters in order to work well independently.

A related issue is I think the article is mostly discussing details in a specific subfield that is confused and doesn't particularly matter – the field's premises should be challenged instead.

And another related issue is the lack of any consideration of win/win approaches, discussion of whether there are inherent conflicts of interest between rational people, etc. A lot of the article topics are related to political philosophy issues (like classical liberalism's social harmony vs. Marxism's class warfare) that have already been debated a bunch, and it'd make sense to connect claims and viewpoints to that the existing knowledge. I think imagining societies with different agents with different amounts of utility or suffering, fully out of context of imagining any particular type of society, or design or organization or guiding principles of society, is not very productive or meaningful, so it's no wonder it's gotten bogged down in abstract concerns like the very repugnant conclusion stuff with no sign of any actually useful conclusions coming up.

This is not the sort of error I primarily wanted to point out. However, the article does a lot of literature summarizing instead of making its own claims. So I noticed some errors in the summarized ideas but that's different than errors in the articles. To point out errors in an article itself, when its summarizing other ideas, I'd have to point out that it has inaccurately summarized the ideas. That requires reading the cites and comparing them to the summaries. Which I don't think would be especially useful/valuable to do. Sometimes people summarize stuff they agree with, so criticizing the content works OK. But here a lot of it was summarizing stuff the author and I both disagree with, in order to criticize it, which doesn't provide many potential targets for criticism. So that's why I went ahead and made some more indirect criticism (and included more than one point) for the third error.

But I'd suggest that @Teo Ajantaival watch my screen recording (below) which has a bunch of commentary and feedback on the article. I expect some of it will be useful and some of the criticisms I make will be relevant to him. He could maybe pick out some things I said and recognize them as criticisms of ideas he holds, whereas sometimes it was hard for me to tell what he believes because he was just summarizing other people's ideas. (When looking for criticism, consider if I'm right, does it mean you're wrong? If so, then it's a claim by me about an error, even if I'm actually mistaken.) My guess is I said some things that would work as better error claims than some of the three I actually used, but I don't know which things they are. Also, I think if we were to debate, discussing the underlying premises, and whether this sub-field even matters, would acutally be more important than discussing within-field details, so it's a good thing to bring up. I think my disagreement with the niche that the article is working within is actually more important than some of the within-niche issues.

Offsetting and Repugnance

This section is about something @Teo Ajantaival also disagrees with, so it's not an error by him. It could possibly be an error of omission if he sees this as a good point that he didn't know but would have wanted to think of but didn't. To me it looks pretty important and relevant, and problematic to just ignore like there's no issue here.

If offsetting actually works – if you're a true believer in offsetting – then you should not find the very repugnant scenario to be repugnant at all.

I'll illustrate with a comparison. I am, like most people, to a reasonable approximation, a true believer in offsetting for money. That is, $100 in my bank account fully offsets $100 of credit card debt that I will pay off before there are any interest charges. There do exist people who say credit cards are evil and you shouldn't have one even if you pay it off in full every month, but I am not one of those people. I don't think debt is very repugnant when it's offset by assets like cash.

And similarly, spreading out the assets doesn't particularly matter. A billion bank accounts with a dollar each, ignoring some adminstrative hassle details, are just as good as one bank account with a billion dollars. That money can offset a million dollars of credit card debt just fine despite being spread out.

If you really think offsetting works, then you shouldn't find it repugnant to have some negatives that are offset. If you find it repugnant, you disagree with offsetting in that case.

I disagree with offsetting suffering – one person being happy does not simply cancel out someone else being victimized – and I figure most people also disagree with suffering offsetting. I also disagree with offsetting in epistemology. Money, as a fungible commodity, is something where offsetting works especially well. Similarly, offsetting would work well for barrels of oil of a standard size and quality, although oil is harder to transport than money so location matters more.

Bonus Error by Upvoters

At a glance (I haven't read it yet as I write this section), the article looks high effort. It has ~22 upvoters but no comments, no feedback, no hints about how to get feedback next time, no engagement with its ideas. I think that's really problematic and says something bad about the community and upvoting norms. I talk about this more at the beginning of my screen recording.

Update after reading the article: I can see some more potential reasons the article got no engagement (too specialized, too hard to read if you aren't familiar with the field, not enough introductory framing of why this matters) but someone could have at least said that. Upvoting is actually misleading feedback if you have problems like that with the article.

Bonus Literature on Maximizing or Minimizing Moral Values

https://www.curi.us/1169-morality

This article, by me, is about maximizing squirrels as a moral value, and more generally about there being a lot of actions and values which are largely independent of your goal. So if it was minimizing squirrels or maximizing bison, most of the conclusions are the same.

I commented on this some in my screen recorded after the upvoters criticism, maybe 20min in.

Bonus Comments on Offsetting

(This section was written before the three errors, one of which ended up being related to this.)

Offsetting views are problematic in epistemology too, not just morality/axiology. I've been complaining about them for years. There's a huge, widespread issue where people basically ignore criticism – don't engage with it and don't give counter-arguments or solutions to the problems it raises – because it's easier to go get a bunch more positive points elsewhere to offset the criticism. Or if they already think their idea already has a ton of positive points and a significant lead, then they can basically ignore criticism without even doing anything. I commented on this verbally around 25min into the screen recording.

Screen Recording

I recorded my screen and talked while creating this. The recording has a lot of commentary that isn't written down in this post.

https://www.youtube.com/watch?v=d2T2OPSCBi4

I proposed a game on the Effective Altruism forum where people submit texts to me and I find three errors. I wrote this for that game. This is a duplicate of my EA post. I criticize Against the singularity hypothesis by David Thorstad.

Introduction

FYI, I disagree with the singularity hypothesis, but primarily due to epistemology, which isn't even discussed in this article.

Error One

As low-hanging fruit is plucked, good ideas become harder to find (Bloom et al. 2020; Kortum 1997; Gordon 2016). Research productivity, understood as the amount of research input needed to produce a fixed output, falls with each subsequent discovery.

By way of illustration, the number of FDA-approved drugs per billion dollars of inflation-adjusted research expenditure decreased from over forty drugs per billion in the 1950s to less than one drug per billion in the 2000s (Scannell et al. 2012). And in the twenty years from 1971 to 1991, inflation-adjusted agricultural research expenditures in developed nations rose by over sixty percent, yet growth in crop yields per acre dropped by fifteen percent (Alston et al. 2000). The problem was not that researchers became lazy, poorly educated or overpaid. It was rather that good ideas became harder to find.

There are many other reasons for drug research progress to slow down. The healthcare industry, as well as science in general (see e.g. the replication crisis), are really broken, and some of the problems are newer. Also maybe they're putting a bunch of work into updates to existing drugs instead of new drugs.

Similarly, decreasing crop yield growths (in other words, yields are still increasing but by lower percentages) could have many other causes. And also decreasing crop yields are a different thing than a decrease in the number of new agricultural ideas that researchers come up with – it's not even the right quantity to measure to make his point. It's a proxy for the actual thing his argument relies on, and he makes no attempt to consider how good or bad of a proxy it is, and I can easily think of some reasons it wouldn't be a very good proxy.

The comment about researchers not becoming lazy, poorly educated or overpaid is an unargued assertion.

So these are bad arguments which shouldn't convince us of the author's conclusion.

Error Two

Could the problem of improving artificial agents be an exception to the rule of diminishing research productivity? That is unlikely.

Asserting something is unlikely isn't an argument. His followup is to bring up Moore's law potentially ending, not to give an actual argument.

As with the drug and agricultural research, his points are bad because singularity claims are not based on extrapolating patterns from current data, but rather on conceptual reasoning. He didn't even claim his opponents were doing that in the section formulating their position, and my pre-existing understanding of their views is they use conceptual arguments not extrapolating from existing data/patterns (there is no existing data about AGI to extrapolate from, so they use speculative arguments, which is OK).

Error Three

one cause of diminishing research productivity is the difficulty of maintaining large knowledge stocks (Jones 2009), a problem at which artificial agents excel.

You can't just assume that AGIs will be anything like current software including "AI" software like AlphaGo. You have to consider what an AGI would be like before you can even know if it'd be especially good at this or not. If the goal with AGI is in some sense to make a machine with human-like thinking, then maybe it will end up with some of the weaknesses of humans too. You can't just assume it won't. You have to envision what an AGI would be like, or what many different things it might be like that would work (narrow it down to various categories and rule some things out) before you consider the traits it'd have.

Put another way, in MIRI's conception, wouldn't mind design space include both AGIs that are good or bad at this particular category of task?

Error Four

It is an unalterable mathematical fact that an algorithm can run no more quickly than its slowest component. If nine-tenths of the component processes can be sped up, but the remaining processes cannot, then the algorithm can only be made ten times faster. This creates the opportunity for bottlenecks unless every single process can be sped up at once.

This is wrong due to "at once" at the end. It'd be fine without that. You could speed up 9 out of 10 parts, then speed up the 10th part a minute later. You don't have to speed everything up at once. I know it's just two extra words but it doesn't make sense when you stop and think about it, so I think it's important. How did it seem to make sense to the author? What was he thinking? What process created this error? This is the kind of error that's good to post mortem. (It doesn't look like any sort of typo; I think it's actually based on some sort of thought process about the topic.)

Error Five

Section 3.2 doesn't even try to consider any specific type of research an AGI would be doing and claim that good ideas would get harder to find for that and thereby slow down singularity-relevant progress.

Similarly, section 3.3 doesn't try to propose a specific bottleneck and explain how it'd get in the way of the singularity. He does bring up one specific type of algorithm – search – but doesn't say why search speed would be a constraint on reaching the singularity. Whether exponential search speed progress is needed depends on specific models of how the hardware and/or software are improving and what they're doing.

There's also a general lack of acknowledgement of, or engagement with, counter-arguments that I can easily imagine pro-singularity people making (e.g. responding to the good ideas getting harder to find point by saying some stuff about mind design space containing plenty of minds that are powerful enough for a singularity with a discontinuity, even if progress slows down later as it approaches some fundamental limits). Similarly, maybe there is something super powerful in mind design space that doesn't rely on super fast search. Whether there is, or not, seems hard to analyze, but this paper doesn't even try. (The way I'd approach it myself is indirectly via epistemology first.)

Error Six

Section 2 mixes Formulating the singularity hypothesis (the section title) with other activities. This is confusing and biasing, because we don't get to read about what the singularity hypothesis is without the author's objections and dislikes mixed in. The section is also vague on some key points (mentioned in my screen recording) such as what an order of magnitude of intelligence is.

Examples:

Sustained exponential growth is a very strong growth assumption

Here he's mixing explaining the other side's view with setting it up to attack it (as requiring a super high evidential burden due to such strong claims). He's not talking from the other side's perspective, trying to present it how they would present it (positively); he's instead focusing on highlighting traits he dislikes.

A number of commentators have raised doubts about the cogency of the concept of general intelligence (Nunn 2012; Prinz 2012), or the likelihood of artificial systems acquiring meaningful levels of general intelligence (Dreyfus 2012; Lucas 1964; Plotnitsky 2012). I have some sympathy for these worries.[4]

This isn't formulating the singularity hypothesis. It's about ways of opposing it.

These are strong claims, and they should require a correspondingly strong argument to ground them. In Section 3, I give five reasons to be skeptical of the singularity hypothesis’ growth claims.

Again this doesn't fit the section it's in.

Padding

Section 3 opens with some restatements of material from section 2 which was also in the introduction some. And look at this repetitiveness (my bolds):

Near the bottom of page 7 begins section 3.2:

3.2 Good ideas become harder to find

Below that we read:

As low-hanging fruit is plucked, good ideas become harder to find

Page 8 near the top:

It was rather that good ideas became harder to find.

Later in that paragraph:

As good ideas became harder to find

Also, page 11:

as time goes on ideas for further improvement will become harder to find.

Page 17

As time goes on ideas for further improvement will become harder to find.

Amount Read

I read to the end of section 3.3 then briefly skimmed the rest.

Screen Recording

I recorded my screen and made verbal comments while writing this:

https://www.youtube.com/watch?v=T1Wu-086frA

---

Update: Thorstad replied and I wrote a followup post in response: Credentialed Intellectuals Support Misquoting