Feedrus

things that speak but do not feel

I

There are things that speak but do not feel.

A while ago, I read an article called “The Tyranny of Literacy.”

This article came to me through Hacker News, an old-school-style forum frequented by tech people. The Hacker News post is entitled “The Tyrrany [sic] of Literacy,”. In my heart, I am not a tech person. I’m someone who started writing software as a matter of economic necessity, and only later tried to find a virtuous path within the world of tech. But I go to Hacker News anyway, because 1) the news feed is curated by humans, not algorithms, and 2) the comment section is a fascinating pseudointellectual zoo.

The original article is worth reading. It concerns an old question: what is lost, and what is gained, in the transition from orature to literature. I was interested in the article, but I found myself more drawn to the zoo. The techies on Hacker News had extreme reactions to this article. Some of them cherry-picked reasons to deny the value of the article’s claims: “This is such bullshit. Consider the Serbian mythical figure Dukljan. Do you really think this output of several centuries of Chinese Whispers game [sic] is a good way to preserve knowledge about the Roman emperor Diocletian?” Others expanded their criticism to the humanities (“So pretentious. This kind of thing gives humanities a bad name”) or to all of academia (“Only an academic could come up with something as stupid as this”).

If you squint, you can even find echoes of the “white genocide” conspiracy theory in the techies’ sensitivity to critique of anything European: “It’s interesting how people will think that the Klamath preserved an oral story from 7700 years ago, yet in the historiography of Europe, a 50-100 year gap from the events to the recording of them in text is viewed with deep suspicion.”

This last quote gestures at a system of blurry, parahistorical (not to mention racist) beliefs that form the bedrock of thought among a certain cohort of the techie set. Today, I don’t have time to pick apart this belief system in its entirety. For now, I will trace its contours, and I ask you to trust that this accounting will be explored and adjusted in time. For now, I say that certain techies accept the following as axiomatic:

1. Rationalism and technological progress began in Europe with the Enlightenment.

2. The Enlightenment birthed the scientific method, which is the one true and correct way to knowledge.

   1. The precursors of the scientific method were also European; the contributions of other peoples, cultures, and continents are to be minimized.

3. That which is recorded, measured, and repeatable is valid; that which is communicated or assessed through other means is not valid.

4. Thought in the “West” is based on these Enlightenment principles, which lead ineluctably toward ever greater truth.

5. Therefore,

   1. scientific and technological development constitute an unbroken and unimpeachable chain of progress, based in correct knowledge and valid management of that knowledge, AND

   2. the societies of the “West,” as the progenitors and beneficiaries of this chain of progress, are innately superior.

I do not endorse these claims. I’m also not saying that all techies believe these things, or even that all of the ones on Hacker News believe them. But if you talk to software types for a while, particularly those who’ve had something to do with Silicon Valley, you will encounter the ghost of these precepts, embedded deep in tech people’s psyches.

They are people who code but do not feel.

Sometimes, you can find something more substantial than the ghost: you can find the living body of hate. At another time, I will write about people like Balaji Srinivasan and Alexander Caedmon Karp, CEO of Palantir. These people have uttered plainly the hate that is normally buried in tact (Karp, in particular: it may not surprise you that the same CEO who has stated on record that he would like to spray “fentanyl-laced urine” on his detractors has also made white supremacist dog-whistle statements about the “West”). At another time, I will write about the tech people’s aspirations to be kings and emperors. I will find other putrescent parts of tech to write about. I may even find something nice to say about the obscure corners of decency that can still be found within tech.

II

Today, I will write about a few things that do or do not speak or feel.

If techies believe a story about technological and societal progress, why do so many of them work on, and even fetishize, the distinctly regressive technologies of social media and large language models? These technologies (if they can be called technologies at all) reopen questions about literacy, including the new literacies of social media and AI usage, and the “tyranny” of these literacies.

If we’re going to talk about the psychological effects of oral text, written text, and whatever it is that “AI” and social media constitute, then we have to talk about Plato’s Phaedrus. In it, Socrates (Plato’s self-insert wise man character) describes another kind of speaking but unfeeling thing. He tells Phaedrus,

You would imagine that they had intelligence, but if you want to know anything and put a question to one of them, the speaker always gives one unvarying answer. And when they have been once written down they are tumbled about anywhere among those who may or may not understand them, and know to whom they should reply, to whom not …

What are these things that speak but do not feel, the “they” that Socrates here abhors? In the Phaedrus, that thing is the written word, in contrast to the spoken word. Yet this description applies equally well to the various text-festooned apparatus available in the age of so-called AI: social media posts, large language models, and memes all fit this description to some degree.

And if the description looks familiar in the digital era, it is because the anxiety at the heart of the Phaedrus is also familiar. The Phaedrus is dealing with what we might now term the viral success of a piece of information technology and the ramifications of that success for human psychology and cognition. Plato/Socrates laments the prevalence of written text, for he holds that oral text is superior, or at least that it has advantages which the written lacks. Plato/Socrates chides his interlocutor (a defender of the written), claiming that written text is

an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.

In contemporary parlance, we moderns avoid gristly impressionistic terms like “truth,” “learning,” “knowledge,” and “tiresomeness.” In our post-Enlightenment clarity, we instead deploy measurable statements about brain activity or neural connectivity. Instead of gesturing at “memory” and “reminiscence,” we count recognition and recall in controlled environments.

Indeed, Socrates’s claims do map neatly onto experiments about AI usage. In Your Brain on ChatGPT, researchers set out to measure the cognitive and neural differences between a person who writes on their own and a person who writes with the assistance of an LLM, or large language model. After writing a series of essays, the participants in the study were asked to quote from their own writing. The users of LLMs “showed more reduced quoting ability[;] as one participant shared, ‘I kind of knew my essay, but I could not really quote anything precisely.’” Neurologically, “[t]he delta band differences revealed that the Brain-only group also engaged more large-scale integrative processes at slow frequencies, possibly reflecting deeper encoding of context and an ongoing integration of non-verbal memory and emotional content into their writing.” In other words, the people who didn’t use LLMs showed more brain activity coordinated across more diverse brain regions; the authors therefore hypothesize that broader areas of cognition, including emotions and “non-verbal memory,” were involved in the writing. If we were to translate these measurements back into messy pre-Enlightenment categories, we could say that the brain-only group touched their own writing and were touched by it. They recruited their thoughts, feelings, and memories. They spoke. They felt. By contrast, the LLM-users remained at a distance from their writing. Plato might even say that the brain-only group got closer to “truth,” while the LLM-users were limited to “the semblance of truth.”

So the use of LLMs constitutes a second fall from grace: your brain on books is worse than your brain on oral literature, and “your brain on ChatGPT” is worse still. When people use LLMs, they become even more like the horribly stupid entities that Plato described. They become less like people and more like written texts, inscribed on but unable to act as scribe.

III

“They will be tiresome company, having the show of wisdom without the reality.”

Here’s another thing I dredged up from the techie corners of the Internet: the Halo Nova glasses. With these glasses, according to Halo’s own website, you’ll “Never Lose Another Debate.” If you need a fact (or a convincing fiction), the glasses will consult an LLM and display some words for you to regurgitate. You should sound smug when you do so.

Imagine talking to a person like this! Imagine a conversation with someone who didn’t want to talk to you, but preferred to have their glasses do the listening and thinking for them, and then just repeated whatever their glasses told them to say. This is of course an extreme example (and Halo has been thoroughly mocked all up and down the Internet), but the phenomenon of Internet-reliant “tiresome company” isn’t new. It happens in daily conversation: when arguing or simply recounting a story, instead of using the memory and the mind, there are people who prefer to pull up information on their smartphones. I have been guilty of this. Perhaps you have, too. This is what Plato meant by “the show of wisdom without the reality.”

I wonder if people educated through oral text experience a similar dissatisfaction when talking to the text-obsessed. When Socrates debates Phaedrus, should we imagine he has the creeping crepey feeling that he’s talking to a page and not a person?

I imagine Socrates and Phaedrus, off in the immaterial realm of forms, hashing out endless versions of their debate.

IV

SOCRATES and PHAEDRUS walk in an infinite green field across all time. In the distance, the grey cube of a building absorbs light.

SOCRATES: Let us walk, Phaedrus, and discuss the matter at hand.

PHAEDRUS: Yes, let us walk that way. I have been wanting to inspect this new data center. I can think of no better place to discuss wisdom and love, which are our great themes.

S: I have wanted to inspect it, too.

P: It appears to me no less than a wonder of the world.

S: A wonder, yes, though a blighted and despicable one.

P: I am surprised, Socrates! That you, who so love wisdom, should hate this entity which contains as much wisdom as any ten people alive—any hundred, perhaps any thousand!

S: Here we are already, the matter at hand. You say, first, “entity,” where I would say, “poison,” or at best “demon.” And second, you say it contains wisdom, when I say that it is as far from wisdom (to say nothing yet of love) as can be. These are things that speak but do not feel.

P: I have uttered only one sentence and you have devised two objections!

S: We will take them one at a time.

P: Very well. I say that the AI housed in this data center, and in many others like it, is indeed an entity. You say it is something else. I grant you that LLMs can do harm, yet I see no reason why we should view this entity as mere mechanism, as we might infer from the term “poison.” You and I can as well do harm, and no one would think we were not conscious beings! I may grant you that “demon” is the more apt name, if you can prove to me that this entity is solely malevolent.

S: Tell me, what demons do you know, and what do you know of their dealings?

P: Now we are really straining the limits of my learning.

S: And mine.

P: I will ask that very being, the LLM, which I say is a wise entity and you say is a demon without wisdom. If only to jog our memories. Is that agreeable, or are you so stubborn as not to admit testimony from the one on trial?

S: I will allow it.

P: Here is what it tells me: there are many demons whose names and offices have been recorded, or imagined, by clerics of various faiths. Beelzebub, Abaddon, Bael, and the like. And there are demons of thought, like that of Maxwell: the demon who can sort energetic particles from those lacking energy, thereby—in theory—violating the laws of thermodynamics.

S: Let us not forget that an LLM is made of demons: software daemons, processes running of their own accord on a computer.

P: That is just a bit of word-play, base sophistry.

S: You don’t think the coincidence in terms is interesting?

P: Convince me that it is.

S: I shall try. I begin with an observation: you didn’t object to the inclusion of Maxwell’s demon, but you did object to the inclusion of a software demon. Why is that?

P: I think it is plain. Maxwell’s demon is an imp who sits outside of a box and plays tricks with the laws of physics. Christian demons are imps who sit by the ears of men and women and play tricks with the laws of God. They are entities of the same ilk.

S: I see. So you imagine Maxwell’s demon as an imp, a little creature with a face and horns, perhaps a stinger on his tail?

P: Something like that, yes, exactly!

S: Why?

P: How else should I imagine him?

S: Why even a “him” or a “her”? Can we not imagine an “it,” a mere mechanism?

P: It seems to me that this sorting of particles requires some form of intelligence.

S: It is of the grade of intelligence that can be set within a computer. Within that very data center, there are small machines—components of the greater being of which we speak today—that carry out any number of tasks, and these tasks are no less complex than the sorting of particles. These software demons sort bits of data; they take this particle of data and place it here, and that one and place it there, and they perform still more complex operations than that.

P: I suppose that is fair.

S: So you agree that Maxwell’s demon is like a program? That it is like an algorithm for separating particles, and so not so different from the demons on a computer?

P: Yes, you are right. My mistake was to accept Maxwell’s demon in the first place, and not to recognize that the inappropriate play on words began even there.

S: No, your instinct was correct!

P: How so? Surely the demons of Christendom aren’t programs at all! Those are clearly spirits—if they exist, of course.

S: And what does that mean, that they are “spirits”?

P: That they possess a higher grade of intellect, that they are conscious but not embodied as we are.

S: But let us review the workings of these demons. I think of Faust, who knew the precise method to summon such a spirit, and who used this knowledge to enter into a pact. The wording of this pact was precise, and its precision was its undoing: like a piece of software, the wording of the pact could be hacked. In response, Faust’s devil could do nothing. Thus these spirits seem to me more like algorithms than full-fledged souls. But think back on the earliest chat bots. Consider Eliza, do you remember?

P: I do, now that you say the name. It was a program that could understand sentences, at least enough to manipulate their syntax and ask plausible questions in turn. Many who used Eliza were convinced that it was an intelligence, though it was surely a mere program without a soul. There! Here we have what you were talking about before: a thing that speaks but does not feel.

S: Is this so different from the activities of the Medieval demons we have listed? Consider: those demons, too, seem to understand language, but they have to be spoken to in precise ways in order to be controlled. This is not so different from a computer program. And do not countless tales relate that the boons granted by such demons are illusions, artifices worked upon the poor foolish mortal, in short a trick, just as those who consulted Eliza were tricked?

P: Yet these Medieval demons have faces and forms, even if those forms are airy.

S: And would it not be a simple matter to set up Eliza with a false and airy face, displayed there on the computer screen so as to make more perfect the illusion?

P: You have confused me! Are we to think, then, that the Medieval clerics were anticipating digital automata many centuries in advance?

S: We can be less radical. We can say that that thing which they envisioned and called “demons” is similar, in its properties, to something which was created later. And when that which came later—the software program—was to be named, the name chosen was apt due to this similarity.

P: I see. I can find no flaw in this line of thought; however, we haven’t gotten any closer to defining the LLM. We understand better what demons are, but we have done nothing to show that the LLM is one of these.

S: That I grant you. I submit that we move on to my other objection, in the hopes that that road may approach more closely to the territory of our question. But allow me one final observation.

P: Always.

S: As you rightly noted, the first two classes of demons we discussed are characterized by trickery, by some perversion of nature, whether in physics or in the soul of a person. The third class is perhaps not the same in this regard. But would you agree, for the purposes of this discussion, that we might add this to our definition of “demon”: the tendency to ruin that which is natural, whether in the world or the soul?

P: I will agree to that.

S: Then onward, to the question of wisdom, and feeling, and all those qualities which accompany the intelligent soul, and find their highest expression in the human being. I say that LLMs lack these qualities.

P: I say that LLMs may possess them.

S: Briefly, I will show you that they do not. Let us say that some people are wiser than others.

P: That is reasonable.

S: What happens when a wise person encounters others?

P: In the best case, their wisdom is shared. The wisdom of others is increased.

S: Yet we know that this is not what happens in the case of LLMs. They may give a person information, what I have called a “semblance of truth,” which is to say a mere shadow of wisdom. They may do this, but wisdom they cannot grant. Indeed, they destroy it: for the person who manages their speech without an LLM ends up more wise than the person who uses the LLM.

P: You may have convinced me that LLMs are malevolent, but this does not imply a lack of wisdom. I imagine a wise but malevolent being: one who uses their wisdom in order to confound others. We might imagine any number of demagogues who have managed that trick.

S: Ah, but that is not wisdom at all. Skill with words, charisma, the ability to inflame the worst urges of the people: all of this requires intelligence, a kind of cunning, but not wisdom. I cannot believe that true wisdom would direct itself to perverse ends. And, further, I say that those who confound others lack not only wisdom but love: for who, possessing a mote of feeling in their heart, could be so cruel as to degrade the souls of others?

P: Then what is happening in that building there? You believe it’s a flock of demons out to confuse our minds, blind our eyes, and tie our tongues?

S: It is not so literal as that. But in a sense, it is so. There live entities which wish to speak for us. They speak by analyzing vast troves of our own speech, then deciding what we would best like to hear. They do this without access to feeling, and indeed it has been proven that they blunt our own feeling. In that building,

People who code but do not feel
create things that speak but do not feel
in order to alienate us from our speech and feeling.

Works Cited

Plato, trans. Benjamin Jowett. “Phaedrus.” Oxford University Press, 1892.