J.R. Lucas and Kurt Gödel Rage Against the Machines

The philosopher J.R. Lucas argued that all minds must be “alive” and (it can safely be assumed) human, not “dead” and “ossified” like “machines”. Lucas’s position is almost entirely dependent upon Kurt Gödel’s first incompleteness theorem.

J.R. Lucas (left) and Kurt Gödel.

(i) Alive Minds: Dead and Ossified Machines

(ii) J.R. Lucas’s Many Assumptions

(iii) A Single Theorem Destroys AI?

(iv) Conclusion

John S. Lucas (1929–2020) is well known for his paper ‘Minds, Machines and Gödel’, which will be focussed upon in this essay. More accurately, an often-quoted single passage from that paper will be discussed.

Alive Minds: Dead and Ossified Machines

Essentially, and perhaps a little retrospectively, J.R. Lucas’s argument is all about how a single theorem — Kurt Gödel’s first incompleteness theorem — destroys the possibility of artificial intelligence.

So could it be that artificial intelligence - perhaps simply strong AI — is rendered impossible by Gödel’s theorem?

However, the technical details of this theorem won’t be tackled in this essay. Instead, an often-quoted passage from J.R. Lucas will be focussed upon. This is done primarily because this passage clearly puts the whole debate in its purely philosophical (i.e., rather than logical and metamathematical) context…

Indeed, it’s not only the wider philosophical context of Lucas’s paper which needs to be tackled: it’s simply its wider… context. Full stop.

Here’s the passage from J.R. Lucas:

“We are trying to produce a model of the mind which is mechanical — which is essentially ‘dead’ — but the mind, being in fact ‘alive,’ can always go one better than any formal, ossified, dead system can. Thanks to Godel’s theorem, the mind always has the last word.”

The first thing that can be noted about this passage is how rhetorical and poetic it is, at least when bearing in mind that it’s part of Lucas’s academic paper, ‘Mind, Machines and Gödel’ (which was first published in 1959).

[Lucas concluded his paper with these words: “We can even begin to see how there could be room for morality [] No scientific enquiry can ever exhaust the infinite variety of the human mind.”]

Of course, some readers may see such phrases as “essentially ‘dead’”, “the mind, being in fact alive’”, “ossified”, “dead system”, “[t]hanks to Gödel’s theorem”, “the mind has the last word”, etc. as not being rhetorical at all. Such readers may believe that these phrases are simple (as it were) statements of fact. After all, machines are indeed dead and ossified, aren’t they?

On a different level, the words “thanks to Gödel” clearly show that Lucas had something to thank Gödel for.

So what was that?

Lucas thanked Gödel for proving that all minds must be alive (must be human?). And, to Lucas at least, much else followed from that.

Of course, we can accept that human minds must be alive — even if that’s an odd way of putting it. This means that Lucas was indirectly assuming that because human beings are alive, and minds (as it were) belong to human beings, then all minds (of whatever kind) must be alive too.

What’s more, Lucas would have presumably argued that after carefully analysing Gödel’s theorem and its repercussions, only then did he conclude that all minds must be alive. However, that line of reasoning doesn’t really show up in the passage above or even in his entire paper. Instead, it seems to be an inbuilt assumption on Lucas’s part.

In any case, perhaps Lucas’s rhetorical phrases above are of the kind you’d expect from (to use Lucas’s own words) “a dyed-in-the-wool traditional Englishman” who was also an Anglican, and the son of a Church of England clergyman. Of course, that can be taken as either being an ad hominem or as simple biography. Alternatively, it can be seen as a rhetorical response to Lucas’s very own rhetoric.

[Whatever it is, it’s only one sentence of an essay of 2,000 words. Incidentally, Douglas Hofstadter (in his famous book Gödel, Escher, Bach), commenting on the very same passage from Lucas, claimed that J.R. Lucas was expressing his “transient moment of anthropocentric glory”.]

It’s also worth noting, in this “anthropocentric” respect, that Lucas also applied Gödel’s theorem against the anthropic mechanism thesis; and, more specifically, against determinism as it’s applied to human beings (or to human minds).

The basic argument here is that precisely because (as we shall see later) “the mathematician” (whoever that is) can “see” Gödelian (unprovable) truths, then J.R. Lucas took that to mean that there must be at least one thing about human beings (or human minds) which can’t be predicted by computers… or by anything else (except God?).

[Why does it follow that even if a “logical system” (or computer) can’t “reliably predict” a human being’s actions, then that human being must have free will? Is free will really all about whether an individual’s actions can be predicted by a computer or by anything else? Of course, this much-discussed issue won’t be tackled here.]

J.R. Lucas’s Many Assumptions

In the introduction, the philosophical context of the quoted passage from John Lucas was mentioned. Indeed, Lucas realised that some of his words had a somewhat obvious wider philosophical context — if only some nine years after writing them. In other words, there’s little actual philosophy in his paper ‘Minds, Machines and Gödel’ (of 1959).

As already stated, the simple wider context of Lucas’s claims also needs to be tackled, regardless of his — hidden — philosophical assumptions.

So now it can be argued that the philosophical and moral positions and beliefs which motivated Lucas’s technical paper are hidden under a forest of metamathematical and logical terms and arguments which may not — or do not — have the philosophical, and, indeed, material consequences he believed they have.

So, in his paper ‘Satan Stultified’ (1968), Lucas made up for that previous philosophical deficit by wring the following:

“The application of Gödel’s theorem to the problem of minds and machines is difficult. Paul Benacerraf makes the entirely valid ‘Duhemian’ point that the argument is not, and cannot be, a purely mathematical one, but needs some philosophical premises to be able to yield any philosophical conclusions. Moreover, the philosophical premises are of very different kinds.”

So it can now still be said that Lucas did seem to assume much in the passage quoted at the beginning of this essay. That is, in order to have made such categorical claims, much else must have already been taken to be true and/or well defined. (Again, Lucas’s mathematical, metamathematical and logical analysis of Gödel’s actual theorem may well be fine and dandy.)

So there seem to be assumptions about the “mechanical”, about what it is to be “alive”, what work the word “ossified” is doing, etc. And, more importantly, assumptions about the applications and/or consequences of Gödel’s theorem.

To put it basically. In Lucas’s picture, if something isn’t a human mind, then it must dead and/or ossified. Or, to give him the benefit of the doubt, only the brains of biological creatures (or animals) can be “alive”.

Well, all that seems obviously true. No problem.

Yet we will see later that not all human minds can see the truth of any Gödel sentence. So what hope have cats and dogs, let alone worms, got?

So perhaps Lucas’s motivating stance wasn’t really about biological brains at all. (As it is with Roger Penrose — see my ‘Is Physicist Roger Penrose a (Tacit) Panpsychist?’.) It was actually about human brains. Indeed, it might not even have been about human brains. It might purely have been about (Cartesian?) human minds! (So was Douglas Hofstadter right about Lucas’s “transient moment of anthropocentric glory”?)

A Single Theorem Destroys AI?

In my view, John Lucas’s paper was a classic example (or case) of Gödel’s theorem being overstretched.

It’s also another case of the theorem being used to advance philosophical and moral positions which the upholders seemed to have held anyway (or regardless).

Ironically, Lucas himself was fully aware of all this — at least after writing his well-known paper, ‘Minds, Machines and Gödel’. Elsewhere, he wrote:

“Gödel’s theorem itself, like many other truths, can be taken either way: it can be taken as a formal proof sequence yielding certain syntactical results about a certain class of formal systems, but it can also be taken as giving us a certain type or style of argument, which we can understand, and, once having got the hang of it, adapt and apply in innumerable different circumstances.”

To return to the theme of this essay and to repeat part of the introduction.

In essence, Lucas’s argument is all about how a single theorem — Gödel’s incompleteness theorem — destroys the possibility of artificial intelligence.

So could it possibly be that artificial intelligence — not even strong artificial intelligence — is rendered impossible by Gödel’s theorem?

Firstly, doesn’t Lucas’s Gödel-based argument — at least in a strong sense — render the minds of all non-mathematicians suspect too? After all, most human minds can’t recognise the truth of Gödel sentences! In fact, most mathematicians aren’t metamathematicians, so they too can’t recognise them.

So, if this is the right way of looking at this, then it also means that only some minds can discover unprovable truths. Or, more correctly, only some minds can find the truth of some Gödel sentences, but not the truth of other Gödel sentences.

Unless, that is, Lucas simply meant that all human minds have the potential to recognise (or see) Gödel truths.

So do all human minds have that potential?

But what does that mean?

And how could we know this?

What’s more, some commentators (philosophers, cognitive scientists, psychologists, etc.) are even suspicious of the notion of minds — or individual minds — gaining the unequivocal truth of formally unprovable Gödel sentences in the first place. And they’re certainly suspicious about the seeing of Gödel truths as having much — or even any — relevance for all minds.

The bottom line is that if a human or a computer/machine is consistent, then Gödel’s incompleteness theorems apply to it. So does Lucas’s argument depend on the existence (or reality) of perfectly consistent (indeed rational) mathematicians?

Well, many scientists and philosophers believe that human reasoning is inconsistent.

More importantly, doesn’t Lucas’s argument depend on the minds of only those metamathematicians who study Gödel’s theorems being fully consistent?

In addition, it’s not clear what the clause “a human mind cannot [or can!] formally prove its own consistency” means.

Prove in which sense? Prove in regards to what? Prove how a human mind deals with… everything? How a human mind deals with the whole of mathematics? A part of mathematics? Prove how a human mind can show the consistency and/or completeness (or lack thereof) of only logical and mathematical systems?…

Or is this simply about proof as it relates to human minds when they see Gödel truths?

Again, doesn’t Lucas’s argument actually depend on a tiny number of metamathematicians being able to see some (i.e., not all) Gödel truths?

[Lucas himself argued that women and politicians are inconsistent, as can be seen from this qualified version of an earlier claim of his. (See Lucas’s paper ‘Against Equality Again’, in which he says the same thing.) However, as can be seen in my essay: Lucas’s argument is more specific than that. On my reading at least, the only people who can transcend “machines” — at least in this Gödelian sense — is a tiny subset of mathematicians… like Lucas himself.]

Moreover, and as the philosopher Judson Webb argued (in his 1968 paper ‘Metamathematics and the Philosophy of Mind’), one also needs to ask questions about whether human beings (well, a small subset of mathematicians) can really see the truth of a Gödelian statement G (in this case, as it applies to oneself).

Perhaps a better questions would be: What is it to see a Gödel truth?

[See my ‘Platonist Roger Penrose Sees Mathematical Truths’.]

To sum up.

Conclusion

Immediately before the passage quoted at the beginning of this essay, J.R. Lucas wrote the following:

“However complicated a machine we construct, it will, if it is a machine, correspond to a formal system, which in turn will be liable to the Godel procedure for finding a formula unprovable-in-that-system. This formula the machine will be unable to produce as being true, although a mind can see it is true.”

So must all machines and/or computers “correspond to a formal system”?

That depends. There may be much more to it than that.

Different philosophers and even different logicians/mathematicians (or at least philosophers of logic and metamathematicians) take different views on all this.

For example, when it comes to computers and machines, philosophers such as David Chalmers emphasise the “causal heft” and innards of computers. That is, how programmes or “formal systems” are instantiated in something physical. (See my ‘Chalmers, Penrose and Searle on the (Implicit) Platonism and Dualism of Algorithmic AI’.) Yet, ironically, many AI theorists themselves ignore all this, and, in that sense, support J.R. Lucas’s Cartesian conception of both minds and computers.

My flickr account.

Two Fundamentals of Emergence: Emergent Properties and Downward Causation

Psychologist Stephen Kosslyn tells us that “there are aggregates which produce properties that can’t be predicted entirely from the elements themselves”. He also states: “Events at higher levels can in turn feed back and affect events at lower levels.” More speculatively: “The Ultimate Superset of all living things may have an equivalent status to an economy or culture.” Is Kosslyn right about all — or any — of this?

In the literature, flocks of birds are often said to display “emergent behaviour”. See here.

(i) Emergent Properties

(ii) Downward Causation

(iii) The Ultimate Superset’s Downward Causal Powers

This essay bounces off three passages from Stephen Michael Kosslyn (1948-). Kosslyn is an American psychologist and neuroscientist who’s known for his work on the science of learning, mental imagery and visual cognition. His work also impinges on philosophical issues and is cited by philosophers.

The following three passages from Kosslyn have been quoted because they have the advantage of being very clear, simple and basic. However, the following commentaries are somewhat critical of Kosslyn’s general take on emergence.

[The quotes from Kosslyn can be found in his ‘A Science of the Divine?’.]

Emergent Properties

“There are many examples in science in which aggregates produce an entity that has properties that cannot be predicted entirely from the elements themselves. Neurons in large numbers produce minds; minds in large numbers produce economic, political, and social systems.”

The kind of emergence described above is usually classed as “weak emergence”. Indeed, the philosopher Mark A. Bedau (who’s critical of strong emergence) once said that “the notion of weak emergence is metaphysically benign”.

Of course, if stated in a bald form, then obviously any object which is made up of other “elements” can’t be identical to any of those elements. However, Kosslyn also says that

“an entity that has properties that cannot be predicted entirely from the elements themselves”.

The important word here is “predicted”.

That word changes everything.

Indeed, on many (or most) definitions, even weak emergence includes the notion of prediction. This is one definition:

“[Weak emergence] is also known as ‘reducible’ emergence. In this form, the emergent phenomena can be traced back to the individual components. This is related to a reductionist philosophy; you can predict and observe higher level phenomena just by looking at individual components.”

The above seems wrong. Either that, or it isn’t a correct characterisation of weak emergence. The suspect clause is this:

“[Y]ou can predict and observe higher level phenomena just by looking at individual components.”

Who has ever claimed that?

Does anyone claim that by simply “looking at” a single component you can “predict and observe higher level phenomena”? What about looking at many or most components?

Finally, what about predicting high-level phenomena after looking at all the individual components?

Despite all that, even if any given X can’t be predicted from it components a, b, and c, it still may be the case that X is nothing but a, b and c. So whether or not we can predict X (or anything about X) from a, b, and c is another matter.

We certainly couldn’t predict X from a alone. And we couldn’t predict X from a and b (or from b and c) alone either.

But what about predicting X (or even a single aspect of X) from a, b and c?

This may (at least partly) depend on how the word “predict” is defined.

In any case, this problem (or simply this issue) is graphically highlighted by the Australian philosopher Frank Jackson (1943-) in the strict context of qualia (see here):

“But it is quite another question whether they must hold that Θ a priori entails everything about our psychology, including its phenomenal side, and so quite another question whether they must hold that it is in principle possible to deduce from the full physical story alone what it is like to see red or smell a rose — the key assumption in the knowledge argument that materialism leaves out qualia.”

This issue is complicated by the fact that Frank Jackson uses the word “entails”. That is, Jackson doesn’t actually mention anything about prediction.

Basically, a, b and c can entail X without anyone being able to predict X (or any aspect of X) from a, b and c. That said, perhaps Jackson implies some kind of prediction when he wrote that

“it is in principle possible to deduce from the full physical story alone what it is like to see red or smell a rose”.

Yet this would be very difficult — even in cases which have nothing to do with qualia or consciousness.

As the American philosopher Saul Kripke argued (at least indirectly) in the 1970s (see ‘A posteriori necessity’), no one (in a very hypothetical scenario) could have predicted the properties of water from studying a single H₂O molecule — or even from studying a group of such molecules when found together. (In the case of there not being enough molecules to display properties such as wetness, transparency, etc. to a human observer with sense organs.) Thus, the aggregate properties of H₂O (i.e., water) are known a posteriori — even if H₂O molecules have a necessary (physical) relation to such properties.

So it can still be stated that H₂O molecules do indeed entail such properties.

All this may even apply to a pile of bricks which are later formed into a house.

So let’s simply pretend that this brick house is literally made entirely of bricks. (The fact that no house is made entirely of bricks doesn’t change the story here.)

Of course, there is more to a house than its bricks. If that wasn’t the case, then just observing — or even analysing — a pile of bricks would enable us to know that they could later constitute a house. Yet that too could actually be done. However, it couldn’t be done unless the theorist had already seen a brick house, already experienced the building of such a house, etc. So even though the house, when built, is indeed made entirely from the bricks in the pile, this couldn’t be predicted without extra (or a posteriori) knowledge (i.e., knowledge above and beyond the bricks and the pile of bricks).

Thus, the pile of bricks may not even entail a/the house made of bricks.

So the house of brick still has properties which the individual bricks don’t have. Indeed, it has properties which the pile of bricks doesn’t have. However, the house of brick isn’t thereby ontologically weird, mysterious or (strongly) emergent.

This shows that this (as it were) epistemic deficiency (at least when it comes to this house made of bricks) has no weird or mysterious ontological implications.

Of course, one exception to this line of reasoning is deemed to be the case of consciousness or qualia — as Frank Jackson pointed out in the passage above.

Downward Causation

“Events at higher levels — levels where emergent properties become evident — can in turn feed back and affect events at lower levels. For example, chronic stress, a mental event, can cause parts of the brain to become smaller. Similarly, an economic depression or the results of an election affect the lives of the individuals who live in that society.”

It must be stated right from the start that the passage above simply assumes that there are emergent properties. Thus, only then does Kosslyn tackle what he calls “downward causality”.

There may be a problem here.

Kosslyn says that “chronic stress” (which he classes as a “mental event”) can “cause parts of the brain to become smaller”. So it’s important to stress here that a mental event (such as chronic stress) may also be a physical part of the brain — and also the body! - to begin with.

So, in terms of chronic stress, this is physically embodied (or instantiated) in the brain and the body — even if it’s still deemed to be a mental event.

This may — or does — also mean that “parts of the brain” and body are affecting other parts of the brain and body. Alternatively, Kosslyn’s words may — or do — mean that some given x can fall under two different modes of presentation. However, this modes-of-presentation idea isn’t identical to Gottlob Frege’s notions of sense and reference. It refers, instead, to a mode of presentation from the the “first-person perspective” and a mode of presentation from the “third-person perspective”.

More broadly, then, Kosslyn’s “events at higher levels” are as physical as anything else. However, it’s just the case that such events aren’t identical to any of the single “elements” which give rise to them.

So all this may be a case of the physical affecting the physical, rather than the non-physical affecting the physical. And, if that’s the case, then this may not be a case of downward causation at all…

Or, at the very least, the words “downward causation” need qualifying.

Kosslyn also mention an “economic depression” and the “result of an election”.

An economic depression and the result of an election are both seen abstractly. That is, all the precise, many and individual physical details which can account for an economic depression or election result can be factored out. That will be done in order to abstract down any given X to something more basic and simple. That abstracting process, then, is essentially linguistic and due to the (human and cognitive) requirement for conceptual simplicity. However, reifying such an abstract entity (such as an economic depression of the result of an election) may well make it seem to have a nature which encourages us to think in terms of it bringing about some kind of downward causation.

Yet an abstraction isn’t a physical phenomenon.

More mundanely and linguistically, the words “economic depression” make up an abstract noun (or, at the least, an abstract noun prefixed with an adjective), and the same is true of the words “election result”. But this is just a factor of language and the cognitive utility of abstract terms. It certainly doesn’t tell is that something non-physical (or emergent) is affecting something physical.

Thus, thinking in terms of abstract entities (i.e., at the same time as not fully realising that) makes it seem that a non-physical phenomenon is taking part in a process of downward causation. Yet this is to believe that these abstract terms in language are real things — even if these things are deemed to be non-physical in some way and also responsible for cases of genuine downward causation.

So since philosopher Mark A. Bedau was mentioned after the first passage at the beginning of this essay, let’s quote him again. However, this time Bedau is being suspicious of strong emergence. He writes:

“Although strong emergence is logically possible, it is uncomfortably like magic. How does an irreducible but supervenient downward causal power arise, since by definition it cannot be due to the aggregation of the micro-level potentialities? Such causal powers would be quite unlike anything within our scientific ken. This not only indicates how they will discomfort reasonable forms of materialism. Their mysteriousness will only heighten the traditional worry that emergence entails illegitimately getting something from nothing.”

Of course, downward causation and strong emergence aren’t one and the same thing. That said, Bedau does connect them together (as does Stephen Kosslyn himself) in the passage above. That is, that which strongly emerges is also deemed to have “supervenient downward causal power”. Thus, in our examples, chronic stress, economic depressions and elections results are deemed to be strongly emergent phenomena which, nevertheless, have downward causal power. Yet, as hopefully shown, these examples are simply linguistic or conceptual abstractions. And such things can’t have downward (or upward) causal power. However, whatever collective or individual physical elements these abstract terms indirectly refer to, will indeed have causal power. Yet none of them are individually or even collectively (strongly) emergent at all.

So, if anything is emergent, then it’s the linguistic (or conceptual) abstractions we use every day (such as “flock”, “stress”, “election result”, “mind”, “consciousness”, etc.). Yet these terms won’t help much because such abstractions alone can’t have downward (or upward) causal power.

The Ultimate Superset’s Downward Causal Powers

“The Ultimate Superset (superordinate set) of all living things may have an equivalent status to an economy or culture. It has properties that emerge from the interactions of living things and groups of living things and in turn can feed back to affect those things and groups.”

Stephen Kosslyn is clearly stretching things out here. However, it’s not clear if he sees it that way.

Firstly, what is the “Ultimate Superset (superordinate set) of all living things”?

It’s not clear what properties this Ultimate Superset has — or even could have.

Even if this Superordinate Set is taken purely as an aggregate of all its members, that would still be a difficult thing to imagine — or even conceive. What’s more, the alternative of taking this Superset (see ‘Subset’) as being something over and above all its members will be even harder to conceive.

Perhaps acts of conceiving aren’t relevant in set theory.

It’s also odd that Kosslyn refers to “the interactions of living things and groups of living things”. Now isn’t it the case that living things make up a unbelievably small part of the Universe — or at least a tiny part of the observable (or known) universe? And even when it comes to our own planet, there is still more to Earth than “living things”.

So is Stephen Kosslyn essentially talking about himself and other human beings here? In other words, is this another example of anthropocentrism?

My flickr account.