AI systems have moved from mining data and facts to winning arguments. This essay describes the work done to achieve this.
Douglas Adams’ Reason Software
To get the point across about AI and argumentation, Gilead Amit (in ‘Computers that can argue will be satnav for the Moral Maze’ for the New Scientist) mentions Douglas Adams’ novel Dirk Gently’s Holistic Detective Agency. This novel features computer software called Reason, which is used for nefarious purposes by the Pentagon. (How predictable.) Amit concludes:
“We’re not quite there yet. Machines have beaten us at logical games like Go and games of bluff and chance like poker. But no computer has ever come close to beating humans where it counts: in an argument.”
It is that last sentence which stands out. Which adult humans have computers never beaten in an argument? All adult humans? Really? Amit wrote these words in 2016. So, to give him the benefit of the doubt, AI has moved on tremendously in the last ten years or so. That is said because chatbots nowadays can indeed win arguments with adult humans. Of course, it may depend on what the word ‘win’ means here. In other words, sometimes it’s hard to say just who has won, or even if winning is always important.
We can bring in the cognitive scientists Hugo Mercier and Dan Sperber here. (Specifically, their “argumentative theory of reasoning”.) Sperber and Mercier argue that human reason didn’t evolve to help us find “the solitary truth”, but rather as a social tool to persuade others, as well as to evaluate the arguments made by others.
To be a sceptic about AI for a moment. If AI systems argued/debated to win, then surely that would be a bad thing. Indeed, Douglas Adams’ Reason software would have been made flesh. This is the reality behind this fictional software:
- Decide the conclusion: You specify in advance what conclusion you wish to reach.
- Find the facts: You feed facts into the program.
- Generate arguments: The software constructs a series of logical steps to connect the premises to your conclusion.
On a related theme. Amit quotes the computer scientist Noam Slonim:
“‘A lot of the questions we encounter in life are not factual. Questions where there is no one clear answer.’”
This seems to apply to winning arguments too.
Amit also tells his readers that a “full-blown argument machine sounds as implausible as Douglas Adams’ tongue-in-cheek Reason software”. He then concluded:
“It seems hard to believe that anyone would trust a machine to tell them, say, how to vote, or suggest what they should think about a certain issue.”
Why is Amit assuming that a machine will literally tell people how to vote? Surely it will (or it should) only offer arguments as to why someone should vote in a particular way. It needn’t literally tell them. Amit himself sees this when he then uses the word “suggest” (as in “suggest what they should think about a certain issue”) in a slightly different context.
Where To Find Arguments
It may be surprising to find out where certain AI researchers have gone to find “good arguments […] to use to train an AI to argue as we do”. In this particular case, the computer scientist Chris Reed’s “favourite source is the BBC radio show Moral Maze, in which panellists debate the ethics of an issue of the day”. Why that show? It’s because of its “quasi-legal cut and thrust, laced with appeals to our emotions”. It’s not the case that Reed has become completely reliant on the BBC’s Moral Maze, only that he has analysed its debates “to build a general framework for the essence of human argument”.
In terms of detail. It was in July 2012 that Reed’s team “performed their first real-time argument analysis, of a Moral Maze episode on the ins and outs of British colonial rule in Kenya”.
The researchers found good sources elsewhere too. They went to
“online forums [which] contain surprisingly well-structured arguments, despite users being inclined to wear their heart on their sleeve”.
There are many well-structured arguments in some online forums. More so than in many national newspapers or in the House of Commons… Yes, these researchers tried the UK parliament too. Amit says that they are “not good reference material: too much performative swagger and too many procedural interjections and references to previous debates”. Chris Reed himself says: “There is much less quality argument there than you would either expect or hope.”
Why did Reed expect better arguments in the House of Commons?
Whether it’s online debates or debates in the Commons, readers may be wondering about the actual processes used to extract good arguments from all this data. Not much detail is given about this by Amit. What detail he does give is that “[c]laim and counterclaim, and the connections between them, were represented on a giant touchscreen in a form ready to be fed into an AI”.
That’s a little bit helpful, but not much. The reference to “connections between them” certainly needs to be fleshed out. As for the touchscreen, it had “argument maps” on it.
The same point can be made about Ivan Habernal and Iryna Gurevych and their research team. In this case:
“In 2016 they asked nearly 4,000 people to say which of two arguments, each making the same case in different ways, they found the more convincing — and to explain why.”
Again, that’s not very helpful, and neither is Amit’s account of what happened to all this data:
“Over 80,000 responses later, they now have a database that can be used to teach computational systems to rank the arguments they process, and so argue more convincingly.”
How does all that data teach computational systems to rank the arguments they process? How does the AI extract the argument (or argument forms) from all the sentences which include (what Amit calls) “claims”?
Arguments, Not Facts
You can have all the data in the world, and even data that includes human arguments, but readers still need to know what the researchers did with all that. Prima facie at least, it seems like the AI simply represented the arguments and their relations (as well as the relations between premises and conclusions) in a very similar way to how it represents facts and other data.
This is about whether a machine can “graduate from facts to arguments”. Yet aren’t there facts (perhaps loosely construed) about arguments — or at least about argument forms?
Amit knows that in order for
“[a] machine [to be] able to formulate an argument — not just search [for] information [it must] also synthesise [that information] it into more or less reasoned conclusions”.
According to Amit, this would be a “research engine”, not a search engine. That said, a machine can just as easily search for arguments, as Amit shows in his article. After that, the machine can then synthesise those arguments — whatever that may mean.
What Is an Argument?
Amit says that an argument is a “claim backed up by evidence”. That’s a very odd definition of an argument. Even if we focus on arguments in the plural, it’s still an odd definition. (This seems to be an traditional account of a scientific process than a definition of an argument.) However, Amit does say that it’s a “rough answer”. So perhaps it’s only odd to those who’ve read too much philosophy of logic.
The problem with this definition is also one of the problems Amit’s article attempts to confront: the move from fact to argument. After all, an AI has easy access to much evidence, just as it has easy access to many facts. Evidence is never conclusive: it is what is done and said with the evidence. This must apply to what is claimed too.
Amit does add more to his definition. He informs his readers that “the team hopes to shift their focus to flagging up evidence that supports the claim”. So how is the claim “All swans are white” flagged up with evidence which supports it? And is any of this constitutive of an argument?
In terms of evidence. Amit distinguishes “anecdotal data” from “expert testimony”. The aim here is to make an AI “learn how much weight to give [these] different forms of evidence”.
Let’s now give more details on some of the big names in AI and argumentation mentioned by Amit.
Chris Reed, Douglas Walton, and Iryna Gurevych
We do get a basic clue from the computer scientist Chris Reed about the technical aspects of what Amit has discussed. His argument interchange format (AIF) is a language which is used to help machines understand the relationships between premises and conclusions. So here we have something logically concrete: the relationships between premises and conclusions.
Reed deals with computational argumentation too. He founded the Arg-tech laboratory, which builds software to map out human debates.
It’s surely argument forms which must be studied, not arguments in general. The Canadian academic and author Douglas N. Walton (who died in 2020) called them “argumentation schemes”. Apparently, his schemes became the blueprint which AI scientists use today. As it is, Walton himself didn’t initially focus on AI at all.
Douglas Walton did the same as what Chris Reed is now doing with the Moral Maze. He catalogued many everyday and legal argument structures (like “argument from analogy” and “argument from expert opinion”) into forms that computer programmers can translate into code.
The computer scientist Iryna Gurevych carries out what’s called “argument mining”. Argument mining is a subfield of natural language processing (NLP). It extracts and analyises argumentative structures in human languages. By picking out premises, claims, and counter-arguments, it maps out the inferential logic behind human person’s opinions.
Gurevych uses the word “claim”, just like Amit. The AI detects the claim and the premises (which involve data or evidence) which support it. Thus, we have various relationships between claims and premises. How do they relate to one another? Does another claim support or reject the initial claim?
Like Douglas Walton, Gurevych focuses on argument schemes, such as arguments from analogy, cause-and-effect, or expert testimony. Similarly, just as with Chris Reed and the BBC’s Moral Maze, so Gurevych’s team builds Natural Language Processing models which scan vast amounts of text in order to detect whether a sentence is a claim, supporting evidence for a claim, or a rejection of a claim.
Conclusion
It’s worth rounding off this essay by stating that chatbots (or AIs generally) don’t need to instantiate consciousness, display “real intelligence”, be real persons, “genuinely understand”, “know what it’s like” to be a human person or a sex worker, etc. to take part in debates and offer good arguments. Personally, chatbots have often shown me that my reasoning is faulty, not only that my facts are wrong. Surely, this is a good thing.
