Generative AI is the technology behind the wave of new online tools used by millions around the world. Some can answer queries on a huge range of topics in conversational language. Others can generate realistic looking photographs from short text prompts. As the technology is ever more widely deployed, what are its current strengths and its weaknesses?

The Economist Science Editor Alok Jha is joined by Deputy Editor Tom Standage, Science Correspondent Abby Berticks, and Global Business and Economics Correspondent Arjun Ramani to discuss this new era of AI.

Tom, The Economist has written about AI many, many times in its various forms over the years. What’s changed since the last time we were really interested in it that’s made it … made AI much better than perhaps it used to be?

What happened in 2017 was that some researchers at Google came up with a better attention mechanism called: ‘The Transformer’ and that’s what the ‘T’ in ‘GPT’ stands for. And so, essentially, that made these systems a lot better, they could sort of come up with more … longer pieces of coherent output, whether that’s text or computer code or whatever. So, there was a technical breakthrough and that took a while to ripple through the community. So, that’s one of the things that changed.

But the other thing that changed is this technology became much more visible; of course what happened last year is that a much more capable model, GPT 3.5, was launched as chat GPT, literally as a chatbot, which you know anyone could sign up for and once you got to the front of the waiting list, you could go and talk to it. And, you know, you’ll have heard these numbers that 100 million people tried it within the first two months and that’s, you know, reckoned to be the fastest adoption of a consumer technology in history.

So, the thing that really changed is that, suddenly, there was a way that anyone could use this technology and they came up with all sorts of amazing uses for it and asked it to do all sorts of extraordinary things and that was what really put it on the map.

I think one of the huge strengths of these large language models is that they’re able to, kind of, crunch and churn through such like scads of unlabelled data; so normally, like in the past with AI, you always needed like your ‘thing’ and a label – so that required humans to kind of go through and label them. But these large language models, you just … you chuck in the internet, and you get out of it a blurry picture that is basically taken of hundreds of billions of words. And it honestly just seems to do well.

I think a lot of people are still kind of baffled why it’s doing so well at so many tasks. Like, it generates convincing text, it’s very good at pattern matching, style transfer is one of the other things. Like, you ask it to … like … ‘Oh, write a love letter in the style of a pirate from the 14th Century that has an Irish accent but is from like the Bahamas.’ And then it’s also pretty good at passing standardized tests, it seems. Like it passed the U.S Medical Licensing exam. It’s passed some legal tests. Basically, very good at kind of text things.

At the moment, I think, you know, one of the big opportunities is writing code. The great thing about writing code with these systems – and I do still write some code – is that if the code is slightly wrong, you find out straight away because, you know, either the interpreter or the compiler chokes on it, or the output of the code isn’t quite what you were expecting. So, you had this very tight feedback loop and if it’s slightly wrong, you find out you find out pretty quickly.

In terms of weaknesses, I think one of them is the lack of transparency, like it’s kind of a black box. We… you can have access to kind of what’s going inside – the attention weights, what those values are – but they don’t mean a lot to us. There’s over a hundred billion of these weights and that’s very, very complex and hard for us to understand and what they’re doing. And, so … yeah I think the main weakness is that it’s such a complex system and we don’t really understand it fully.

If your job is to find out new facts, that’s actually not something that these systems are really in a good position to do; and I was talking to people within the Foreign Office, the British Government, the other day and, also they were saying: ‘Well, what’s the impact of this?’ And I was saying: ‘Look, our job of whether we work in government, or the intelligent services, or journalism, is to find new facts. And they’ve got to be right, right? You’ve got to … you really don’t want to just take any old stuff coming out of one of these systems. So, if the accuracy matters, then these systems are, you know, maybe not so great.

The reliability of the models need to be improved I think before, you know, they start automating huge amounts of processes and businesses. But, I mean, there’s a huge amount of economic activity I think that’ll get affected by this.

I mean the paper put out by some economists at Open AI that said that, you know, around 20% of the US Workforce could have around 50% of their tasks affected by generative AI in the next few years, right. So, a lot of tasks that we do on a day-to-day basis could be helped by these models over the long term; but there’s some interesting, you know, research in the economics of innovation that talks about how if you want to get what we called an ‘intelligence explosion’, or if you want to get, you know, exponentially increasing rates of economic growth in any given domain, you need to automate the entire process. If you’ve only automated, you know, 90% of it or 99% of it, that doesn’t get you nearly the same effect because the slowest part of the process, which is maybe probably the human acts, is what’s called a rate determining step. So we will end up slowing things down.

So, I think that’s probably, you know, what is likely to happen in my mind, where, you know, we use AIs to help us with research – which frankly we are already doing. But it still is not able to get 100% of the way there. So, ultimately, the pace of progress continues as it has been.

So, they would’ve become super intelligent and turned us into paperclips, if it wasn’t for those pesky humans getting in the way of making them more intelligent.

• Generative AI (noun): the use of artificial intelligence that is able to produce text, images etc.
• A wave (noun): a larger than usual number of events of a similar type happening within the same period.
• A prompt (noun): an instruction given to AI by a human using natural language.
• Widely (adverb): including a lot of different places.
• To deploy (verb): to use something, especially in an effective way.
• An era (noun): a period of time where new developments or events happen.
• A mechanism (noun): a part of machine or set of parts that work together.
• To stand for (phrasal verb): the first letter or letters of a word or name.
• Coherent (adjective): clear, understandable and connected to other ideas.
• Code (noun): a language used to programme computers and software.
• A breakthrough (noun): an important discovery that helps to improve a situation.
• To ripple through (phrasal verb): to slowly move through a space or group in a subtle way.
• To sign up for something (phrasal verb): to join or register to do or use something.
• To reckon (verb): to think or believe.
• Adoption (noun): accepting or starting to use something new.
• All sorts of (phrase): a lot of different types or kinds of.
• To put something on the map (idiom): to make something famous.
• To churn/ crunch through something (phrasal verb): to process large amounts of something.
• Scads of (noun): a large amount or number.
• To go through something (phrasal verb): to examine and check something.
• To chuck (informal, verb): to throw or put something in a place carelessly.
• Blurry (adjective): difficult to see clearly.
• Baffled (adjective): to not understand or be able to explain something.
• Convincing (adjective): making you believe that something is true or correct.
• Slightly (adverb): a little, a bit.
• Straight away (phrase): immediately.
• A compiler (noun): a person or thing who collects (compiles) something.
• To choke on something (phrasal verb): to not accept or find it difficult to accept something.
• A (tight) feedback loop (noun): a (very responsive) continuous system for collecting information and then implementing responses.
• Transparency (noun): the quality of being done in an open way and easy to understand.
• A black box (noun): a system that produces results without the user seeing how it works.
• Attention weights (noun): numerical values that show how much importance each item has.
• To be in a (good) position to do something (phrase): to be able to do something (well).
• The Foreign Office (noun): the department of the UK Gov. that deals with other countries.
• Any old (phrase): to emphasise that the quality or type of something is not important.
• A (research) paper (noun): a piece of writing on a subject written by an expert.
• To put out something (phrasal verb): to produce or publish information about something.
• On a day-to-day basis (phrase): happening every day as normal.
• Exponentially (adverb): in a way that becomes quicker and quicker or larger and larger.
• A domain (noun): an area of interest.
• To determine something (verb): to control something, to dictate what will happen.
• Frankly (adverb): in an honest and direct way.
• Pace (noun): the speed at which something moves or changes.
• Pesky (adjective): annoying or causing trouble.
• To get in the way of something (phrasal verb): to make it difficult for something to happen.

In this text, there are several examples of cleft sentences using ‘what’. We can change the part of a sentence we focus on by changing the structure of a sentence. We can use ‘what’ clause + ‘be’ + emphasis to focus attention on a specific noun, verb, or a whole sentence.

A ‘what’ cleft is used to highlight the most interesting or important part of a sentence. It draws attention to key information by reorganizing the sentence structure:

Normal sentence: “We got to our hotel and realised that our room had been double booked.”

With emphasis using ‘what’: “What happened was we got to the hotel and realised that our room had been double booked.”

The ‘what…’ clause can also come at the end of a sentence: ‘We got to our hotel and realised (that) what (had) happened was our hotel room had been double booked.’

Emphasising a noun:

• Structure: What + understood information + is/was + noun
• Example: “What I hated most was the insects everywhere.”

Emphasising a verb:

• Structure: What/All + subject + do/does/did + is/was + verb
• Example: “What they did was move us to another town.”

Emphasising a whole sentence/ clause:

• Structure: What + verb phrase + is/was + focus clause
• Example: “What happened was we got to the hotel and realised that our room had been double booked.”

**When speaking, stress the key information with your voice (e.g., “What I NEED now is a HOLIDAY“).

Examples in the text:

• What happened in 2017 was that some researchers at Google came up with…: what happened + was + emphasis: the ‘what’ cleft structure is used to focus on the whole sentence, explaining why AI has improved so much.
• … what happened last year is that a much more capable model: what happened + is + emphasis: the ‘what’ cleft structure is used to focus on the whole sentence, explaining another reason why AI has improved. *is or was can both be used in the what cleft structure.
• … that was what really put it on the map: emphasis + was/is + what clause: the ‘what’ cleft structure is used here at the end of a sentence, with the emphasis coming before, focusing on the noun (or pronoun) ‘that’ (the extraordinary things AI could do).
• … the output of the code isn’t quite what you were expecting: emphasis + was/is + what clause: the ‘what’ cleft structure is used here at the end of the sentence, with the emphasis coming before, focusing on the noun ‘output’.

https://www.bbc.co.uk/learningenglish/english/course/towards-advanced/unit-14/session-1

https://www.englishpodcastscollected.com/wp-content/uploads/2024/06/The-Economist-Generative-AI-What-is-it-good-for.pdf

https://www.englishpodcastscollected.com/wp-content/uploads/2024/06/The-Economist-Generative-AI-What-is-it-good-for.docx