✨ In case of interest

Reading of the Week 04/05/2022: AI and Large Language Models: Behind the Hype

Sam Nutt
Sam Nutt
This week for my Reading I looked at an emerging technology, Large Language Models, a type of AI that analyses huge datasets of language and can effectively 'create' new language based on a machine-learning algorithm that improved itself by testing its predictions against human-set problems.

If successful, LLM's could be transformative for local government, for example automating and delivering services like triaging based on literally all the knowledge recorded on the internet - but there are many critics to the technology, which I detail below. If you have any questions, then let me know!

A.I. Is Mastering Language. Should We Trust What It Says?
New York Times Magazine, Steven Johnson, 15 April 2022 (link - open incognito to access behind paywall)

On NYT Magazine on AI: Resist the Urge to be Impressed
Emily Bender, Professor in Linguistics at University of Washington (link)

One area in which AI has advanced rapidly very recently is in text analysis and creation, through large language models (LLMs). There are a number of ways that this technology might be impactful for (local) government: for example improving the assistive technologies for people living alone or with disabilities, or creating virtual assistants which can direct residents in complex questions of what government services or benefits might suit their circumstances. Some researchers, however, have an even greater vision of the technology’s promise as a route to General Artificial Intelligence, but many others are wary about the technology and its potential harms, governance questions and purpose. 

The technology promise and the hype

The articles above focus on the most well known LLM, which comes from OpenAI, a research laboratory which was founded and initially funded by a number of large tech companies and moguls. In 2020 OpenAI began offering access to a new programme called Generative Pre-Trained Transformer 3 (GPT-3). This is a neural-net ‘AI’, in which connections between ‘neurons’, connections between nodes that signify actions, superficially akin to a human brain, are stacked layer upon layer, are strengthened or weakened based on the results of repeated trial by error against problems set for the neural net. The neural-net is trained on huge amounts of data - 700 gigabytes in the case of GPT-3 - which consists of text from Wikipedia and digitised books. 

What is the result? The NYT Magazine describes some of the feats of the technology: “GPT-3 has been trained to write Hollywood scripts and compose nonfiction… You can deploy GPT-3 as a simulated dungeon-master”. It is the same technology that predicts the end of a sentence in a Gmail advert, or which famously wrote this article in the Guardian. The NYT article supposes a future in which you might ask an LLM, connected to the internet, far more complex versions of questions we currently ask Siri or Alexa. It references how GPT-3 taught itself to code via webpages that include examples of computer codes. 

The ultimate promise of those who most believe in this technology is that it can resemble or equate to ‘Artificial General Intelligence’, 

The warning from the expert

Is this a form of intelligence? One influential academic paper, from Emily Bender, a US Professor of Linguistics, as well as ex-Google researcher Timnit Gebru and others described this type of LLM as “stochastic parrots”, what the NYT article describes as “software…using randomization to merely remix human-authored sentences”. Instead, as Bender argued in an email to the author of the NYT article, the only innovations in LLMs are the “hardware, software and economic innovations which allow for the accumulation and processing of enormous data sets”. If this is the case, rather than a form of genuine intelligence, which understands the concepts behind the words it is creating, then is this, as the author suggests, a “pursuit of a false oracle?”. 

Indeed, some of the effects of LLMs are in fact harmful anyway. As they are trained on datasets derived from the good, bad and ugly of the internet, they are prone to reproducing the worst tendencies and biases already embedded in humans and language: for example, by reproducing racism. If you wanted to train in ‘values’ into an LLM, then there is a follow up question of which values, decided by whom, which is difficult to answer. As this question is so hard to solve, this problem might well be innate to the technology. 

There is also a question of transparency. Meta, Facebook’s parent company, recently published a competitor LLM completely in the open, code and all, so that researchers can analyse the underlying code and datasets which actually inform the model - OpenAI have not done this and indeed only sell access to GPT-3. Without the ability to interrogate the models and data driving a technology that could shape society, how can we anticipate or know of its potential impacts, good or bad?

On top of this is the harmful political economy of LLMs. For example, the costs of running the necessary thousands of computer servers is so large financially that it locks out small start-ups and prevents proper competition. Or, there is a question of who labels the data on which AIs are taught? These are ‘manual labour’ jobs of the AI industry, but despite their fundamental importance they are outsourced to people in developing countries working in poor conditions on poor salaries. 

Governments might also consider who is driving the technology’s development. Should the direction of travel be decided by a handful of unaccountable, powerful techno-utopian individuals who all have their own sense of values and ethics, or should these technologies be regulated or even driven by government and democratic and representative processes? One of the key elements in LLMs is the sheer size of data and processing power needed to be at the cutting edge of the technology, which limits who can work on this endeavour to those with immense resources - currently, mostly large private companies. 

If governments want to ensure that potentially powerful technologies are developed in the public interest, then it may be on them to regulate the responsible use and research of these technologies, or even, as the NYT article suggests, do it themselves - perhaps via a large well-resourced international collaboration resembling CERN for particle physics.