We Need to Talk... About AI Safety

Tuesday, 18 June 2024

By Maury Shenk

We Need to Talk... about AI Safety

At the conclusion of the AI Seoul Summit in late May, the participating 27 countries (plus the EU) issued a Ministerial Statement that began with a focus on AI safety:

It is imperative to guard against the full spectrum of AI risks, including risks posed by the deployment and use of current and frontier AI models or systems and those that may be designed, developed, deployed and used in future.

But less than a week earlier, Jan Leike, the head of AI “alignment” research at market leader OpenAI, had resigned and published on X concerns about the limited extent of safety work at OpenAI:

I believe much more of our bandwidth should be spent getting ready for the next generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact, and related topics.

These problems are quite hard to get right, and I am concerned we aren't on a trajectory to get there.

How do we solve this disjunction between AI safety aspirations and practical neglect by companies and others who are racing to deliver market-leading AI? The answer, as Leike put it, is “quite hard to get right”.

I believe that a significant part of the solution must lie in persistent and open conversations aimed at consensus on the way forward, leading to proportionate agreements and regulation that keep us on track for AI safety. Such conversations are a large part of what we are pursuing through the 695th Lord Mayor’s Ethical AI Initiative, and the Safe and Responsible AI Information Hub (Saihub) project that I am leading.

To address an objection I often hear that conversations about AI safety spread fear and doubt about AI, it is important to emphasise that main reasons to attend to AI safety include the tremendous positive potential of AI – which significantly exceeds likely harms – and the fact that widespread and increasing development and penetration of AI is inevitable. As discussed in more detail below, there are good precedents for taking safety concerns seriously, even for technologies with major positive impacts.

Tribalism, Dialogue and Regulation

Some might also say that conversation as a solution to a thorny problem like AI safety is a cop-out, and one that plays into the hands of those who want to press for AI progress without guardrails, while other people keep talking. However, such criticisms underestimate the power of broad government, societal and business consensus that can be built through determined dialogue.

Let’s start by recognising that not all of us are going to agree. Close to one extreme are the views on Marc Andreessen in his October 2023 Techno-Optimist Manifesto:

We believe Artificial Intelligence is best thought of as a universal problem solver. And we have a lot of problems to solve. …

We believe any deceleration of AI will cost lives. Deaths that were preventable by the AI that was prevented from existing is a form of murder.

Towards the other end of the spectrum are those of Kate Crawford in her 2021 book Atlas of AI:

When AI’s rapid expansion is seen as unstoppable, it is possible only to patch together legal and technical restraints on systems after the fact … . But these will always be partial and incomplete responses in which technology is assumed and everything else must adapt. But what happens if we reverse this polarity and begin with the commitment to a more just and sustainable world? How can we intervene to address interdependent issues of social, economic, and climate injustice? Where does technology serve that vision? And are there places that AI should not be used, where it undermines justice?

It is becoming increasingly common in recent years for public debates to be dominated by such strong, opposing points of view. But it remains possible to resist this trend and build consensus, and broad conversations are the way to get there. It is better that we make progress now, before waiting for serious accidents to drive the discussion. There are strong precedents for a consensus-driven approach.

An example of safety regulation being driven by accidents is in aviation. Widespread fatal accidents in the early years of human flight eventually led to strict safety regulation and steadily improving safety. There is now a nearly universal consensus that realising the benefits of air travel must go hand-in-hand with safety regulation.

Another controversial technology for which there is widespread regulatory/governance consensus, reached without serious incident, is human germline engineering. There is general consensus that DNA editing (which is achieving impressive medical benefits, including playing a crucial role in developing Covid-19 vaccines) should not be applied to human embryos. This consensus is not without dissent – e.g. there have been a couple of high-profile counter-examples involving China – but remains relatively durable.

We should hope that similar consensus can be achieved with respect to AI safety. Even if such consensus is not broad enough to address all AI safety issues – which is nearly certain to be the case – progress can achieve major benefits on some of the specific AI safety challenges that we face.

Conversations on AI

There is promising evidence that effective conversations on AI safety can achieve traction.

Exhibit one is the two international AI safety summits. The first, at Bletchley Park in the UK in November 2023, was notable primarily for the fact that it happened, including that the United States and China both participated and signed on to the Bletchley Declaration at the end of the summit. There was some criticism of the summit for focusing too much on existential risk from AI, but much broader safety issues were also considered.

The second summit in Seoul, mentioned above, broadened the scope of the first, with the MInisterial Statement addressing issues of safety, innovation and inclusivity. While some advocates of AI safety complained that safety was no longer the exclusive focus at this summit, it is in fact not possible to consider AI safety issues without broader context. And the summit made good, specific progress on safety issues, including publication of the International Scientific Report on the Safety of Advanced AI, led by Yoshua Bengio. China again attended the summit but did not sign the final Ministerial Statement.

A third AI safety summit is planned to take place in Paris in February 2025.

Here in the UK, there is a lot of activity on AI safety. To take a few examples:

The UK AI S afety Institute is a government agency (part of the Department for Science, Innovation, and Technology) whose mission includes “foster[ing] collaboration across companies, governments, and the wider research community to mitigate risks and advance publicly beneficial research”.
The 695th Lord Mayor’s Ethical AI Initiative, where I am on the Steering Committee, has been focusing both on developing new professional certifications on ethical AI and on convening discussions related to ethical AI. Its outreach has included participation in the Walbrook AI Accord on quality infrastructure for AI, aligned to the ISO 42001 standard.
ControlAI has been delivering campaigns and building a network focused on AI risk, including a major campaign on harm from deepfakes.
With a group of collaborators, I have been building the Safe and Responsible AI Information Hub (Saihub), which aims to be the leading source of information on safe and responsible artificial intelligence, including AI regulation and governance, and technical solutions for AI safety.

Crucially, all of these initiatives aim not just to engage a wide group of stakeholders in conversation, but also focus on specific aspects of the complex challenges of AI safety. Such specific discussions can lead to real progress and action on AI safety. We need to talk!

About Maury Shenk

Maury Shenk is an experienced entrepreneur, lawyer and investor. Among other things, he is Founder & CEO of AI-enabled edtech startup LearnerShape, founder of the Safe and Responsible AI Information Hub (Saihub), co-organiser of a series of consultations on AI at St George's House in Windsor Castle, and a member of the Steering Committee of the 695th Lord Mayor's Ethical AI Initiative.