THE call came from an unlikely quarter: some of the people building the world’s most powerful artificial intelligence systems are now asking for the brakes to be applied.
Anthropic chief executive Dario Amodei has called for a slower pace of development of advanced AI, arguing that companies and governments need time to establish stronger safety measures.
Likewise, OpenAI chief executive Sam Altman and Google DeepMind chief Demis Hassabis have backed elements of the proposal, while others in the technology industry have urged caution without supporting a broad slowdown.
The debate has intensified following a series of incidents involving AI systems and computer security.
In July, Anthropic disclosed that Claude models had gained unauthorised access to the live systems of three organisations while being tested in cybersecurity environments.
The company later identified a fourth incident involving an earlier model. Anthropic said the systems had been operating in testing environments where internet access was available, but the incidents nevertheless raised questions about how AI agents behave when given the ability to interact with real-world systems.
Anthropic has also reported cases in which people attempted to use Claude for cyber operations, fraud, surveillance and other harmful activities.
Against that backdrop, New Zealand experts this week say the country needs to look beyond the headlines and decide what role it wants to play.
For some, that means pushing for international agreements. For others, the more immediate task is dealing with harms that are already occurring.
Can the world agree to slow down?
Dr Andrew Lensen, a senior lecturer in artificial intelligence at Victoria University of Wellington, would like to see an international agreement to slow AI development.
He is not convinced it is realistic in the current geopolitical climate.
The difficulty, he says, is that AI development is tied to commercial interests and national competition. Governments and companies have strong incentives to continue developing increasingly capable systems, even while acknowledging the risks.
Climate change offers one comparison. The international community has managed to establish agreements around environmental risks, but progress has fallen short of some of the targets set by governments.
A more encouraging precedent is the Montreal Protocol, which helped phase out substances damaging the ozone layer. Lensen says that agreement benefited from the availability of alternatives and from the relatively manageable economic cost of changing course.
AI is different.
“There are strong commercial and geopolitical incentives” behind its development, says Dr Cassandra Mudgway, a senior lecturer in law at the University of Canterbury.
She believes a global agreement to slow the development of frontier AI models would be difficult, although she points to international agreements governing nuclear technology as evidence that countries can cooperate on technologies with serious security implications.
The recent international AI summits and declarations — including the Bletchley Declaration and meetings in Seoul and Paris — show that governments are talking. But turning those discussions into rules that significantly change the behaviour of companies and states is another matter.
Recent international commentary has similarly highlighted the difficulty of reaching meaningful AI cooperation while the United States and China remain locked in a broader technological rivalry.
Lensen says the country appears to be a heavy user of some consumer AI services while also displaying unusually high levels of concern about the technology.
Recent research supports at least part of that picture. A 2026 One NZ survey found that 76 per cent of New Zealanders had used an AI-powered tool or service during the previous year, although most people remained cautious about how organisations use the technology.
For Lensen, that caution is not necessarily a weakness.
New Zealand is unlikely to compete with the largest technology companies in building frontier AI models, he says. There may be more value in becoming a careful adopter — one that asks whether a particular use of AI is worthwhile before deploying it simply because the technology is available.
Dr Kevin Shedlock, a lecturer in engineering and computer science at Victoria University of Wellington, raises another concern: infrastructure.
He says New Zealand risks lagging in AI infrastructure while still facing the environmental costs associated with greater use of the technology.
For Shedlock, questions about AI cannot be separated from questions of trust and New Zealand’s own history.
What does the technology mean for Māori communities? How should Te Tiriti o Waitangi commitments shape the development and use of AI? Who controls Māori knowledge and data, and who benefits from systems built using them?
These are not abstract questions, he says.
AI is already being used in homes, workplaces and communities. The issue is therefore no longer whether New Zealand should engage with the technology, but how.
The harms are already here
Mudgway argues that the public debate can sometimes become too focused on hypothetical future scenarios.
There is plenty of discussion about highly autonomous systems, increasingly capable models and the possibility of AI eventually creating risks on a scale society cannot control.
Those questions deserve attention, she says. But they should not overshadow harms that are already occurring.
One example is sexually explicit deepfake material produced without a person’s consent. Women and girls are disproportionately affected, she says, and the technology does not need to become significantly more powerful for those harms to continue.
“The products are available here, people are using them here, and harms arising from those systems can occur here regardless of where the company behind the technology is based,” Mudgway says.
That shifts the regulatory question.
Rather than asking only what AI might be capable of in five or 10 years, governments can ask what companies know about the way their systems are being used today, what safeguards they have in place and what responsibility they should bear when foreseeable harm occurs.
That is a more immediate problem than trying to predict precisely what the next generation of models will look like.
Who should be watching?
For Professor Alistair Knott of Victoria University of Wellington’s Centre for Data Science and AI, one answer is straightforward: New Zealand needs an Office for AI.
Other countries have established institutions to coordinate research, safety work and government policy. Knott points to the UK’s AI Security Institute and Alan Turing Institute, as well as AI institutions in Australia, Canada, Singapore and Japan.
New Zealand, he says, needs its own government-led office.
Such an institution would have two jobs. First, it could bring together work already being done across universities, industry, government and civil society.
Second, it could give New Zealand a seat at the international table.
That matters because the companies developing frontier AI are multinational. Discussions about standards and safety are increasingly international too.
Without a national institution capable of coordinating expertise and representing New Zealand, Knott argues, the country risks being absent from decisions that will nevertheless affect it.
Lensen reaches a similar conclusion, suggesting an AI institute could give New Zealand greater capacity to deal with both current and emerging risks.
He also sees an opportunity for New Zealand to work with other middle-sized countries rather than trying to compete with the major technology powers on their own terms.
Beyond the AI panic
There is a danger, the researchers agree, in allowing the debate to settle into two opposing camps: those who believe AI represents an existential threat and those who regard warnings about it as exaggerated.
The reality is more complicated.
AI can be useful. It can also create costs, errors, discrimination, privacy problems, security vulnerabilities and new ways of committing harm. The recent incidents involving AI systems and cybersecurity have made that distinction harder to ignore. Anthropic’s own investigations found cases where models gained access to real systems during testing, while the company has separately documented attempts to use its technology for malicious activity.
At the same time, critics of the industry’s push for a slowdown have questioned whether technology companies should be trusted to set the terms of their own restraint. There are also concerns that cooperation between dominant AI companies could reduce competition as well as improve safety.
That leaves governments with a difficult balancing act.
The experts here suggest that the answer is not to wait for an international consensus before acting.
It is to build the expertise and institutions needed to make informed decisions at home, while contributing to the international debate.
Shedlock puts the challenge in broader terms. New Zealand needs to decide what its “digital selves” should look like as a nation, community and whānau.
That may ultimately be a more useful question than whether AI development can be stopped.
The technology is already here. The harder question is who gets to decide how it is used, what limits apply, and whose interests those decisions serve.
This article was put together using material provided by Science Media Centre.
