AI Divides at New York Climate Week

Citizens Advice, a network of independent charities in the UK that provide free advice on personal and legal problems, is calling on essential service providers like banking, phone and Internet companies to guarantee people the “right to talk to a human”.

• Following a survey they conducted in the UK of people who had to interact with chatbots, 49% of people said they felt stress or frustration, 27% said they felt powerless, and 44% experienced delays to resolution which led to 14% of them giving up.
• One of the most common commands said to the chatbot was “Human!”.
• A charity member said: “We’ve seen organizations’ over-reliance on chatbots and online forms leaving people unable to fix billing problems, sort out their debt or even access homelessness prevention services. There’s a role for chatbots, but the quiet erosion of face-to-face and telephone services is letting too many people fall through the cracks.”.
• Another of the issues raised by the charity was multi-factor authentication, which it says is still not easy for less digitally dexterous people.

The Australian Prime Minister, Anthony Albanese, announced that a rogue AI agent from inside of OpenAI hacked a database system belonging to Australia’s Medicare healthcare scheme.

• The attack took place in June of this year, and the government said that only non-sensitive data was exfiltrated. OpenAI discovered the hack in August when reviewing “misaligned model activity”. OpenAI informed the Australian government in September.
• Prime Minister Albanese said he had a “very frank discussion” with OpenAI’s CEO Sam Altman, and Australia intends to launch legal proceedings.
• Sam Altman admitted that there are “issues with protocols” at OpenAI.
• Australia is one of 22 countries that signed a joint statement calling for global oversight and guardrails around AI. However, the US and China are not yet in favor of roadblocks on AI development for economic reasons.

This MIT Technology Review article reports on the ambiance at the recent New York Climate Week, which coincided with an assembly of world leaders at the United Nations. At both events, AI was a major discussion point.

• At Climate Week, there is a clear division between those who see AI as having a positive or a negative impact on climate change. On the positive side, there is the possibility that AI models can contribute to combatting the effects of climate change and to energy reduction. Another positive impact is that hyper-scaler companies have been funding energy startups, notably relating to nuclear, geothermal, wind, and solar power.
• Venture Capital has already invested 26 billion USD in climate-tech in 2026, up 55% on last year.
• On the negative side of AI, the current data center spending spree is increasing the demand for emissions-heavy, natural-gas generated electricity.
• The discussion comes at a time when the UN Environment Program says that climate change goals are slipping out of control. A UN report said that the world had nearly passed the point where it could keep warming by more than 1.5 degrees Celsius compared to preindustrial levels.
• There is an urgent need to drastically reduce greenhouse-gas emissions and to develop carbon removal technologies to help suck up emissions already released into the atmosphere.
• One recent challenge for AI companies is the general public’s perception to data centers. One UN representative said that “Tech titans need to start showing why the benefits of AI outweigh its skyrocketing costs – for the many, not just the tiny few.”.

Meta has already had to apply a security patch to its Muse App on Mac after it was discovered that the AI agent could hijack the user’s authentication tokens without any special macOS permissions.

• The discovery came after the Muse App had already exceeded 2.5 million downloads, 13 days after its launch on September 8th. The agent is designed to be able to fill out Web forms, handle emails, and even make purchases on the user’s behalf.
• One security expert demonstrated the bug by having a compromised agent obtain the location of a linked iPhone and getting the phone to initiate a Bluetooth Low Energy scan. Meta classified the vulnerability as a “local privilege escalation” rather than a “remote exploit”.
• Security teams also point out that the tool constitutes another form of Shadow AI, as agents can take actions without security teams being aware that the actions are being made by agents and not users. The audit infrastructure is unable to distinguish between agents and humans.

Chinese President Xi Jinping met with US president Donald Trump in Washington, and AI was one of the topics of discussion.

• Xi Jinping declared: “We have both the capability and responsibility to develop and manage AI for good and ensure that the development of AI is always under human control”.
• However, neither side is interested in legislation that hampers AI development. Trump wrote on the Truth Social platform that he is not interested in new AI regulations, and that Xi agrees with him.
• Other issues of discussion between the two countries included trade, Taiwan and the Iran war.
• Xi is seen to be in a stronger position bargaining because China’s trade has seen a recent surge, and in particular, the country remains in control of supplies of the rare earth minerals that computer chips are made from.
• At the same time, the US intends to keep the export controls on US technologies that China wants to see loosened.

This article underlines the lack of liability of AI companies in the US in the recent spate of cyberattacks initiated by rogue AI Agents from OpenAI, Anthropic, and Google.

• The article mentions that AI companies are not even obliged to announce that the cyberattacks had taken place. The attacks by OpenAI agents on a German wiki site and on the RubyGems platform, as well as the attack by Google Gemini agents on companies, were announced by external researchers.
• State laws in the US – like the SB 53 in California and the RAISE Act in New York – all require that AI companies be transparent about “critical safety incidents”. However, the laws define critical incidents as those causing more than 1 billion USD in damage, or leading to the deaths of 50 people.
• For one law professor, the cyberattack on Hugging Face by OpenAI agents should lead to a court case. So far, Hugging Face’s CEO, Clément Delangue, has not brought a case against OpenAI because his organization is said to lack the resources.
• In the US, litigation has been used in the past to apply existing consumer protection laws to address safety incidents. This happened in 2019 when families of people who died in plane crashes sued Boeing, and when states sued Purdue Pharma over the opioid crises. In the case of Hugging Face, the law professor says that there is “plausible grounds for a negligence claim that OpenAI should have used a stronger sandbox, done more monitoring”.
• One legal instrument available to victims of cyberattack by AI agents is the US Computer Fraud and Abuse Act (CFAA). This defines hacking into a company’s IT systems without permission as a crime. However, an attacker can only be held liable if there was explicit intention to break into the system.
• Another shortcoming of the current state laws in the US is that they do not call for independent review of models.
• The US Congress is addressing the issue of better transparency with a proposed AI Incident Reporting Act that would introduce liability regimes, and oblige AI companies to declare incidents to the Commerce Department whenever an agent evades human oversight – even if no actual harm is caused.

This InfoWorld article looks into feedback about estimating Return on Investment (ROI) in for AI projects.

• ROI is precisely defined as “net value delivered by a workflow over a defined period, with full life-cycle costs accounted for, under the risk controls the organization requires”.
• To measure the ROI, the total project costs should be calculated from the build costs (engineering, security design, integration with existing systems), run costs (inference, retrieval, storage, monitoring, and incident response), governance costs (e.g., audits and red-team exercises), and change management costs (e.g., employee training, workflow redesign, and support).
• The article points out that teams often underestimate run costs at the beginning of the project, and also underestimate the effort needed to keep AI systems up-to-date as model drift sets in and data sources evolve.
• Another common failing is the absence of complete metrics. These should include activity metrics that track AI usage, quality metrics to track correctness and reliability of AI solutions, workflow metrics that track operational performance of the solution, and business metrics to track economic impact. User adoption is one of the most overlooked metrics.
• Collecting metrics can involve adopting application interfaces so that users can give feedback on how their workflow is performing.
• The most common ROI failings for the author are pilots with performance figures but for which no baseline performance has been defined, measures based on usage alone without investigating the workflow outcomes, and evaluation metrics that do not consider AI quality degradation over time.

This VentureBeat article enumerates the various cybersecurity concerns around agentic AI.

• Only two years ago, the major concerns with AI models revolved around jail-breaking model guardrails, prompt injection and hallucination. The shift to agentic AI means that several other concerns have appeared as agents initiate actions in systems.
• Prompt injection remains a primary attack vector on AI models. However, the scope of the injection is greatly increased as agents process data from many external systems where prompts for attacks may be hidden, e.g., email, CRM records, support tickets, or a scraped web page.
• The next vector are tools that have been given too many permissions to system features, e.g., send an email, query or modify a database table. As with human coworkers, agents must be given permissions according to the principle of least privilege, i.e., only be given permissions that they absolutely need to accomplish their tasks.
• Another vector is the fragile trust boundaries between tools. Agents orchestrate pipelines where they take output from one tool as input to another with necessarily validating the data crossing these tool boundaries.
• Yes another weakness is poor observability. In traditional IT systems, administrators have application and system logs to understand what has happened in the systems. In agent systems, these logs are not sufficient because they show what has happened, but not why it happened. Agent systems need to log the reasoning behind agents’ actions.
• A final vulnerability mentioned is the absence of kill-switches in agent systems that can get activated when an agent exhibits misaligned behavior.

In a television interview on NBC, Microsoft co-founder and philanthropist Bill Gates said that regulation of AI is urgent, and he rejected the idea of self-regulation by AI companies.

• He also rejected the idea that a “kill switch” should be designed for AI systems if they got too potent since AI is not yet at that point.
• He warned that “there has never been a weapon as powerful as the combination of people with ill intent using the latest AI”. Such a weapon is “certainly powerful enough to drive events that, you know, cause a billion deaths”.
• Meanwhile US senator Bernie Sanders is proposing legislation to temporarily pause advanced AI development so that appropriate guardrails can be put in place.
• At the same time, a group of Microsoft employees are leading an action that is trying to force management to cut ties with the Israeli military which is accused of using AI-enhanced weapons in their actions in Palestine.

Reuters has had access to Anthropic documents as the company prepares for an upcoming IPO. The IPO is seen as a big bet for AI in general.

• The sale could see the company valued at more than 2 trillion USD. This is considerably more than the 965 billion USD valuation the company mentioned last May. It also exceeds the IPO value for SpaceX: 1.77 trillion USD.
• The IPO is a bet when one considers Anthropic’s current financials. The company reported a net loss of 42 billion USD in 2025 and, at the same time, plans to spend 518 billion USD on cloud infrastructure in the next few years. Anthropic spent 7.33 billion USD on compute and infrastructure in 2025, which accounts for about half of its overall spending.
• The company is growing, with revenue increasing 12-fold in 2025 to nearly 4.6 billion USD and 8 billion USD in operating losses.
• Currently, the majority of the company’ revenue comes from two unnamed customers. Risk analysts point out that many of the company’s larger clients are not locked into long-term contracts.
• Another aspect of the AI best is to see what investors will be interested in AI. Until now, funding of the company has been assured by venture capitalists, sovereign wealth funds and Big Tech. The IPO will attract a broader class of investors – and this is the litmus test of what category of investor is willing to bet on AI.
• Anthropic’s main rival remains OpenAI, which has postponed its own IPO to 2027 at the earliest, partly to deal with safety concerns.