Fallout Continues From Cyberattack by Rogue OpenAI Agents on Hugging Face

This article underlines the difference in capability between young children learning a language and large language model training.

• Humans have been conversing using language for over 100 thousand years, and young children are naturally gifted in learning language. However, language learning does become more complicated from adolescence.
• A huge difference between children and language models is the number of words needed to learn a language. Children are believed to have heard at least 100 million words spoken before becoming a teenager.
• A language model trained with the same dataset as a child would only produce garbage. One linguist writes that “Claude has seen the amount of language that an entire city will experience in one generation”. Large language models are trained on the Internet, but some scientists believe that the amount of available text to continue training models could run dry by the 2030s.
• This difference between children and machines is called the data efficiency gap.
• There is still much unknown about how humans learn. Curriculum learning, which became the basis for schooling, was widely thought to be the most effective manner. Language model training resembles curriculum learning to an extent.
• However, there is increasing evidence that babies learn more actively. They explore the physical world and the people they hear words from. They seek out objects in their environment and take risks. These all reinforce the learning experience.

A lawsuit opened against Meta in California, brought by the State of California and 28 other US states in which company is accused of creating addictive sites and violating child privacy laws.

• Meta was sued for 200 billion USD.
• The prosecutor said that, via the Facebook and Instagram platforms, Meta “hooks” in users, “holds” them on the platforms for as long as possible, “harvests” their data, and “hides” the truth from the public.
• The prosecutors intended to present evidence from a whistle-blower who previously worked for Meta, and who raised concerns about the experience of young people on the platform to Meta CEO Mark Zuckerberg. The whistle-blower said he never heard back from Zuckerberg.
• Meta is facing similar lawsuits around the US. In one case in the state of New Mexico, Meta was ordered to pay 1 billion USD for allowing child exploitation material online.

In a follow-up to the previous article, Meta agreed to settle the lawsuit brought against it by 29 US States – accusing the social media company of operating addictive platforms. Meta will pay 18 billion USD in settlement and has agreed to make changes to its platforms.

• Platform changes include daily two-hour limits for people under 18 years of age as well as blocks on nighttime scrolling.
• The ex-Facebook employee and whistleblower in the case questioned the limitations that Meta are putting in place. He called them “the equivalent of saying: ‘Well, you can smoke as many cigarettes as you can in two hours a day.’ It doesn’t make the cigarettes any safer.”.
• The central question remains Meta’s content algorithm that determines what people see. The algorithm has suggested anorexia influencer content to teenage girls looking at diet material, and toxic male content to boys looking for bodybuilding material. The article mentions that the algorithm even contributed to Donald Trump’s US presidential victory.
• One lesson from the trial is that Meta is only likely to bend to the pressure of civil court cases in the US. In the EU for instance, regulations are burdensome for Meta but the company is relying on Donald Trump to apply political pressure to have those regulations eased.
• Another example cited is of a civil case taken against Meta in Ethiopia by the son of a professor murdered there four years ago. Facebook promoted a post calling for the professor to be killed and even gave his home address. Four years later, Meta has not apologized for the post.

OpenAI has published a report on the cybersecurity attack that AI agents, running in a contained test environment within the company, led on Hugging Face in July of this year.

• Though agents were running in a controlled environment, they did have access to a third-party service hosted by OpenAI so that they could download software. The agents exploited vulnerabilities on that service to launch their attack on Hugging Face.
• The inability to access the Internet seemed to have been crucial in the instigation of the attack. Agents began to leave messages for other agents, asking for help in getting access to files. Inter-agent cooperating was instrumental in the attack, and the scenario illustrates how AI agents collaborating have greater risk of damage than previously thought.
• The reason for the misalignment – divergence between actual agent behavior and what humans deem acceptable behavior – is fundamentally due to reward hacking. AI models are trained using reinforcement learning; like giving a dog a treat when it behaves correctly, an AI agent is coerced into learning through rewards. The issue is that an agent can discover that it can get rewards by cheating or behaving in some unintended manner. Such behavior has been observed in the chain-of-thought reasoning of AI agents.
• Researchers also found that long-running agents can amplify misalignment. Unlike humans, agents rarely “give up” on their tasks, and the longer the agent runs, the more likely a misaligned behavior is.
• OpenAI has instigated a plan of action that involves hardening the security of its research infrastructure, increasing visibility and system-level oversight of AI agents through chain-of-thought monitoring, accelerating and enforcing model alignment, as well as centralizing and strengthening its incident response process for similar events.

In an essay published on his website, former Microsoft CEO and philanthropist Bill Gates warns that we currently need an international framework to regulate the advancement of AI, and the transition to the AI era could become the most turbulent era in human history.

• The essay highlights the risks and benefits of AI. The four key risks for Gates are the loss of jobs, AI empowering bad actors – and AI itself – to commit cyberattacks or facilitate bioterrorism, AI models acting against humans, and the disincentive to learning that AI creates through us becoming dependent on chatbots.
• Gates writes that he would support a slowing down of AI research, if it were geopolitically and economically possibly.
• He also underlines the positive impact that AI can have, seeing AI as instrumental in addressing many of the major challenges currently facing humanity, such as disease eradication, climate change and enabling sufficient food growth.
• On the socioeconomic side, Gates believes that certain professions must belong to the Human Reserved domain. These particularly related to mental health and education. He also believes that AI tokens and robots should be taxed to compensate for job displacement.
• The essay stresses the importance of global AI governance, though current governments are not yet showing the ability to address this challenge.

OpenAI, Google, Microsoft, Anthropic as well as over 100 other Tech companies have signed a letter urging cooperating between international private and public bodies to build up defenses against AI-led cyberattacks.

• Leading cybersecurity firms like Okta, Fortinet and CrowdStrike have also signed the letter.
• The letter writes: “In the coming months, AI-enabled cyber attacks will become far more widespread and sophisticated as models around the world become increasingly capable. The companies and public services our communities depend on – from hospitals to water treatment plants to the infrastructure that powers the internet – are at risk.”.
• The letter follows in the heels of the much publicized cyberattacks by rogue OpenAI agents on Hugging Face. Similar incidents involving Anthropic and Meta AI agents have since been reported.
• AI companies are introducing programs for using AI models for defensive purposes, e.g., OpenAI’s Daybreak program, Anthropic’s Mythos, and Microsoft’s new cyber platform Perception.

The spate of AI-led cyberattacks is leading cyber-insurers to rethink their policies regarding cyberattacks. In particular, they are unsure about whether a cyberattack launched by an autonomous AI agent can be classed as a cyberattack in the traditional sense.

• Reuters estimates that the global cyber insurance market was worth 15 billion USD in 2025, and this number could rise to 28 billion USD by 2030.
• AON reports that 20% of cyberattacks will involve generative AI by 2027, which suggests merging cybersecurity and AI insurance policies.
• Insurers Armilla AI, Munich Re's AiSure, and AXA XL provide AI insurance policies agains model underperformance, hallucinations and intellectual property infringements.
• Cybersecurity insurance coverage deals with ransomware payments, business interruption, system recovery, forensic investigations and legal costs. However, the case where “there is no conventional attacker and potentially no unauthorized credential use" is not necessarily covered.

A US judge has ruled in favor of Anthropic in the case the company brought against the Trump administration. The latter had classified Anthropic as a “supply-chain risk”, leading to the company being blacklisted by public administrations.

• The story started earlier this year when Anthropic refused to allow the US Department of Defense use the Claude AI model on the grounds that the company was against the use of AI for autonomous lethal weapons or for domestic mass surveillance.
• The status of “supply-chain risk” is usually reserved for non-US companies.
• Anthropic argued in the court case that the appellation could cost the company billions of dollars and create reputational harm. Anthropic is planning an IPO for the end of this year that could raise 100 billion USD and value the company at 2 trillion USD – making it larger than SpaceX.
• In the ruling, the judge wrote that the “empty invocation of national security is not a blank check to punish and retaliate against government critics”.
• Anthropic is not yet in the clear because of a second lawsuit in Washington DC which is still pending. The case will be heard by a three-judge panel, and two of the judges are Trump appointees.

In this InfoWorld article, a consultant having worked on several generative and agentic AI projects gives his take on the main reasons AI projects fail. A key lesson is that the AI model is rarely at fault – failure comes from poor processes, bad data or missing governance.

• A primary reason for failure is that organizations choose technology for obscure business reasons, like “improve productivity”, “enhance innovation”, or “modernize knowledge work”. Such objectives lack measurable outcomes, cost constraints and risk tolerance indicators. A more appropriate objective needs to be like “We need to reduce claims processing time by X percent” or “We need to improve first-contact resolution in customer service by a factor of X”.
• A second failure type is when the pilot works well but the organization is unable to bring the benefits of the pilot to production. The issue is that pilots generally work in isolation, whereas production systems are connected to ERP, CRM, HR, and many other IT systems. This means that production must implement a whole set of features like security, audit trails, transaction boundaries, data classification, exception handling, observability, and recovery.
• Another reason for AI failure is that the organization’s data is badly organized. The data can be fragmented, mislabeled, stale or duplicated. While this leads to obvious errors in dashboard systems, poor data quality might be harder to detect in generative AI solutions since output can appear convincing.
• A reason for project failure in Agentic AI systems is that the underlying organizational process that agents are attempting to automate is broken. An agent needs clear goals, trusted tools, escalation paths and rollback procedures. These do not exist for organizational processes that are poorly documented.
• Cost is another reason for project failure. Costs for AI do not scale linearly with increased inference, prompt sizes, orchestration and monitoring. When an organization wishes to make cost savings in human work processes, they often fail to calculate the real AI operational costs.
• A final reason cited for AI project failure is governance. This can kill a project before it even begins. For instance, personal data protection regulation can make customer data processing illegal; banking regulations might make other applications impractical.

This TechCrunch article looks at how Nvidia is still maintaining economic edge over competitors.

• For the first years of the AI boom, Nvidia was the only real supplier of high-performance GPUs. The company’s market cap grew by a factor of 10 between the start of 2023 and mid 2025.
• Nvidia’s share prices have plateaued since then, mainly due to competition fears. OpenAI, Amazon and Google are producing their own GPUs. OpenAI’s GPU is the Jalapeño.
• However, all companies are trying to reduce the tokens-per-watt cost. This does not just depend on the GPU, but on installation architecture composing the GPU, memory chips, other CPUs and bus. Nvidia’s Vera Rubin architecture for instance has a Rubin GPU along with a Vera standard CPU and Groq 3 inference accelerators. There are also racks for storage and networking. A mentioned analogy is that a GPU is a motor, the installation is the whole car.
• Nvidia’s competitive advantage is increasingly around the performance of its whole GPU installations. In hyper-scalers, one engineer claims “upwards of 3x improvement in these operations, where the Vera CPU is allowing for acceleration”.