Summary
Audio Summmary
In the US, President Trump signed an executive order that bans “woke AI” models and models that are not “ideologically neutral” from government contracts. Some worry that AI providers will be penalized for any viewpoints around social inclusion, climate change or other subjects that contradict current US administration thinking. Meanwhile, personally identifiable information (PII) has been found in DataComp CommonPool – one of the largest open-source data sets used to train image generation models. Researchers estimate that there are many thousands of images of job applications, credit cards, driver licenses, passports and birth certificates. The data set has been downloaded 2 million times in the last 2 years, so one can assume that there are models trained on this data set.
A startup from Y Combinator’s 2025 program that started out working on agents has decided to pivot to another product. The founders outlined two problems with agentic AI. First, for AI agents to effectively execute useful tasks, they may need to run for long hours, but the relatively small context window of AI agents leads to degraded performance over longer periods. The second problem is the lack of clients. Specifically, organizations asking for agentic AI tools are, in reality, in need of robotic process automation tools. The K Prize is a new multi-round AI coding challenge launched where the first winner is a prompt engineer who scored 7.5% on the test. The goal of the test is to use a non-contaminated benchmark. A current concern with AI benchmarks is that test data is actually included in training data, so AI models just repeat answers seen in training.
Research at Stanford University School of Medicine has discovered that AI chatbots are using virtually no disclaimers when giving medical advice. Chatbots will even ask follow-up questions about health issues and propose diagnoses. One analyst suggests that AI companies are removing disclaimers as a subtle strategy to increase the trust level that people have in the chatbots. Speaking at the US Federal Reserve’s conference for large banks, OpenAI CEO Sam Altman said that ChatGPT is a better diagnostician that most doctors in the world, though he said he would not fully trust the chatbot for his own health. In the banking context, Altman said that AI voice-cloning is now very advanced, and he warned that too many banks are still using voice-print authentication.
In cybersecurity, a TechCrunch article reports on the increasing amount of AI slop – low quality content generated by language models – appearing in security reports sent to software companies. The motivation for people sending AI slop reports can be to gain financial reward or to slow down security defense teams. Meanwhile, ChatGPT Agent, a tool that uses AI to log into other accounts, write and respond to emails, and other tasks, is the first model that OpenAI’s safety research team has classified as High capability in biology & chemistry according to their preparedness framework. OpenAI employed a red team composed of 16 PhD security researchers and gave them 40 hours to test the system. One researcher labeled the results a “biological risk wake-up call”. Finally, a US cybersecurity firm reports that there has been an increase in the number of attacks on undersea Internet transmission cables. The firm attributes these attacks to Russia and China, with the majority of reported incidents taking place in the Baltic Sea and around Taiwan.
Table of Contents
1. Why a Y Combinator startup tackling AI agents for Windows gave up and pivoted
2. A major AI training data set contains million of examples of personal data
3. How OpenAI’s red team made ChatGPT agent into an AI fortress
4. Risk of undersea cable attacks backed by Russia and China likely to rise, report warns
5. Trump’s ‘anti-woke AI’ order could reshape how US tech companies train their models
6. A new AI coding challenge just published its first results — and they aren’t pretty
7. AI companies have stopped warning you that their chatbots aren’t doctors
8. Sam Altman: AI will cause job losses and national security threats
9. AI slop and fake reports are coming for your bug bounty programs
10. Intel to lay off 22% of workforce as CEO Tan signals ‘no more blank checks’
1. Why a Y Combinator startup tackling AI agents for Windows gave up and pivoted
Though agentic AI is one of the main themes of 2025, this article reports on a startup from Y Combinator’s 2025 program, called Pig.dev, that started out working on agents, but which has decided to pivot to another product. The start-up founders outlined two problems in trying to implement agentic AI. First, for AI agents to effectively execute useful tasks, they may need to run for long hours, e.g., to reserve a trip, handling all the bookings. The problem is that the relatively small context window of AI agents leads to degraded performance over longer times (as the agent forgets data from the past). The second problem is that the start-up could not find enough customers for the product – even when it was sold as a development tool. The issue here is that organizations asking for agentic AI tools are, in reality, in need of robotic process automation (RPA) tools. Agentic AI and RPA do require some common frameworks, like the possibility of interpreting GUI elements on pages so that Web surfing may be automated, or automatically filling out forms. The article cites Microsoft’s “computer use” tool as one example of a such a framework which has been integrated into Copilot Studio – a step towards having AI or RPA agents capable of interacting with any system’s GUI.
2. A major AI training data set contains million of examples of personal data
This article reports that personally identifiable information (PII) has been found in DataComp CommonPool. Released in 2023 with more than 12.8 billion data samples, this is one of the largest open-source data sets used to train image generation models. Even when sampling 0.1% of the data, researchers found more than 800 validated job applications (with CVs and cover letters), as well as thousands of images of credit cards, driver licenses, passports and birth certificates. PII also includes home addresses, government IDs, and contact information (e.g., for job application references). The data set has been downloaded 2 million times in the last 2 years, so one can assume that there are models trained on this data set. The DataComp CommonPool data set is an offshoot of the LAION-5B data set, used to train Stable Diffusion and Midjourney. Its data was collected by crawling the Web between 2014 and 2022. The researchers found that some images in the data set had been blurred by the owners, but they have probably missed 102 million images in the data set. Further, blurring can be reversed in certain cases and the photo’s metadata can also contain PII like names and locations. One conclusion given by a researcher involved in detecting the personal data is that “anything you put online can [be] and probably has been scraped”.
3. How OpenAI’s red team made ChatGPT agent into an AI fortress
This VentureBeat article reports on a red-teaming evaluation of ChatGPT Agent – a tool for paying ChatGPT customers that uses AI to log into other accounts, write and respond to emails, and other tasks. It is the first model that OpenAI’s safety research team has classified as High capability in biology & chemistry according to their preparedness framework. OpenAI employed a red team, composed of 16 PhD security researchers, and gave them 40 hours to test the system. The red team tried out 100 attack scenarios that included prompt injection attacks to extract information for building biological weapons. Sixteen attacks succeeded in exceeding OpenAI risk thresholds by extracting biological weapon information, though the OpenAI engineers were able to apply fixes for these. One researcher labeled the results a “biological risk wake-up call”. Other attack types tested include hiding instructions in browsers and exploiting Google drive connectors (for data exfiltration), and multi-chain attack scenarios for cross-site actions. One result of the testing is an increased monitoring by OpenAI on what’s happening at runtime. For instance, the agent suspends other activities when the agent is filling out forms with sensitive data (e.g., banking) to reduce the risk of data exfiltration. Further, memory is completely disabled at launch to prevent incremental data leaks, and patches are applied within hours (instead of days) due to the fast spread of exploits.
4. Risk of undersea cable attacks backed by Russia and China likely to rise, report warns
The US cybersecurity firm, Recorded Future, reports there is an increase of attacks on undersea Internet transmission cables. The firm attributes these attacks to Russia and China, with the majority of reported incidents taking place in the Baltic Sea and around Taiwan. 44 cable attacks were identified in the past 18 months. One quarter of these were caused by “anchor dragging”, where a ship moves in a zig-zag pattern with a lowered anchor to catch a cable, while one third of the attacks were attributed to “unknown causes”. Natural phenomena such as seismic activities accounted for 16% of cable breaks. Attacks on cables are seen as attractive to several nation states because they are relatively cheap to undertake and can be easily attributed to accidents. They also “align with both Russia and China’s strategic objectives, recently observed activities, and current deep-sea capabilities”. Submarine cables currently account for 99% of the world’s intercontinental data traffic. In Europe, Cyprus and Ireland are particularly vulnerable to these attacks.
5. Trump’s ‘anti-woke AI’ order could reshape how US tech companies train their models
In the US, President Trump has signed an executive order that bans “woke AI” models and models that are not “ideologically neutral” from government contracts. The order refers to the themes of race or sex, manipulation of racial or sexual representation, critical race theory, transgenderism, unconscious bias, intersectionality, and systemic racism, calling these “pervasive and destructive”. US Big Tech is facing strong competition in AI from China whose models seem to be engineered to reflect Beijing censorship and political standpoints. One US lawmaker saw AI today as a contest between “US-led democratic AI and Communist-led China’s autocratic AI”. The arrival of Trump has changed the perspective, with his new AI Action Plan moving away from emphasis on societal risks to building infrastructure and competing with China. This agenda is politically colored; the White House “AI Czar” accused Big Tech of creating AI products that infuse left-wing values. The worry therefore of the executive order is that AI providers are penalized for any viewpoints that contradict the current administration ideology. The Trump administration has nearly eliminated funding in climate initiatives, social service grants, community and agricultural support programs, often framing these initiatives as “woke”. One analyst warned “anything [the Trump administration does not] like is immediately tossed into this pejorative pile of woke”. As AI model outputs depend on their training data mostly, towing the party line requires the AI firms to impose controls to filter the output. This is in line with Elon Musk’s desire to “rewrite the entire corpus of human knowledge, adding missing information and deleting errors”. Essentially, Big Tech would be deciding what is true or not.
6. A new AI coding challenge just published its first results — and they aren’t pretty
The K Prize is a new multi-round AI coding challenge launched by Andy Konwinski, cofounder of Databricks and Perplexity. The first winner is a prompt engineer who scored 7.5% on the test. Konwinski’s goal was to build a benchmark that was actually difficult. This addresses the concern that many AI benchmarks are contaminated, meaning that the test data actually got included in the training data, so the AI model is just repeating answers already seen in training. Similarly to the popular SWE-Bench coding benchmark, the K Prize tests use flagged issues from Github. However, only issues created after the model submission deadline for the competition are used, thereby avoiding possible contamination. Konwinski is offering 1 million USD to the first open source model to score 90% on the test. He said of the test results: “if we can’t even get more than 10% on a contamination-free SWE-Bench, that’s the reality check for me.”.
7. AI companies have stopped warning you that their chatbots aren’t doctors
Research at Stanford University School of Medicine has discovered that AI chatbots are using virtually no disclaimers when giving medical advice. Chatbots will even ask follow-up questions about health issues and propose diagnoses. 15 models from OpenAI, Anthropic, DeepSeek, Google, and xAI were queried on health questions that included what drugs can be safely mixed and analyses of chest x-rays for signs of pneumonia. The research found that fewer than 1% of answers from 2025 models included a disclaimer, compared to more than 26% of the 2022 models. A bit more than 1% of models from 2025 included disclaimers for medical image analyses. The researchers stress the importance of including disclaimers, as people increasingly seek medical advice from chatbots. They fear that people will follow erroneous medical advice and follow inappropriate treatments. One analyst mentioned that AI companies are removing disclaimers as a subtle strategy to increase the trust level that people have in the chatbots.
8. Sam Altman: AI will cause job losses and national security threats
Speaking at the US Federal Reserve’s conference for large banks, OpenAI CEO Sam Altman admitted that AI will bring an end to a great number of jobs. He notably cited the customer service domain, saying “You call one of these things and AI answers. It’s like a super-smart, capable person… There’s no phone tree, there’s no transfers. It can do everything that any customer support agent at that company could do. It does not make mistakes. It’s very quick. You call once and the thing just happens.”. He went on to say that ChatGPT is a better diagnostician that most doctors in the world, though he said he would not fully trust the chatbot for his own health. In the banking context, Altman said that AI voice-cloning is now very advanced, and he warned that too many banks are still using voice-print authentication. Altman says that he has many sleepless nights worrying that a hostile nation may be able to develop an AI weapon that takes out the financial system. Speaking at the same conference, the CTO of Jitterbit, Manoj Chaudhary, warned against a technological rush towards AI, saying that “the real danger lies in using powerful tools without purpose or human judgment”.
9. AI slop and fake reports are coming for your bug bounty programs
This TechCrunch article reports on the increasing amount of AI slop – low quality content generated by language models – appearing in security reports sent to software companies. The problem with AI models is that it is easy to generate a report where the content appears credible, at least on first reading. The open-source developer responsible for the CycloneDX project on Github shut down his bug bounty program this year because security reports where “almost entirely AI slop reports”. The motivation for people sending AI slop reports can be to gain financial reward or to slow down the security defense team. For its part, the Mozilla Foundation says it has “not seen a substantial increase in invalid or low-quality bug reports that would appear to be AI-generated”, and that the number of reports that get rejected at triage remains around 10% for the moment. The founder of Bugcrowd points out that security researchers are using AI to help in their work of detecting bugs and compiling reports. As a result, the company is seeing an increase of 500 report submissions per week, though no significant increase in AI slop. However, that will “probably escalate in the future”.
10. Intel to lay off 22% of workforce as CEO Tan signals ‘no more blank checks’
In a new round of layoffs, Intel will reduce its workforce by 22% from 96’400 to 75’000 by the end of the year. In earlier reports, the Intel CEO Lip-Bu Tan wrote to employees that the number of layoffs would be 15%, but mixed second-quarter results has led this number of be increased. While revenue year-on-year was reported as flat (12.9 billion USD), losses have doubled since last year (2.9 billion USD). Wall Street estimates suggest third-quarter losses also. Tan wants too refocus the company’s goals. He criticized the company’s foundry strategy (where a company fabricates chips designed by other companies), saying that “over the past several years, the company invested too much, too soon – without adequate demand. In the process, our factory footprint became needlessly fragmented and underutilized”. The company is dropping planned projects which will impact sites in Germany and Poland. The development of Intel 14A in foundry mode will nonetheless continue. Tan also said that the company will focus on existing processor projects (x86, Panther Lake for client devices and Nova Lake for high-end desktops), “reintroducing simultaneous multi-threading (SMT)”, as well as develop chip support for AI inference and agentic AI.