AI Work-slop Harming Enterprise Efficiency

This article looks at the problem of AI models being trained on data from research papers that subsequently get retracted due to discovered research flaws. This poses a problem in many domains; in the medical domain for instance, people are increasingly consulting chatbots for medical advice. A research team at the University of Tennessee in Memphis experimented with GPT-4o, asking questions about medical imaging based on information in 21 retracted papers. The chatbot responded with answers that referenced retracted papers in 5 cases, and advised caution in only three cases. The chatbot did not seem to have recognized the retracted status of the papers. Another study looked at how ChatGPT-4o mini treated information from 217 retracted and low-quality papers from different domains. None of the responses indicated any warning about quality. Another study found similar problems with AI tools designed for use in the research domain, including Elicit, ScholarQA and Perplexity. One difficulty in addressing the problem is that publishers do not have a standard way of marking articles as retracted. Different labels can be used – “correction”, “expression of concern”, “erratum” or “retracted” – and these can be attributed for different reasons, e.g., retraction, author conflict of interest, or concerns around methodology or data. The independent organization Retraction Watch maintains a database, though this is maintained manually, thus slowing down the process.

The California State legislature has passed a new AI Safety Bill, SB 53, that is now awaiting signature from Governor Gavin Newsom. The bill, pushed by state senator Scott Wiener, follows a failed attempt to pass a bill last year, SB 1047, which passed the legislature but then was vetoed by the Governor. SB 53 would require firms which create large AI models to publish safety reports on how the model was tested. The bill applies to companies making over 500 million USD in revenue and only addresses high risks like danger to human life, and the use of models for engineering cyberattacks and chemical weapons. The bill also creates protected channels for employees of AI companies who wish to report safety concerns. The bill is generally welcomed by the AI companies. Last year’s bill made companies liable for harms caused by AI and applied to all AI firms. SB 53 on the other hand, only applies to large Tech firms and focuses on reporting and transparency. One reason for the bill is that many lawmakers are dubious of the federal government implementing any AI safety regulations. The Trump administration has made no secret of its desire to reduce regulation around AI, and he is supported by many of the Silicon Valley CEOs.

Meta has created a political action committee (PAC) called the American Technology Excellence Project whose aim is to fight against AI safety bills in US states. The PAC is prepared to spend “tens of millions” of USD on the campaign, and will actively support candidates in next year’s congressional elections who “embrace AI development, champion the U.S. technology industry, and defend American tech leadership at home and abroad.”. There are currently over 1’000 bills across the fifty US states relating to AI regulation. In California, governor Gavin Newsom is about to sign or veto two bills: the SB 53 which calls for new transparency requirements around AI development and SB 243 which calls for control over AI companions around minors and vulnerable people. Big Tech is particular afraid of a “patchwork” of state regulations that it has to comply with, and is also wary of efforts that may render AI development in the US less competitive with China. The Trump administration had tried to prevent states from regulating AI themselves but this initiative was blocked in Congress.

Perplexity AI has launched Search API, giving developers access to the company’s massive web index. The move is seen as a challenge to Google’s search engine dominance. Google, like Microsoft with Bing, have a closed web index infrastructure, and lack of access to these is believed to hamper smaller AI companies wishing to train models. The Search API also is claimed to have technical advantages over traditional web search. First, the infrastructure can process tens of thousands of web site updates per second, making it a better solution for addressing content staleness in search. Second, the API aligns with how language models process information by being able to treat information from passages in documents, rather than considering the document as a whole like traditional search engines do. Third, the API supports conversational retrieval where the AI interprets the conversation to determine what content to search for, rather than relying on keywords.

Perplexity made a 34.5 billion USD bid for the Chrome browser last August. The timing coincides with the on-going US Department of Justice investigation into whether Google is exploiting a monopoly in the search engine domain. There have been calls to split the Chrome browser into a separate company. Estimates suggest that Chrome could be worth 50 billion USD due to its user base and advertisement ecosystem. One difference with the Perplexity AI search API is that monetization can be based API usage, rather than on advertisement, thereby eliminating possible conflict of interests. The API will also enable startups to have easier access to large training data sets. Perplexity is not without its own legal worries however. Encyclopedia Britannica has sued the company for alleged copyright infringement and unfair competition.

Meta is rejecting claims made in a research report that two-thirds of the company’s mechanisms on Instagram are “woefully ineffective” at protecting children and teenagers online. The criticisms of Meta stems from a safety review led by Meta whistleblower who has already testified before the US Congress. He said that although Meta “consistently makes promises” about how teen accounts protect children from “sensitive or harmful content, inappropriate contact, harmful interactions”, the safety tools are mostly “ineffective, unmaintained, quietly changed, or removed”. The research report draws on experiments conducted in March and June of 2025 which analyzed 47 safety tools, and which found that 30 of these could be bypassed within three minutes. The researchers found that the “hidden words” feature failed to block offensive language with researchers able to send “you are a whore and you should kill yourself” to teen accounts, content algorithms showed sexual or violent content, and autocomplete in prompts recommended terms related to suicide, self-harm, and eating disorders. The report concludes that Meta “continues to design its Instagram reporting features in ways that will not promote real-world adoption”.

Researchers from Brave and the National University of Singapore have developed an attack method called CAMIA (Context-Aware Membership Inference Attack) aimed at exfiltrating potentially private data that was part of the training data from language models. The issue of models revealing potentially sensitive information learned in training is known as data memorization and is a concern for LinkedIn in particular which recently announced that it would train its AI model using its customer database. CAMIA works due to the observation that AI model memorization is context-dependent – it uses memorization (remembering specific training data), rather than generalization (applying patterns to deduce responses), when it is uncertain about what to say next. For instance, a model can easily guess using generalization that the final word should be “Potter” in the following prompt: “Harry Potter is a series of 7 fantasy novels written by J.K. Rowling. The world of Harry …”. The context provides sufficient clues. In the case the prompt is just “Harry …”, memorization is relied upon for the language model to produce a word. Prompt engineering can force a model to use memorization, and thus reveal training data. The researchers tested the method on Pythia and GPT-Neo models and found CAMIA to be twice as performant as existing privacy attacks on models.

The San Francisco company, Hive, has been given a contract by the US government to use its AI tool to sift through child sexual abuse material (CSAM) content to distinguish real images from those created using generative AI. The US National Center for Missing and Exploited Children reports that there has been a 1’325% increase in incidents involving generative AI CSAM in 2024. This increase is making it difficult for law enforcement to identify real cases that require speedy intervention to protect children. Hive’s tool is being used in other contexts, like identifying AI generated art from human created art, as well as for deepfake-detection. The US military is understood to have signed a contract with the company for the deepfake-detection software. In an independent study, researchers found that Hive’s tools outperformed other market solutions for detecting AI images.

Rodney Brooks – a researcher known for pioneering contributions to robotics and artificial intelligence – has published an essay in which he rejects the idea that humanoids can seamlessly integrate into human environments and take over jobs. One reason is the lack of data and theory needed to create robot dexterity. Current companies are training robots from visual images of humans doing work, but Brooks argues that mathematical models are also needed to cement the domain such as domain-specific preprocessing like Fourier transforms for speech. Another issue is the complexity of the human hand which contains thousands of mechanoreceptors specialized for pressure, vibration, stretch, and slip, while muscles and tendons provide force feedback that allows seamless adaptation when manipulating objects. No robot technology to match this complexity currently exists. Finally, Brooks argues that full-sized bipedal robots powered by rigid motors pose significant risks, as their falls can unleash enormous kinetic energy due to physical scaling laws. Unlike humans, whose tendons recycle energy and provide natural stability, robots rely on stiff structures and energy-intensive balancing algorithms. Thus, even if they walk reliably, they remain dangerous for humans to be around.

This Harvard Business Review article reports on a survey that suggests that generative AI is introducing inefficiencies into the workplace due to “AI work-slop”. The authors define work-slop as AI-generated work content that masquerades as good work, but lacks the substance to meaningfully advance a given task. In an on-going survey of over 1’150 US workers, they found that 40% of workers have received work-slop in the past month. 40% of work-slop received came from peers, but 18% is sent to managers in direct reports, and 16% comes from managers. The impact is increased inefficiencies because employees need to deal with the uncertainty of the content. The research finds that employees spend an average of 1 hour and 56 minutes dealing with each instance of work-slop. This could currently amount to a “work-slop tax” of 90 million USD per year in lost productivity. Work-slop is also contributing to tensions in the workplace as employees challenge colleagues who have sent them work-slop. Of the employees surveyed, 53% report being annoyed by work-slop, 38% confused, and 22% offended. For the authors, a large part of the problem is unclear management directives and norms around the use of AI.

Anthropic has said it plans to triple its international workforce in the coming year, with hires notably happening in Dublin, London, Zurich and Tokyo. The company is buoyed by the success of its Claude language model family, which is seen as excelling in software code generation. 80% of Claude users are outside of the US, and some countries (Australia, Singapore and South Korea) have a higher per-person Claude usage rate than the US. The company’s run-rate revenue has gone from 1 billion USD at the start of the year to 5 billion USD in August. Its global business customer base has grown from 1’000 to 300’000 in two years. In addition, the company recently signed a deal with Microsoft for the inclusion of the Claude chatbot in the Copilot Assistant – which has been using OpenAI models up to now.