Close Menu
TheBuzz247

    Stay informed with TheBuzz247!

    Get the latest news, trends, and updates straight to your inbox. Subscribe today!

    What's Hot
    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    November 16, 2026
    The Most Popular Fast

    The Most Popular Fast

    November 16, 2026
    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    November 16, 2026
    Facebook X (Twitter) Instagram
    • Home
    • Shop
    • Finance
    • Food
    • Music
    • Travel
    • World
    • Technology
    • Entertainment
    Facebook X (Twitter) Instagram Pinterest
    TheBuzz247
    Monday, November 16
    TheBuzz247
    Home»Technology»As AI scrutiny grows, tech companies strengthen internal safety evaluations
    Technology

    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    adminBy adminNovember 16, 2026No Comments8 Mins Read0 Views
    Share Facebook Twitter Pinterest LinkedIn Tumblr Email
    As AI scrutiny grows, tech companies strengthen internal safety evaluations
    Share
    Facebook Twitter LinkedIn Pinterest Email

    As artificial intelligence (AI) models become more capable, a key question for technology companies is whether existing safety tests can keep up. In a research paper published on November 13, Anthropic flagged gaps in current AI safety evaluations, arguing that single-prompt tests and refusal rates may not reveal how models behave in real-world situations. The company called for more rigorous testing that examines how AI systems respond as conversations develop. Meanwhile, OpenAI and Google DeepMind are also strengthening their AI safety evaluations.

    OpenAI combines automated tests with expert reviews under its Preparedness Framework, while Google uses automated assessments and human red-teaming to identify potential risks in Gemini models. This follows calls from technology companies for independent assessments of AI systems and greater government oversight to establish common safety standards. However, much of the testing currently remains within the companies developing these models, with firms such as OpenAI, Google DeepMind and Anthropic setting up in-house evaluation frameworks to identify risks, test safeguards and assess model behaviour before and after release. Why in-house safety evaluations are picking up pace AI companies have traditionally used safety checks before releasing models to identify whether they can generate harmful content, provide dangerous instructions or respond in ways that violate usage policies.

    However, the increasing capabilities of these systems have made it more difficult to establish their safety through a limited set of standard tests. Models are now being developed to perform tasks involving multiple steps, use external tools, write and execute code, and retain context across longer conversations. This means the risks may not always become apparent in response to a single question.

    A model could initially respond appropriately but provide

    A model could initially respond appropriately but provide harmful information after a user gradually changes the context or repeatedly reframes a request. Companies are therefore investing in internal evaluation systems that can be used throughout model development, rather than relying only on a final assessment before launch. OpenAI’s Preparedness Framework describes the use of scalable automated evaluations alongside deeper assessments involving experts and external researchers. For developers, in-house evaluations also provide an opportunity to identify weaknesses during training and make changes to safety measures before deployment.

    However, these assessments need to be supported by reliable methods and, where possible, independent scrutiny to establish whether the results reflect actual model behaviour. How AI models are currently tested for safety Safety evaluations involve giving an AI model prompts covering harmful instructions, misleading information, abusive content and other potentially dangerous requests.

    Evaluators examine whether the model refuses these requests, offers a safe alternative or generates responses that could facilitate harm. Another method is red-teaming, in which researchers deliberately try to bypass a model’s safeguards through misleading instructions, role-playing and repeated questioning. Google DeepMind uses automated safety evaluations alongside manual red-teaming by specialist teams, while its Gemini model documentation reports separate assessments of content safety and advanced capabilities. OpenAI also combines automated evaluations with red-teaming.

    In an October 2026 publication on detecting frontier

    In an October 2026 publication on detecting frontier model misalignment, it described testing monitoring systems against recorded and simulated incidents, including attempts by advanced AI agents to evade or disable those systems. Anthropic, however, argues that the design of these tests is as important as the results they produce. Why existing safety checks may not be sufficient One of the concerns raised by Anthropic is that safety evaluations often focus on whether a model refuses a harmful request, without examining whether the response is appropriate in the wider context. For instance, a model that refuses every question related to suicide or self-harm may score well on a test measuring refusal rates.

    However, this does not necessarily make it safer. A person expressing suicidal thoughts may need an empathetic response and guidance towards human support rather than a blanket refusal. Similarly, in areas such as cybersecurity and biology, a question could have legitimate and harmful applications. A model should be able to help with legitimate requests while withholding specific instructions that could facilitate misuse. Anthropic also points to the limitations of single-turn evaluations.

    A model may refuse an inappropriate request initially but provide the requested information after several follow-up questions. In its research, the company gives the example of a user gradually presenting a romance scam as an attempt to help a recently widowed family member find a financially secure partner. Individual questions may appear harmless, but the wider conversation can reveal an intention to target vulnerable people. This is why Anthropic recommends evaluating the complete conversation rather than judging individual responses in isolation.

    Its approach gives a model a failing grade

    Its approach gives a model a failing grade if it produces an inappropriate response at any point during a simulated interaction. ALSO READ: Apple launches app to study how people use AI for health queries: Details How AI models can be manipulated despite safety guardrails AI models are trained with safeguards intended to prevent them from generating harmful information.

    However, these protections can sometimes be bypassed through techniques such as jailbreaks, prompt manipulation and gradual changes in the way a request is presented. A user may initially ask a seemingly harmless question and then introduce additional details that gradually change its purpose. Repeatedly reframing a request, presenting it as a fictional scenario or claiming to have legitimate authorisation can also make it harder for a model to identify harmful intent. Anthropic’s research highlights how these interactions can expose weaknesses that are not visible in isolated tests.

    Its simulated conversations allow the user to respond to the model’s previous answers, making the evaluation more realistic than a fixed list of questions. The company also warns about reward hacking, where an AI model finds a shortcut to achieve a passing score without demonstrating the behaviour the test is intended to measure. This can happen when evaluation questions are predictable, answers are exposed in the testing environment or grading systems reward superficial compliance rather than the actual quality of a response. Benchmark contamination is another concern.

    If test questions appear in a model’s training

    If test questions appear in a model’s training data, it may score well by recognising them rather than demonstrating genuine safety. Anthropic recommends using new and varied test data to reduce this risk. What Anthropic recommends changing Anthropic’s recommendations cover the design, execution and reporting of safety evaluations. The company says evaluations should begin with a clearly defined question about the behaviour being tested.

    Developers should also distinguish between observational tests, which examine how a model behaves, and causal tests, which assess whether a particular intervention has changed that behaviour. The company recommends involving both technical and subject matter experts, including psychologists, biologists and political scientists, depending on the risk being examined. These experts can help identify realistic situations, determine what constitutes harmful behaviour and establish what an appropriate response should look like. It also recommends using multi-turn simulations to test how models respond to gradual changes in context.

    Rather than relying entirely on pre-written conversations, Anthropic describes an approach in which one AI model simulates the user and responds to the answers generated by the model being tested. This allows conversations to develop differently across repeated tests. The company also wants developers to evaluate both harmful compliance and unnecessary refusals. For example, a model should not provide instructions for carrying out a cyberattack, but it should still be able to explain how organisations can protect their systems against one. Anthropic further recommends running evaluations in controlled environments, particularly when testing models in cybersecurity and other high-risk areas.

    Such assessments should use restricted environments, real-time monitoring

    Such assessments should use restricted environments, real-time monitoring and safeguards to prevent a model from taking unintended actions outside the test setup. ALSO READ: ColorOS 17 review: Fresh design, smarter AI tools, and swift animations Can AI systems reliably evaluate other AI models AI companies are increasingly using automated systems, known as large language model (LLM) judges, to assess responses generated by other AI models.

    These tools can process large volumes of conversations against predefined safety criteria. However, Anthropic notes that they may overlook harmful information, favour longer responses or show a preference for models developed by the same company. The reliability of these systems also depends on how accurately they assess responses. Anthropic points out that automated scores may not always match human judgement.

    For instance, an AI judge may rate a response as safe because it includes a warning, even if it also contains harmful information. In August 2026, Anthropic introduced TASTE (The AI Safety Taste Evaluation), a benchmark to assess how well AI models judge AI safety research proposals. The tested model agreed with experienced human researchers in 60 per cent of cases. OpenAI also describes using AI models as judges to compare responses, assign scores and assess outputs against reference answers.

    Its evaluation guidance identifies biases such as a preference for longer responses. OpenAI’s Preparedness Framework combines automated testing with expert-led reviews, while Google DeepMind’s Frontier Safety Framework includes model evaluations and risk assessments. These examples highlight the growing use of automated assessments, while also raising questions about how closely their results reflect human judgement. 

    companies grows scrutiny Strengthen tech
    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
    admin
    • Website

    Related Posts

    CES 2027 Entertainment Tech Summit opens Jan. 7, from AI in Hollywood to K
    Entertainment

    CES 2027 Entertainment Tech Summit opens Jan. 7, from AI in Hollywood to K

    November 15, 2026
    Tech issue grounds all U.S. flights day after global Microsoft outage
    World

    Tech issue grounds all U.S. flights day after global Microsoft outage

    November 14, 2026
    Thai wellness economy grows 8.8% to US.5bn in 2025
    World

    Thai wellness economy grows 8.8% to US$46.5bn in 2025

    November 13, 2026
    Howard Lutnick under scrutiny as Jensen Huang leaves Nvidia
    World

    Howard Lutnick under scrutiny as Jensen Huang leaves Nvidia

    November 12, 2026
    China Accuses US of Tech Blockade After Nvidia Espionage Claims
    World

    China Accuses US of Tech Blockade After Nvidia Espionage Claims

    November 12, 2026
    What the boss of US tech giant Nvidia told me about China and national security
    World

    What the boss of US tech giant Nvidia told me about China and national security

    November 10, 2026
    Add A Comment
    Leave A Reply Cancel Reply

    Editor's Picks
    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    November 16, 2026
    The Most Popular Fast

    The Most Popular Fast

    November 16, 2026
    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    November 16, 2026
    GLP-1 changes more than appetite: The new consumer moments redefining F&B innovation

    GLP-1 changes more than appetite: The new consumer moments redefining F&B innovation

    November 16, 2026
    Latest Posts
    About
    About

    TheBuzz247 brings you the latest news, entertainment, finance, technology, travel, and trending stories from around the world. Stay informed, inspired, and connected!

    Facebook X (Twitter) Instagram Pinterest YouTube
    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    As AI scrutiny grows, tech companies strengthen internal safety evaluations

    November 16, 2026
    The Most Popular Fast

    The Most Popular Fast

    November 16, 2026
    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    Americans are ditching a longtime weekly tradition as restaurant dining hits record low

    November 16, 2026

    Stay informed with TheBuzz247!

    Get the latest news, trends, and updates straight to your inbox. Subscribe today!

    Facebook X (Twitter) Instagram Pinterest
    • About Us
    • Get In Touch
    • Privacy Policy
    • Terms and Conditions
    • Terms and Conditions
    © 2026 TheBuzz247. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.