Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    8BitDo’s new collection celebrates the NES’s 40th anniversary

    October 18, 2025

    TiVo won the court battles, but lost the TV war

    October 18, 2025

    Motorola’s Razr Ultra and the Marshall Emberton II top this week’s best deals

    October 18, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » The Time Sam Altman Asked for a Countersurveillance Audit of OpenAI
    Business

    The Time Sam Altman Asked for a Countersurveillance Audit of OpenAI

    News RoomBy News RoomMay 22, 20254 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    Dario Amodei’s AI safety contingent was growing disquieted with some of Sam Altman’s behaviors. Shortly after OpenAI’s Microsoft deal was inked in 2019, several of them were stunned to discover the extent of the promises that Altman had made to Microsoft for which technologies it would get access to in return for its investment. The terms of the deal didn’t align with what they had understood from Altman. If AI safety issues actually arose in OpenAI’s models, they worried, those commitments would make it far more difficult, if not impossible, to prevent the models’ deployment. Amodei’s contingent began to have serious doubts about Altman’s honesty.

    “We’re all pragmatic people,” a person in the group says. “We’re obviously raising money; we’re going to do commercial stuff. It might look very reasonable if you’re someone who makes loads of deals like Sam, to be like, ‘All right, let’s make a deal, let’s trade a thing, we’re going to trade the next thing.’ And then if you are someone like me, you’re like, ‘We’re trading a thing we don’t fully understand.’ It feels like it commits us to an uncomfortable place.”

    This was against the backdrop of a growing paranoia over different issues across the company. Within the AI safety contingent, it centered on what they saw as strengthening evidence that powerful misaligned systems could lead to disastrous outcomes. One bizarre experience in particular had left several of them somewhat nervous. In 2019, on a model trained after GPT‑2 with roughly twice the number of parameters, a group of researchers had begun advancing the AI safety work that Amodei had wanted: testing reinforcement learning from human feedback (RLHF) as a way to guide the model toward generating cheerful and positive content and away from anything offensive.

    But late one night, a researcher made an update that included a single typo in his code before leaving the RLHF process to run overnight. That typo was an important one: It was a minus sign flipped to a plus sign that made the RLHF process work in reverse, pushing GPT‑2 to generate more offensive content instead of less. By the next morning, the typo had wreaked its havoc, and GPT‑2 was completing every single prompt with extremely lewd and sexually explicit language. It was hilarious—and also concerning. After identifying the error, the researcher pushed a fix to OpenAI’s code base with a comment: Let’s not make a utility minimizer.

    In part fueled by the realization that scaling alone could produce more AI advancements, many employees also worried about what would happen if different companies caught on to OpenAI’s secret. “The secret of how our stuff works can be written on a grain of rice,” they would say to each other, meaning the single word scale. For the same reason, they worried about powerful capabilities landing in the hands of bad actors. Leadership leaned into this fear, frequently raising the threat of China, Russia, and North Korea and emphasizing the need for AGI development to stay in the hands of a US organization. At times this rankled employees who were not American. During lunches, they would question, Why did it have to be a US organization? remembers a former employee. Why not one from Europe? Why not one from China?

    During these heady discussions philosophizing about the long‑term implications of AI research, many employees returned often to Altman’s early analogies between OpenAI and the Manhattan Project. Was OpenAI really building the equivalent of a nuclear weapon? It was a strange contrast to the plucky, idealistic culture it had built thus far as a largely academic organization. On Fridays, employees would kick back after a long week for music and wine nights, unwinding to the soothing sounds of a rotating cast of colleagues playing the office piano late into the night.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleFEMA Has Canceled Its 4-Year Strategic Plan Ahead of Hurricane Season
    Next Article Ricoh is finally making a GR IV camera, and it’s coming in the fall

    Related Posts

    Spit On, Sworn At, and Undeterred: What It’s Like to Own a Cybertruck

    October 17, 2025

    The AI Industry’s Scaling Obsession Is Headed for a Cliff

    October 17, 2025

    A Plan to Rebuild Gaza Lists Nearly 30 Companies. Many Say They’re Not Involved

    October 16, 2025

    Feds Seize Record-Breaking $15 Billion in Bitcoin From Alleged Scam Empire

    October 16, 2025

    ‘Sovereign AI’ Has Become a New Front in the US-China Tech War

    October 15, 2025

    Mark Cuban Would Still Have Dinner With Donald Trump

    October 14, 2025
    Our Picks

    TiVo won the court battles, but lost the TV war

    October 18, 2025

    Motorola’s Razr Ultra and the Marshall Emberton II top this week’s best deals

    October 18, 2025

    The future I saw through the Meta Ray-Ban Display amazes and terrifies me

    October 18, 2025

    Don’t Fall for Sketchy iPhone VPNs—Here Are the Only 3 You Should Use

    October 18, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    News

    Facebook’s new button lets its AI look at photos you haven’t uploaded yet

    By News RoomOctober 17, 2025

    Meta has rolled out an opt-in AI feature to its US and Canadian Facebook users…

    AI can’t even turn on the lights

    October 17, 2025

    Pokémon Legends: Z-A Rotom Phone review: better camera, higher jumps

    October 17, 2025

    Amazon’s Ring now works with video surveillance company Flock

    October 17, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.