Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Israel-Tied Predatory Sparrow Hackers Are Waging Cyberwar on Iran’s Financial System

    June 21, 2025

    Samsung’s Galaxy Watch 7 has returned to its lowest-ever price

    June 21, 2025

    The Verge’s guide to Amazon Prime Day 2025

    June 21, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » AI Is a Black Box. Anthropic Figured Out a Way to Look Inside
    Business

    AI Is a Black Box. Anthropic Figured Out a Way to Look Inside

    News RoomBy News RoomMay 23, 20243 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    Last year, the team began experimenting with a tiny model that uses only a single layer of neurons. (Sophisticated LLMs have dozens of layers.) The hope was that in the simplest possible setting they could discover patterns that designate features. They ran countless experiments with no success. “We tried a whole bunch of stuff, and nothing was working. It looked like a bunch of random garbage,” says Tom Henighan, a member of Anthropic’s technical staff. Then a run dubbed “Johnny”—each experiment was assigned a random name—began associating neural patterns with concepts that appeared in its outputs.

    “Chris looked at it, and he was like, ‘Holy crap. This looks great,’” says Henighan, who was stunned as well. “I looked at it, and was like, ‘Oh, wow, wait, is this working?’”

    Suddenly the researchers could identify the features a group of neurons were encoding. They could peer into the black box. Henighan says he identified the first five features he looked at. One group of neurons signified Russian texts. Another was associated with mathematical functions in the Python computer language. And so on.

    Once they showed they could identify features in the tiny model, the researchers set about the hairier task of decoding a full-size LLM in the wild. They used Claude Sonnet, the medium-strength version of Anthropic’s three current models. That worked, too. One feature that stuck out to them was associated with the Golden Gate Bridge. They mapped out the set of neurons that, when fired together, indicated that Claude was “thinking” about the massive structure that links San Francisco to Marin County. What’s more, when similar sets of neurons fired, they evoked subjects that were Golden Gate Bridge-adjacent: Alcatraz, California governor Gavin Newsom, and the Hitchcock movie Vertigo, which was set in San Francisco. All told the team identified millions of features—a sort of Rosetta Stone to decode Claude’s neural net. Many of the features were safety-related, including “getting close to someone for some ulterior motive,” “discussion of biological warfare,” and “villainous plots to take over the world.”

    The Anthropic team then took the next step, to see if they could use that information to change Claude’s behavior. They began manipulating the neural net to augment or diminish certain concepts—a kind of AI brain surgery, with the potential to make LLMs safer and augment their power in selected areas. “Let’s say we have this board of features. We turn on the model, one of them lights up, and we see, ‘Oh, it’s thinking about the Golden Gate Bridge,’” says Shan Carter, an Anthropic scientist on the team. “So now, we’re thinking, what if we put a little dial on all these? And what if we turn that dial?”

    So far, the answer to that question seems to be that it’s very important to turn the dial the right amount. By suppressing those features, Anthropic says, the model can produce safer computer programs and reduce bias. For instance, the team found several features that represented dangerous practices, like unsafe computer code, scam emails, and instructions for making dangerous products.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleYou can buy an LG OLED TV starting at just $736
    Next Article US Official Warns a Cell Network Flaw Is Being Exploited for Spying

    Related Posts

    A False Start on the Road to an All-American Bitcoin

    June 20, 2025

    A Deep Learning Alternative Can Help AI Agents Gameplay the Real World

    June 20, 2025

    This AI Model Never Stops Learning

    June 20, 2025

    Those Creatine Gummies You Bought Online Might Not Contain Any Creatine

    June 20, 2025

    How Private Equity Killed the American Dream

    June 20, 2025

    eBay and Vestiaire Collective Want an Exemption from Trump’s Tariffs

    June 18, 2025
    Our Picks

    Samsung’s Galaxy Watch 7 has returned to its lowest-ever price

    June 21, 2025

    The Verge’s guide to Amazon Prime Day 2025

    June 21, 2025

    Most Cheap Laptops Only Last a Few Years. The Framework Laptop 12 Could Last a Decade

    June 21, 2025

    Final Fantasy fans, now is the time to get into Magic: The Gathering

    June 21, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    Gear

    Gear News This Week: Adobe Wants to Make iPhone Photos Better, and TCL Brings Flexibility to Atmos

    By News RoomJune 21, 2025

    The larger JBuds Party ($70) offers 30 watts of power to make it “one of…

    The Mysterious Inner Workings of Io, Jupiter’s Volcanic Moon

    June 21, 2025

    The music industry is building the tech to hunt down AI songs

    June 21, 2025

    Meta’s Oakley Smart Glasses Have 3K Video—Watch Out, Ray-Ban

    June 21, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.