Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    ByteDance’s Other AI Chatbot Is Quietly Gaining Traction Around the World

    October 20, 2025

    Covid Shots Are About to Be Widely Available Again

    October 20, 2025

    Oura redesigns app with expanded stress tracking

    October 20, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » Microsoft LASERs away LLM inaccuracies
    News

    Microsoft LASERs away LLM inaccuracies

    News RoomBy News RoomJanuary 31, 20242 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    During the January Microsoft Research Forum, Dipendra Misra, a senior researcher at Microsoft Research Lab NYC and AI Frontiers, explained how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate. 

    With LASER, researchers can “intervene” and replace one weight matrix with an approximate smaller one. Weights are the contextual connections models make. The heavier the weight, the more the model relies on it. So, does replacing something with more correlations and contexts make the model less accurate? Based on their test results, the answer, surprisingly, is no. 

    “We are doing intervention using LASER on the LLM, so one would expect that the model loss should go up as we are doing more approximation, meaning that the model is going to perform bad, right, because we are throwing out information from an LLM, which is trained on large amounts of data,” Misra said. “But to our surprise, we find that if the right type of LASER intervention is performed, the model loss doesn’t go up but actually goes down.”

    Misra said his team successfully used LASER on three different open-source models: RoBERTa, Llama 2, and Eleuther’s GPT-J. He said, at times, model improvement increased by 20 to 30 percentage points. For example, the performance of GPT-J for gender prediction based on biographies went from 70.9 percent accuracy to 97.5 percent after a LASER intervention.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleHulu is cracking down on password sharing, just like Disney Plus and Netflix
    Next Article Amazon left Roomba with a huge mess to clean up

    Related Posts

    Oura redesigns app with expanded stress tracking

    October 20, 2025

    SpaceX launches 10,000th Starlink internet satellite

    October 20, 2025

    Major AWS outage takes down Fortnite, Alexa, Snapchat, and more

    October 20, 2025

    X is changing how it handles links to try and keep you in the app

    October 19, 2025

    X is launching a marketplace for inactive handles

    October 19, 2025

    Time to catch ‘em all again

    October 19, 2025
    Our Picks

    Covid Shots Are About to Be Widely Available Again

    October 20, 2025

    Oura redesigns app with expanded stress tracking

    October 20, 2025

    Should You Cancel Xbox Game Pass? Everything to Know on the Price Hikes and New Features

    October 20, 2025

    How ByteDance Made China’s Most Popular AI Chatbot

    October 20, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    News

    SpaceX launches 10,000th Starlink internet satellite

    By News RoomOctober 20, 2025

    On Sunday, SpaceX launched 56 additional Starlink satellites on separate Falcon 9 rockets, surpassing 10,000…

    Major AWS outage takes down Fortnite, Alexa, Snapchat, and more

    October 20, 2025

    X is changing how it handles links to try and keep you in the app

    October 19, 2025

    X is launching a marketplace for inactive handles

    October 19, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.