Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Amazon’s October Prime Day sale is happening on October 7th

    September 16, 2025

    The unbearable sameness of Liquid Glass

    September 15, 2025

    OpenAI Ramps Up Robotics Work in Race Toward AGI

    September 15, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » Microsoft LASERs away LLM inaccuracies
    News

    Microsoft LASERs away LLM inaccuracies

    News RoomBy News RoomJanuary 31, 20242 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    During the January Microsoft Research Forum, Dipendra Misra, a senior researcher at Microsoft Research Lab NYC and AI Frontiers, explained how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate. 

    With LASER, researchers can “intervene” and replace one weight matrix with an approximate smaller one. Weights are the contextual connections models make. The heavier the weight, the more the model relies on it. So, does replacing something with more correlations and contexts make the model less accurate? Based on their test results, the answer, surprisingly, is no. 

    “We are doing intervention using LASER on the LLM, so one would expect that the model loss should go up as we are doing more approximation, meaning that the model is going to perform bad, right, because we are throwing out information from an LLM, which is trained on large amounts of data,” Misra said. “But to our surprise, we find that if the right type of LASER intervention is performed, the model loss doesn’t go up but actually goes down.”

    Misra said his team successfully used LASER on three different open-source models: RoBERTa, Llama 2, and Eleuther’s GPT-J. He said, at times, model improvement increased by 20 to 30 percentage points. For example, the performance of GPT-J for gender prediction based on biographies went from 70.9 percent accuracy to 97.5 percent after a LASER intervention.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleHulu is cracking down on password sharing, just like Disney Plus and Netflix
    Next Article Amazon left Roomba with a huge mess to clean up

    Related Posts

    Amazon’s October Prime Day sale is happening on October 7th

    September 16, 2025

    The unbearable sameness of Liquid Glass

    September 15, 2025

    Facebook gave our data to Cambridge Analytica and all I got was this $38.36

    September 15, 2025

    The Supreme Court is Google’s last hope to avoid an Epic reckoning in October

    September 15, 2025

    Meta leaks its new smart glasses with a display

    September 15, 2025

    Google thinks it can have AI summaries and a healthy web, too

    September 15, 2025
    Our Picks

    The unbearable sameness of Liquid Glass

    September 15, 2025

    OpenAI Ramps Up Robotics Work in Race Toward AGI

    September 15, 2025

    Facebook gave our data to Cambridge Analytica and all I got was this $38.36

    September 15, 2025

    The Supreme Court is Google’s last hope to avoid an Epic reckoning in October

    September 15, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    News

    Meta leaks its new smart glasses with a display

    By News RoomSeptember 15, 2025

    An unlisted and now-removed video from Meta showed off a new pair of Ray-Ban branded…

    ‘Hades II’ Is Coming to Nintendo Switch This Month

    September 15, 2025

    Google thinks it can have AI summaries and a healthy web, too

    September 15, 2025

    A New Platform Offers Privacy Tools to Millions of Public Servants

    September 15, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.