Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Google Pixel 10 Pro Fold review: finally, a more durable foldable

    October 8, 2025

    Some of my favorite smart kitchen gadgets are on sale for Prime Day

    October 8, 2025

    The Google Pixel Watch 4 is the Android watch to beat

    October 8, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » Microsoft LASERs away LLM inaccuracies
    News

    Microsoft LASERs away LLM inaccuracies

    News RoomBy News RoomJanuary 31, 20242 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    During the January Microsoft Research Forum, Dipendra Misra, a senior researcher at Microsoft Research Lab NYC and AI Frontiers, explained how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate. 

    With LASER, researchers can “intervene” and replace one weight matrix with an approximate smaller one. Weights are the contextual connections models make. The heavier the weight, the more the model relies on it. So, does replacing something with more correlations and contexts make the model less accurate? Based on their test results, the answer, surprisingly, is no. 

    “We are doing intervention using LASER on the LLM, so one would expect that the model loss should go up as we are doing more approximation, meaning that the model is going to perform bad, right, because we are throwing out information from an LLM, which is trained on large amounts of data,” Misra said. “But to our surprise, we find that if the right type of LASER intervention is performed, the model loss doesn’t go up but actually goes down.”

    Misra said his team successfully used LASER on three different open-source models: RoBERTa, Llama 2, and Eleuther’s GPT-J. He said, at times, model improvement increased by 20 to 30 percentage points. For example, the performance of GPT-J for gender prediction based on biographies went from 70.9 percent accuracy to 97.5 percent after a LASER intervention.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleHulu is cracking down on password sharing, just like Disney Plus and Netflix
    Next Article Amazon left Roomba with a huge mess to clean up

    Related Posts

    Some of my favorite smart kitchen gadgets are on sale for Prime Day

    October 8, 2025

    1Password says it can fix login security for AI browser agents

    October 8, 2025

    Logitech’s MX Master 3S is one of the best wireless mice around, and it’s 25 percent off

    October 8, 2025

    Verizon buys the not-quite-5G wireless ISP Starry to expand wireless broadband

    October 8, 2025

    You can still save up to 70 percent on headphones from Bose and Sony today

    October 8, 2025

    Google’s AI try-on imagines your feet in new shoes

    October 8, 2025
    Our Picks

    Some of my favorite smart kitchen gadgets are on sale for Prime Day

    October 8, 2025

    The Google Pixel Watch 4 is the Android watch to beat

    October 8, 2025

    1Password says it can fix login security for AI browser agents

    October 8, 2025

    Jony Ive Says He Wants His OpenAI Devices to ‘Make Us Happy’

    October 8, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    News

    Logitech’s MX Master 3S is one of the best wireless mice around, and it’s 25 percent off

    By News RoomOctober 8, 2025

    While it lacks the haptics of the newer MX Master 4, Logitech’s MX Master 3S…

    Google Pixel Buds 2A review: the right kind of compromise

    October 8, 2025

    Verizon buys the not-quite-5G wireless ISP Starry to expand wireless broadband

    October 8, 2025

    You can still save up to 70 percent on headphones from Bose and Sony today

    October 8, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.