Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Apple’s three-year iPhone plan is a break from the boring

    August 25, 2025

    Meta Has Already Won the Smart Glasses Race

    August 25, 2025

    EcoFlow’s Rapid power bank is the fastest yet

    August 25, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
    News

    Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

    News RoomBy News RoomAugust 4, 20252 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    The AI search startup Perplexity is allegedly skirting restrictions meant to stop its AI web crawlers from accessing certain websites, according to a report from Cloudflare. In the report, Cloudflare claims that when Perplexity encounters a block, the startup will conceal its crawling identity “in an attempt to circumvent the website’s preferences.”

    The report only adds to concerns about Perplexity vacuuming up content without permission, as the company got caught barging past paywalls and ignoring sites’ robots.txt files last year. At the time, Perplexity CEO Aravind Srinivas blamed the activity on third-party crawlers used by the site.

    Now, Cloudflare, one of the world’s biggest internet architecture providers, says it received complaints from customers who claimed that Perplexity’s bots still had access to their websites even after putting their preference in their websites’ robots.txt file and by creating Web Application Firewall (WAF) rules to restrict access to the startup’s AI bots.

    To test this, Cloudflare says it created new domains with similar restrictions against Perplexity’s AI scrapers. It found that the startup will first attempt to access the sites by identifying itself as the names of its crawlers: “PerplexityBot” or “Perplexity-User.”

    But if the website has restrictions against AI scraping, Cloudflare claims Perplexity will change its user agent — the bit of information that tells a website what kind of browser and device you’re using, or if the visitor is a bot — to “impersonate Google Chrome on macOS.” Cloudflare says this “undeclared crawler” uses “rotating” IP addresses that the company doesn’t include on the list of IP addresses used by its bots.

    Additionally, Cloudflare claims that Perplexity changes its autonomous system networks (ASN), a number used to identify groups of IP networks controlled by a single operator, to get around blocks as well. “This activity was observed across tens of thousands of domains and millions of requests per day,” Cloudflare writes.

    In a statement to The Verge, Perplexity spokesperson Jesse Dwyer called Cloudflare’s report a “publicity stunt,” adding that “there are a lot of misunderstandings in the blog post.” Cloudflare has since de-listed Perplexity as a verified bot and has rolled out methods to block Perplexity’s “stealth crawling.”

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleOne of the biggest newsletter platforms now syndicates to Bluesky and Mastodon
    Next Article Amazon is gutting its Wondery podcast studio

    Related Posts

    Apple’s three-year iPhone plan is a break from the boring

    August 25, 2025

    EcoFlow’s Rapid power bank is the fastest yet

    August 25, 2025

    They’re trying to make deep-sea mining happen

    August 24, 2025

    The Framework Desktop and Linux have shown me the path to PC gaming in the living room

    August 24, 2025

    The most fun way to look through old photos

    August 24, 2025

    Bose’s compact TV Speaker is more than $100 off right now

    August 23, 2025
    Our Picks

    Meta Has Already Won the Smart Glasses Race

    August 25, 2025

    EcoFlow’s Rapid power bank is the fastest yet

    August 25, 2025

    A Special Diamond Is the Key to a Fully Open Source Quantum Sensor

    August 25, 2025

    Scientists Find a New Moon Orbiting Uranus

    August 25, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    Business

    How to Become a Vibe Coder

    By News RoomAugust 25, 2025

    Michael Calore: OK.Lauren Goode: Yeah. But yeah, I’d be curious to see what you come…

    They’re trying to make deep-sea mining happen

    August 24, 2025

    The Framework Desktop and Linux have shown me the path to PC gaming in the living room

    August 24, 2025

    The West Texas Measles Outbreak Has Ended

    August 24, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.