Close Menu
Technology Mag

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Amazon’s best Kindles are cheaper than ever at Best Buy

    August 4, 2025

    Amazon is gutting its Wondery podcast studio

    August 4, 2025

    Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

    August 4, 2025
    Facebook X (Twitter) Instagram
    Subscribe
    Technology Mag
    Facebook X (Twitter) Instagram YouTube
    • Home
    • News
    • Business
    • Games
    • Gear
    • Reviews
    • Science
    • Security
    • Trending
    • Press Release
    Technology Mag
    Home » Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
    News

    Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

    News RoomBy News RoomAugust 4, 20252 Mins Read
    Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Email

    The AI search startup Perplexity is allegedly skirting restrictions meant to stop its AI web crawlers from accessing certain websites, according to a report from Cloudflare. In the report, Cloudflare claims that when Perplexity encounters a block, the startup will conceal its crawling identity “in an attempt to circumvent the website’s preferences.”

    The report only adds to concerns about Perplexity vacuuming up content without permission, as the company got caught barging past paywalls and ignoring sites’ robots.txt files last year. At the time, Perplexity CEO Aravind Srinivas blamed the activity on third-party crawlers used by the site.

    Now, Cloudflare, one of the world’s biggest internet architecture providers, says it received complaints from customers who claimed that Perplexity’s bots still had access to their websites even after putting their preference in their websites’ robots.txt file and by creating Web Application Firewall (WAF) rules to restrict access to the startup’s AI bots.

    To test this, Cloudflare says it created new domains with similar restrictions against Perplexity’s AI scrapers. It found that the startup will first attempt to access the sites by identifying itself as the names of its crawlers: “PerplexityBot” or “Perplexity-User.”

    But if the website has restrictions against AI scraping, Cloudflare claims Perplexity will change its user agent — the bit of information that tells a website what kind of browser and device you’re using, or if the visitor is a bot — to “impersonate Google Chrome on macOS.” Cloudflare says this “undeclared crawler” uses “rotating” IP addresses that the company doesn’t include on the list of IP addresses used by its bots.

    Additionally, Cloudflare claims that Perplexity changes its autonomous system networks (ASN), a number used to identify groups of IP networks controlled by a single operator, to get around blocks as well. “This activity was observed across tens of thousands of domains and millions of requests per day,” Cloudflare writes.

    In a statement to The Verge, Perplexity spokesperson Jesse Dwyer called Cloudflare’s report a “publicity stunt,” adding that “there are a lot of misunderstandings in the blog post.” Cloudflare has since de-listed Perplexity as a verified bot and has rolled out methods to block Perplexity’s “stealth crawling.”

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp Reddit Email
    Previous ArticleOne of the biggest newsletter platforms now syndicates to Bluesky and Mastodon
    Next Article Amazon is gutting its Wondery podcast studio

    Related Posts

    Amazon’s best Kindles are cheaper than ever at Best Buy

    August 4, 2025

    Amazon is gutting its Wondery podcast studio

    August 4, 2025

    One of the biggest newsletter platforms now syndicates to Bluesky and Mastodon

    August 4, 2025

    Google dunks on Apple Intelligence in new Pixel 10 ad

    August 4, 2025

    Alienware’s AW2725Q 4K OLED gaming monitor is down to its lowest price ever at Amazon

    August 4, 2025

    Google’s healthcare AI made up a body part — what happens when doctors don’t notice?

    August 4, 2025
    Our Picks

    Amazon is gutting its Wondery podcast studio

    August 4, 2025

    Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

    August 4, 2025

    One of the biggest newsletter platforms now syndicates to Bluesky and Mastodon

    August 4, 2025

    Google dunks on Apple Intelligence in new Pixel 10 ad

    August 4, 2025
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    • YouTube
    • Vimeo
    Don't Miss
    Science

    Scientists Say New Government Climate Report Twists Their Work

    By News RoomAugust 4, 2025

    “The Climate Working Group and the Energy Department look forward to engaging with substantive comments…

    Alienware’s AW2725Q 4K OLED gaming monitor is down to its lowest price ever at Amazon

    August 4, 2025

    The Nintendo Switch 2’s Biggest Problem Is Already Storage

    August 4, 2025

    A New ‘Katamari’ Game, ‘Octopath Traveler 0,’ and More Are Coming to Switch 2

    August 4, 2025
    Facebook X (Twitter) Instagram Pinterest
    • Privacy Policy
    • Terms of use
    • Advertise
    • Contact
    © 2025 Technology Mag. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.