{"id":58963,"date":"2025-07-10T15:10:00","date_gmt":"2025-07-10T15:10:00","guid":{"rendered":"http:\/\/2a10f152-1df7-4c53-927c-97329637171e"},"modified":"2025-07-10T15:10:00","modified_gmt":"2025-07-10T15:10:00","slug":"fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how","status":"publish","type":"post","link":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/","title":{"rendered":"Fed up with AI scraping your content? This open-source bot blocker can help &#8211; here&#8217;s how"},"content":{"rendered":"<figure class=\"c-shortcodeImage u-clearfix c-shortcodeImage-large\">\n<div class=\"c-shortcodeImage_imageContainer\">\n<div class=\"c-shortcodeImage_image\"><picture class=\"c-cmsImage c-cmsImage_loaded\"><source media=\"(max-width: 767px)\" srcset=\"https:\/\/www.zdnet.com\/a\/img\/resize\/d9475c069b3d82e2c009972d15b7007727cd311d\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=768\" alt=\"Anubis: The god of AI bot blockers\"><source media=\"(max-width: 1023px)\" srcset=\"https:\/\/www.zdnet.com\/a\/img\/resize\/73c70adaf252c17767dc5bc5411b6d550a07e7d2\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1024\" alt=\"Anubis: The god of AI bot blockers\"><source media=\"(max-width: 1440px)\" srcset=\"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\" alt=\"Anubis: The god of AI bot blockers\"><img decoding=\"async\" src=\"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\" alt=\"Anubis: The god of AI bot blockers\" width=\"1280\" height=\"737.1727748691101\" fetchpriority=\"low\"><\/picture><\/div>\n<\/p><\/div><figcaption> <span class=\"c-shortcodeImage_credit g-outer-spacing-top-xsmall u-block\">Anubis \/ Elyse Betters Picaro \/ ZDNET<\/span><\/figcaption><\/figure>\n<p>Anyone who runs a <a href=\"https:\/\/www.zdnet.com\/article\/cloudflare-just-changed-the-internet-and-its-bad-new-for-the-ai-giants\/\">website knows how annoying AI bots<\/a> are these days.&nbsp;<\/p>\n<p><a href=\"https:\/\/www.f5.com\/\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">F5<\/a>, the application delivery network company, found that more than <a href=\"https:\/\/www.f5.com\/labs\/articles\/threat-intelligence\/2025-advanced-persistent-bots-report\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">half of all web visits come not from people but from data scrapers<\/a>, including OpenAI, Anthropic, Google, and Perplexity AI bots.&nbsp;<\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/ai-bots-scraping-your-data-this-free-tool-gives-those-pesky-crawlers-the-run-around\/\"><strong>AI bots scraping your data? This free tool gives those pesky crawlers the run-around<\/strong><\/a><\/p>\n<p>(Disclosure: Ziff Davis, ZDNET&#8217;s parent company, filed an April 2025 lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)<\/p>\n<h2>Stopping AI bots in their tracks<\/h2>\n<p>People are sick and tired of wasting money on their sites only to have AI companies rip off everything of value. So, Xe Iaso, a technical educator and part-time bot fighter, wrote an open-source program, <a href=\"https:\/\/github.com\/TecharoHQ\/anubis\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">Anubis<\/a>, to stop AI bots in their tracks. <\/p>\n<p>Anubis isn&#8217;t the only such program. Indeed, Iaso freely admits it&#8217;s &#8220;basically the <a href=\"https:\/\/www.cloudflare.com\/application-services\/products\/turnstile\/\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">Cloudflare. Are you a bot?<\/a> [aka Cloudflare Turnstile] page, but self-hostable.&#8221; This enables you to run it on your own server without incurring any fees. <\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/how-ai-companies-are-secretly-collecting-training-data-from-the-web-and-why-it-matters\/\"><strong>How AI companies are secretly collecting training data from the web (and why it matters)<\/strong><\/a><\/p>\n<p>Anubis is designed to protect websites &#8212; particularly those run by small organizations, open-source projects, and archives &#8212; from the relentless onslaught of automated scrapers that threaten to overwhelm servers and increase hosting costs. The program is a web AI firewall utility. All incoming HTTP connections must successfully pass through it before reaching your actual website. <\/p>\n<p>Tongue in cheek, Iaso describes Anubis as like the ancient Egyptian god, weighing the soul of your connection using one or more challenges to protect upstream resources from scraper bots. It does this by requiring visitors to solve a computational puzzle, which is trivial for PCs but expensive for bots operating at scale.&nbsp;<\/p>\n<p>The system checks whether visitors behave like real browsers, using JavaScript and cookies to verify authenticity. When a bot fails these challenges, it is blocked before reaching the website&#8217;s core resources. <\/p>\n<p><!----><\/p>\n<h2>Anubis is an uncaptcha<\/h2>\n<p>Now, you may ask, &#8220;Isn&#8217;t this just a CAPTCHA? And, <a href=\"https:\/\/www.zdnet.com\/article\/ai-can-now-solve-recaptcha-tests-as-accurately-as-you-can\/\">aren&#8217;t AI programs just as good at solving those as people are<\/a>?&#8221;&nbsp;<\/p>\n<p>That&#8217;s true &#8212; they are. But as Iaso says, &#8220;Anubis is an uncaptcha. It uses features of your browser to automate a lot of the work that a CAPTCHA would, and right now, the main implementation is by having it run a bunch of cryptographic math with JavaScript to prove that you can run JavaScript in a way that can be validated on the server.&#8221; <\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/cloudflare-just-changed-the-internet-and-its-bad-new-for-the-ai-giants\/\"><strong>Cloudflare just changed the internet, and it&#8217;s bad news for the AI giants<\/strong><\/a><\/p>\n<p>She is well aware that many people are hesitant to run JavaScript due to security and privacy concerns. She&#8217;s working on a non-JavaScript version of Anubis, but it&#8217;s not here yet. It will be a while. On a Reddit thread, Iaso said she&#8217;s &#8220;am working on a better one that doesn&#8217;t rely on JS, but oh god, <a href=\"https:\/\/www.reddit.com\/r\/technology\/comments\/1ltwjxb\/comment\/n1vz5xh\/\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">it is going to be a hell of a thing to implement.<\/a>&#8221; <\/p>\n<p>Anubis is written in Go and licensed under the open-source MIT License. It&#8217;s designed to be &#8220;as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.&#8221; On average, the program uses less than 128 MB of RAM on the server side. <\/p>\n<p>Most of the workload is handled by visitors&#8217; PCs and smartphones. Still, the end-user processing load is so low that ordinary users won&#8217;t notice. Indeed, since Anubis operates transparently, there are no CAPTCHAs to solve or images to click; most people won&#8217;t even know that anything is happening.&nbsp;<\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/this-proxy-provider-i-tested-is-the-best-for-web-scraping-and-its-not-iproyal-or-marsproxies\/\"><strong>This proxy provider I tested is the best for web scraping &#8211; and it&#8217;s not IPRoyal or MarsProxies<\/strong><\/a><\/p>\n<p>The proof-of-work runs in the background, and only those with outdated browsers or JavaScript disabled may encounter issues. It&#8217;s another story for bot farms &#8212; their load quickly adds up.<\/p>\n<p>In a blog, Iaso says: <\/p>\n<blockquote readability=\"18.617992177314\">\n<p>At a high level, <a href=\"https:\/\/www.tigrisdata.com\/blog\/anubis\/\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">Anubis has a big old set of rules<\/a> in <a href=\"https:\/\/anubis.techaro.lol\/docs\/admin\/policies\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">your bot policy file<\/a>. If clients match a rule, they are either passed through, blocked, or selected for secondary screening. By default, Anubis is meant to instantly work by stopping all the bleeding and letting administrators sleep without downtime alerts waking them up. This means that it&#8217;s overly paranoid and aggressively challenges everything, similar to Cloudflare&#8217;s &#8220;I&#8217;m under attack&#8221; mode. <\/p>\n<p>My intent was that admins would start out with Anubis being quite paranoid and then slowly lessen the paranoia as they find better patterns and match out ways to do things. Users tend to use Anubis in its default configuration, but this default configuration interferes with RSS feed readers and other &#8220;good bots.&#8221; <\/p>\n<\/blockquote>\n<h2>A nuclear response<\/h2>\n<p>The result is a tool that Iaso describes as a &#8220;bit of a nuclear response.&#8221;<\/p>\n<p>&#8220;This will result in your website being blocked from smaller scrapers and may inhibit &#8216;good bots&#8217; like the Internet Archive. You can configure bot policy definitions to explicitly allowlist them, and we are working on a curated set of &#8216;known good&#8217; bots to allow for a compromise between discoverability and uptime,&#8221; Iaso says.<\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/reddit-sues-anthropic-for-scraping-its-users-content-without-consent\/\"><strong>Reddit sues Anthropic for scraping its users&#8217; content without consent<\/strong><\/a><\/p>\n<p>Many groups were ready for a nuclear response. Organizations such as GNOME, FFmpeg, and UNESCO have adopted Anubis to protect their online infrastructure. Since its release in January 2025, Anubis has been downloaded over 200,000 times and is credited with helping numerous organizations avoid outages and reduce the burden of unwanted AI scraping. <\/p>\n<p>According to Duke University, a happy Anubis user, the school&#8217;s library systems have <a href=\"https:\/\/dukespace.lib.duke.edu\/server\/api\/core\/bitstreams\/816ef134-55cf-49f6-9a8b-1e8a2324b1ff\/content\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">successfully blocked about 90 percent of unwanted traffic<\/a> and over 4 million unwanted HTTP requests per day, while improving service performance with minimal blockage for real users. <\/p>\n<h2>How to install and run Anubis<\/h2>\n<p>There are <a href=\"https:\/\/anubis.techaro.lol\/docs\/admin\/installation\/\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">several ways to install and run Anubis<\/a>.&nbsp;<\/p>\n<p>Typically, Anubis is meant to sit between your reverse proxy and your target service. Support is currently free. You can access it via its <a href=\"https:\/\/github.com\/TecharoHQ\/anubis\/issues\/new?template=Blank+issue\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">GitHub issue page<\/a> or, for live chat, join Iaso&#8217;s <a href=\"https:\/\/patreon.com\/cadey\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">Patreon<\/a> and ask in the Patreon Discord channel. There&#8217;s also a commercial version of Anubis named <a href=\"https:\/\/anubis.techaro.lol\/docs\/admin\/botstopper\" target=\"_blank\" rel=\"noopener nofollow\" class=\"c-regularLink\">BotStopper<\/a>, which, at this point, just offers organizations more control over the program&#8217;s branding. <\/p>\n<p><strong>Also:&nbsp;<\/strong><a href=\"https:\/\/www.zdnet.com\/article\/how-global-threat-actors-are-weaponizing-ai-now-according-to-openai\/\"><strong>How global threat actors are weaponizing AI now, according to OpenAI<\/strong><\/a><\/p>\n<p>The battle between bot developers and defenders promises to be never-ending. Anubis&#8217;s creators are updating the tool to counter new evasion tactics, such as headless browsers and advanced browser fingerprinting. The goal is to keep the internet accessible for humans while making it uneconomical for abusive bots to operate at scale. This is not easy.&nbsp;<\/p>\n<p>If you find the project useful, do support it. She can use all the help you can give. <\/p>\n<p><em>Get the morning&#8217;s top stories in your inbox each day with our <\/em><a href=\"https:\/\/www.zdnet.com\/newsletters\/\"><strong><em>Tech Today newsletter.<\/em><\/strong><\/a><\/p>\n<div id=\"pinbox-2a10f152-1df7-4c53-927c-97329637171e\" class=\"c-shortcodePinbox-carousel\">\n<div class=\"c-listingCarouselHorizontal\">\n<h4 class=\"c-sectionHeading\">Artificial Intelligence<\/h4>\n<p> <!----> <span class=\"c-listingCarouselHorizontal_loadingIndicator\"><\/span> <\/div>\n<\/div>\n<p>READ MORE <a href=\"https:\/\/www.zdnet.com\/article\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/\">HERE<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Meet Anubis, the self-hosted firewall that&#8217;s stopping AI bots in their tracks.READ MORE HERE&#8230;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"colormag_page_layout":"default_layout","footnotes":""},"categories":[62],"tags":[],"class_list":["post-58963","post","type-post","status-publish","format-standard","hentry","category-zdnet-security"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fed up with AI scraping your content? This open-source bot blocker can help - here&#039;s how 2026 | ThreatsHub Cybersecurity News<\/title>\n<meta name=\"description\" content=\"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security &amp; Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fed up with AI scraping your content? This open-source bot blocker can help - here&#039;s how 2026 | ThreatsHub Cybersecurity News\" \/>\n<meta property=\"og:description\" content=\"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security &amp; Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/\" \/>\n<meta property=\"og:site_name\" content=\"ThreatsHub Cybersecurity News\" \/>\n<meta property=\"article:published_time\" content=\"2025-07-10T15:10:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\" \/>\n<meta name=\"author\" content=\"TH Author\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@threatshub\" \/>\n<meta name=\"twitter:site\" content=\"@threatshub\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TH Author\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/\"},\"author\":{\"name\":\"TH Author\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#\\\/schema\\\/person\\\/12e0a8671ff89a863584f193e7062476\"},\"headline\":\"Fed up with AI scraping your content? This open-source bot blocker can help &#8211; here&#8217;s how\",\"datePublished\":\"2025-07-10T15:10:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/\"},\"wordCount\":1191,\"publisher\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.zdnet.com\\\/a\\\/img\\\/resize\\\/9d443a586ab9762f6240fe41e761b81036a2ca46\\\/2025\\\/07\\\/09\\\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\\\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\",\"articleSection\":[\"ZDNet | Security\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/\",\"url\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/\",\"name\":\"Fed up with AI scraping your content? This open-source bot blocker can help - here's how 2026 | ThreatsHub Cybersecurity News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.zdnet.com\\\/a\\\/img\\\/resize\\\/9d443a586ab9762f6240fe41e761b81036a2ca46\\\/2025\\\/07\\\/09\\\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\\\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\",\"datePublished\":\"2025-07-10T15:10:00+00:00\",\"description\":\"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security & Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.zdnet.com\\\/a\\\/img\\\/resize\\\/9d443a586ab9762f6240fe41e761b81036a2ca46\\\/2025\\\/07\\\/09\\\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\\\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\",\"contentUrl\":\"https:\\\/\\\/www.zdnet.com\\\/a\\\/img\\\/resize\\\/9d443a586ab9762f6240fe41e761b81036a2ca46\\\/2025\\\/07\\\/09\\\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\\\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fed up with AI scraping your content? This open-source bot blocker can help &#8211; here&#8217;s how\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/\",\"name\":\"ThreatsHub Cybersecurity News\",\"description\":\"%%focuskw%% Threat Intel \u2013 Threat Intel Services \u2013 CyberIntelligence \u2013 Cyber Threat Intelligence - Threat Intelligence Feeds - Threat Intelligence Reports - CyberSecurity Report \u2013 Cyber Security PDF \u2013 Cybersecurity Trends - Cloud Sandbox \u2013- Threat IntelligencePortal \u2013 Incident Response \u2013 Threat Hunting \u2013 IOC - Yara - Security Operations Center \u2013 SecurityOperation Center \u2013 Security SOC \u2013 SOC Services - Advanced Threat - Threat Detection - TargetedAttack \u2013 APT \u2013 Anti-APT \u2013 Advanced Protection \u2013 Cyber Security Services \u2013 Cybersecurity Services -Threat Intelligence Platform\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#organization\"},\"alternateName\":\"Threatshub.org\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#organization\",\"name\":\"ThreatsHub.org\",\"alternateName\":\"Threatshub.org\",\"url\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/05\\\/Threatshub_Favicon1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/05\\\/Threatshub_Favicon1.jpg\",\"width\":432,\"height\":435,\"caption\":\"ThreatsHub.org\"},\"image\":{\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/threatshub\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.threatshub.org\\\/blog\\\/#\\\/schema\\\/person\\\/12e0a8671ff89a863584f193e7062476\",\"name\":\"TH Author\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g\",\"caption\":\"TH Author\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fed up with AI scraping your content? This open-source bot blocker can help - here's how 2026 | ThreatsHub Cybersecurity News","description":"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security & Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/","og_locale":"en_US","og_type":"article","og_title":"Fed up with AI scraping your content? This open-source bot blocker can help - here's how 2026 | ThreatsHub Cybersecurity News","og_description":"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security & Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.","og_url":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/","og_site_name":"ThreatsHub Cybersecurity News","article_published_time":"2025-07-10T15:10:00+00:00","og_image":[{"url":"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280","type":"","width":"","height":""}],"author":"TH Author","twitter_card":"summary_large_image","twitter_creator":"@threatshub","twitter_site":"@threatshub","twitter_misc":{"Written by":"TH Author","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#article","isPartOf":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/"},"author":{"name":"TH Author","@id":"https:\/\/www.threatshub.org\/blog\/#\/schema\/person\/12e0a8671ff89a863584f193e7062476"},"headline":"Fed up with AI scraping your content? This open-source bot blocker can help &#8211; here&#8217;s how","datePublished":"2025-07-10T15:10:00+00:00","mainEntityOfPage":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/"},"wordCount":1191,"publisher":{"@id":"https:\/\/www.threatshub.org\/blog\/#organization"},"image":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#primaryimage"},"thumbnailUrl":"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280","articleSection":["ZDNet | Security"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/","url":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/","name":"Fed up with AI scraping your content? This open-source bot blocker can help - here's how 2026 | ThreatsHub Cybersecurity News","isPartOf":{"@id":"https:\/\/www.threatshub.org\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#primaryimage"},"image":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#primaryimage"},"thumbnailUrl":"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280","datePublished":"2025-07-10T15:10:00+00:00","description":"ThreatsHub Cybersecurity News | ThreatsHub.org | Cloud Security & Cyber Threats Analysis Hub. 100% Free OSINT Threat Intelligent and Cybersecurity News.","breadcrumb":{"@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#primaryimage","url":"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280","contentUrl":"https:\/\/www.zdnet.com\/a\/img\/resize\/9d443a586ab9762f6240fe41e761b81036a2ca46\/2025\/07\/09\/69f94205-2a62-4ffe-9e2e-80a73a1dd824\/anubis-the-god-of-ai-bot-blockers.jpg?auto=webp&amp;width=1280"},{"@type":"BreadcrumbList","@id":"https:\/\/www.threatshub.org\/blog\/fed-up-with-ai-scraping-your-content-this-open-source-bot-blocker-can-help-heres-how\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.threatshub.org\/blog\/"},{"@type":"ListItem","position":2,"name":"Fed up with AI scraping your content? This open-source bot blocker can help &#8211; here&#8217;s how"}]},{"@type":"WebSite","@id":"https:\/\/www.threatshub.org\/blog\/#website","url":"https:\/\/www.threatshub.org\/blog\/","name":"ThreatsHub Cybersecurity News","description":"%%focuskw%% Threat Intel \u2013 Threat Intel Services \u2013 CyberIntelligence \u2013 Cyber Threat Intelligence - Threat Intelligence Feeds - Threat Intelligence Reports - CyberSecurity Report \u2013 Cyber Security PDF \u2013 Cybersecurity Trends - Cloud Sandbox \u2013- Threat IntelligencePortal \u2013 Incident Response \u2013 Threat Hunting \u2013 IOC - Yara - Security Operations Center \u2013 SecurityOperation Center \u2013 Security SOC \u2013 SOC Services - Advanced Threat - Threat Detection - TargetedAttack \u2013 APT \u2013 Anti-APT \u2013 Advanced Protection \u2013 Cyber Security Services \u2013 Cybersecurity Services -Threat Intelligence Platform","publisher":{"@id":"https:\/\/www.threatshub.org\/blog\/#organization"},"alternateName":"Threatshub.org","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.threatshub.org\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.threatshub.org\/blog\/#organization","name":"ThreatsHub.org","alternateName":"Threatshub.org","url":"https:\/\/www.threatshub.org\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.threatshub.org\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.threatshub.org\/blog\/coredata\/uploads\/2025\/05\/Threatshub_Favicon1.jpg","contentUrl":"https:\/\/www.threatshub.org\/blog\/coredata\/uploads\/2025\/05\/Threatshub_Favicon1.jpg","width":432,"height":435,"caption":"ThreatsHub.org"},"image":{"@id":"https:\/\/www.threatshub.org\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/threatshub"]},{"@type":"Person","@id":"https:\/\/www.threatshub.org\/blog\/#\/schema\/person\/12e0a8671ff89a863584f193e7062476","name":"TH Author","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/066276f086d5155df79c850206a779ad368418a844da0182ce43f9cd5b506c3d?s=96&d=mm&r=g","caption":"TH Author"}}]}},"_links":{"self":[{"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/posts\/58963","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/comments?post=58963"}],"version-history":[{"count":0,"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/posts\/58963\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/media?parent=58963"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/categories?post=58963"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.threatshub.org\/blog\/wp-json\/wp\/v2\/tags?post=58963"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}