{"id":57760,"date":"2024-11-26T09:22:00","date_gmt":"2024-11-26T09:22:00","guid":{"rendered":"http:\/\/6255e7b1-7fe3-4692-8acb-f2d94517934d"},"modified":"2024-11-26T09:22:00","modified_gmt":"2024-11-26T09:22:00","slug":"i-tested-9-ai-content-detectors-and-these-2-correctly-identified-ai-text-every-time","status":"publish","type":"post","link":"https:\/\/www.threatshub.org\/blog\/i-tested-9-ai-content-detectors-and-these-2-correctly-identified-ai-text-every-time\/","title":{"rendered":"I tested 9 AI content detectors – and these 2 correctly identified AI text every time"},"content":{"rendered":"
\n
\n
\"ai<\/picture><\/div>\n

<\/div>

diyun Zhu\/Getty Images<\/span><\/figcaption><\/figure>\n

When I first examined whether it’s possible to fight back against AI-generated<\/a> plagiarism, and how that might work, it was January 2023, just a few months into the world’s exploding awareness of generative AI<\/a>. <\/p>\n

This is an updated version of that original January 2023 article. When I first tested GPT detectors, I used three: the GPT-2 Output Detector<\/a> (this is a different URL than we published before), Writer.com AI Content Detector<\/a>, and Content at Scale AI Content Detection<\/a> (which is apparently now called BrandWell<\/a>). <\/p>\n

Also: How to use ChatGPT: Everything you need to know<\/a><\/strong><\/p>\n

The best result was 66% correct from the GPT-2 Output Detector. I did another test in October 2023 and added three more: GPTZero<\/a>, ZeroGPT<\/a> (yes, they’re different), and Writefull’s GPT Detector<\/a>. Then, in the summer of 2024, I added QuillBot<\/a> and a commercial service, Originality.ai<\/a>, to the mix. This time, I’ll also be adding Grammarly’s beta checker<\/a>.<\/p>\n

In October 2023, I removed the Writer.com AI Content Detector from our test suite because it failed back in January 2023, it failed again in October, and it failed in summer 2024. However, it now appears to work, so I’m including it in the test suite. See below for a comment from the company, which their team sent me after the original article was published in January.<\/p>\n

Also: 88% of workers would use AI to overcome task paralysis, Google study says<\/a><\/strong><\/p>\n

I’ve re-run all the tests to see how the detectors perform today. While I had two strong successes, the big takeaway seems to be just how inconsistent the results are from one AI checker to another.<\/p>\n

<\/p>\n

What I’m testing for and how I’m doing it<\/h2>\n

Before I go on, though, we should discuss plagiarism and how it relates to our problem. Merriam-Webster defines “plagiarize<\/a>” as “to steal and pass off (the ideas or words of another) as one’s own; use (another’s production) without crediting the source.” <\/p>\n

This definition fits AI-created content well. While someone using an AI tool like Notion AI<\/a> or ChatGPT<\/a> isn’t stealing content, if that person doesn’t credit the words as coming from an AI and claims them as their own, it still meets the dictionary definition of plagiarism.<\/p>\n

Also: The best AI image generators to try<\/a><\/strong><\/p>\n

In this experimental article, I’ve asked ChatGPT to help out. My words are in normal and bold text. The AI’s words are italicized. After each AI-generated section, I’ll show the results of the detectors. At the end of the article, we’ll look at how well the detectors performed overall. <\/p>\n

Here are the test results for the above text, which I wrote myself: <\/p>\n