OpenAI's Candid Admission: AI Writing Detectors Are Ineffective

Politics, News Sept. 10, 2023, 5:12 a.m.

OpenAI acknowledges AI writing detectors unreliability, while emphasizing ChatGPTs inability to discern AI-generated text. Human discernment remains key, as AI detectors have high false positives.

Last week, OpenAI released a promotional blog post offering valuable insights for educators, showcasing innovative ways in which teachers are harnessing the power of ChatGPT as an educational tool. Additionally, OpenAI provided a set of recommended prompts to assist educators in incorporating this technology into their teaching methods. In a corresponding FAQ section, OpenAI candidly acknowledged a widely recognized truth: AI writing detectors are ineffective, often erroneously penalizing students with false positives.

In the FAQ segment titled "Do AI detectors work?", OpenAI unequivocally stated, "In short, no." They underscored the lack of reliability in distinguishing between AI-generated and human-generated content among existing detection tools, including those developed by OpenAI.

In a comprehensive exposé we covered in July, experts roundly criticized AI writing detectors like GPTZero, branding them as "mostly snake oil." The inherent flaw in these detectors lies in their reliance on unverified detection metrics, leading to a high incidence of false positives. Ultimately, there exists no foolproof method to consistently differentiate AI-generated text from its human counterpart, as these detectors can be easily circumvented by simple rephrasing techniques. Coincidentally, OpenAI discontinued its AI Classifier in the same month—a tool with a dismal 26 percent accuracy rate in detecting AI-generated text.

OpenAI's new FAQ also dispelled a significant misconception: the idea that ChatGPT possesses the ability to discern AI-generated content. OpenAI clarified, "Additionally, ChatGPT has no 'knowledge' of what content could be AI-generated. It will sometimes fabricate responses to queries such as 'did you write this [essay]?' or 'could this have been written by AI?'. These responses are random and lack any factual basis."

Furthermore, OpenAI acknowledged the propensity of its AI models to generate false or inaccurate information, a concern we have previously addressed. OpenAI explained, "Sometimes, ChatGPT sounds convincing, but it might provide incorrect or misleading information (commonly referred to as 'hallucination' in academic literature). It may even invent quotes or citations, rendering it unsuitable as a sole research source."

In a notable incident from May, a legal professional faced repercussions for citing six fictitious cases obtained from ChatGPT.

While automated AI detectors may prove ineffective, human discernment still plays a crucial role in identifying AI-generated content. Educators, for instance, can rely on their familiarity with a student's typical writing style to detect sudden changes in style or capability. Additionally, some careless attempts to pass off AI-generated work as human-written often contain telltale signs, such as the phrase "as an AI language model." This indicates that someone has copied and pasted ChatGPT output without due diligence. In a recent instance published in the scientific journal Nature, humans identified the phrase "Regenerate response" in a scientific paper—an unmistakable marker originating from ChatGPT's interface.