The advent of AI has brought with it exciting promises for improving and streamlining workflow in many sectors. From automated content creation to customer service to advanced analytics, the possibilities of large language models (LLMs) seem to have no end. However, one industry that AI has entered has sparked a large amount of controversy: education.
After the introduction of sophisticated chatbots, users quickly realized the possibility of generating entire essays. The appearance of apparently AI-generated homework disconcerted educators in classrooms across the globe. As AI continues to advance as a technology, its potential for misuse as a cheating tool only strengthens. In response to rising concerns over AI-generated work in classrooms, Turnitin, an anti-cheating platform, expanded its focus from checking work for plagiarism to detecting the use of AI.
While this may seem a boon for worried teachers, it prompts the question: How accurate is the Turnitin AI detector? The answer could have implications for the future of academia and suggest possible solutions for implementing AI in tomorrow’s classrooms.
The Need for AI Detection in Academia
Since their introduction to the public, an extensive application for LLMs has been as generative writing tools. Popular AI apps, such as ChatGPT, can generate entire essays with the input of a prompt. While this comes with a variety of uses, such as for aid in drafting emails and reports, it can be a tempting shortcut for students who have been tasked with answering a teacher’s writing prompts.
When the work of tools such as ChatGPT began showing up in classrooms, educators responded with concerns about this new trend and the long-term impact it will have on their students. Having an AI write your essay may sound like a convenient path to a good grade, but not doing the work means students will miss out on the opportunity to learn. To make matters worse, while AI-generated content can sometimes be easy to spot, the sophistication of text-based AI tools can be convincing enough to pass as human-made.
Using generative AI to write an assignment is different from more conventional means of academic dishonesty, such as plagiarism. A copied text can often be spotted by searching for the original work online or in a database. However, generative AI uses the broad dataset of LLMs to create new responses, which would not be detected by traditional plagiarism detectors. To help educators adapt to this change in technology, Turnitin, a popular anti-cheating platform used broadly across academia, introduced AI detection into their platform.
How Turnitin’s AI Detector Works
The role of the Turnitin AI detector is to determine how much of a submitted work has been generated by a text-based chatbot. Most AI detectors function in a similar method as the AIs they are designed to detect, in which they are trained on large sums of data of both human and AI-generated works. The detector then compares its dataset against the works it analyzes and flags markers associated with AI.
Turnitin’s detector further implements a concept known as perplexity, which refers to the predictability of a text. Generally speaking, a human-written text will have a greater degree of perplexity or unpredictability than an AI-generated one, as AI models tend to follow more patterns in their writing.
Another factor that Turnitin’s detector examines is “burstiness,” or the variance in sentence length and construction. While humans tend to integrate a mix of long and short sentences into their writing, with variations in their syntax, AI writing tends to be more uniform.
The Accuracy of Turnitin and Other AI Detectors
Since its introduction, the Turnitin AI detector has helped many educators identify assignments that may have been generated by AI. However, all AI detectors are fallible and can produce false positives. This may lead to situations in which a student’s work can be erroneously identified as AI-generated.
False negatives, or instances in which AI-generated work is not flagged by the detector, are also a possibility. As AI chatbots grow more advanced in their capabilities to mimic human writing, instances of false negatives may become increasingly likely.
Applications of AI Detection in Education
Despite the fallibility of AI detectors, AI detection tools such as Turnitin remain valuable in the classroom. These tools can still identify AI-generated works on a large scale, allowing educators to maintain academic integrity in their courses. Teachers who utilize platforms such as Turnitin should understand their capabilities and limitations, take a balanced approach to their students’ work, and apply their best judgment.
Although the preliminary response to the introduction of AI to education may have largely been one of concern over academic dishonesty and lost opportunities for learning, a more fundamental understanding of AI and its possibilities may help better prepare students for an AI-driven world.
The Future of AI in the Classroom
As AI detectors like the one offered by Turnitin continue to help educators identify AI-generated works, new trends in AI use in the classroom have emerged. Out of 200 million papers analyzed by the Turnitin AI detector, only 3% were found to have over 80% AI-written text. This may indicate a trend in which AI chatbots are implemented less as a replacement for writing essays and more as a helpful writing tool for ideas.
However, there is no doubt that AI applications are still being used in the classroom and beyond. To help students prepare for an AI-dominated future, educators may want to focus less on avoiding AI altogether and more on teaching students how to ethically integrate AI into their work. AI doesn’t appear to be going away anytime soon, so learning how to work with generative AI models could prove to be a valuable job skill after graduation. AI also has potential as an aid for teachers, such as helping them develop lesson plans or streamline their grading.
The Implications of the Turnitin AI Detector
Altogether, the Turnitin AI detector can be a useful tool, if not the final solution to AI in the classroom. As generative text chatbots such as ChatGPT go through more evolutions and acquire greater degrees of sophistication, AI detectors will similarly need to adapt to keep up. However, AI detection does not need to be where AI education stops. As educators, students, and platforms such as Turnitin grapple with the implications of AI, a broader understanding of the uses of this technology may be needed to prepare for what lies ahead in tomorrow’s classrooms.
