How to Identify Text Generated by AI?
How do we know if a text was generated by AI or by a human?
As with the boom of AI writing assistant, we are seeing a rise of AI text detection tools and ways to decipher whether text was generated by AI or by a human.
Let’s take a look.
Why You Should Be Aware of AI-generated Text
Authenticity – false information, fake news, misleading content
- Privacy – could be used to impersonate a person
- Quality – poor quality, incoherent, lack of creativity
- Originality – plagiarism
- Compliance – in some industries using AI-generated content is regulated
The above screenshot is from ChatGPT for the prompt: What should the intro of this article be?
How to Detect AI-generated Text
AI Detectors by AI
Currently (although things are moving very fast) several AI ‘detectors’ by AI are flooding into the market. Some of these tools are free, and some are subscription fee-based.
Here are some of the AI-detecting tools you can test and use:
Perplexity refers to the complexity of sentences. Humans write complex sentences and often use different words to refer to the same things. So, lower perplexity can often indicate the use of AI. AI tools write simple, smaller sentences using repetitive words or phrases such as ‘it’ instead of synonyms or relevant words.
Burstiness refers to unevenness in the writing tone and style. Although every person has a unique writing style, the style is more complex and unpredictable. The tone varies too, and so does the choice of words. Sometimes, the sentences could be short, and sometimes they could be long. GPTZero can detect these spikes to indicate if a human or AI wrote the text.
3. GPT-2 Output Detector
4. Giant Language model Test Room (GLTR)
AI Detection by a Human
There are a few limitations with AI detectors by AI – the biggest issue being accuracy. The AI dectection tools are NOT always accurate – they can mislabel both AI-generated and human-written text.
Just like plagiarism tools, AI-text detection tools have different levels of accuracy. Some tools may have an accuracy rate as low as 50%. Others may have higher accuracy, but none of them are 100% accurate, even though they may claim otherwise.
For this reason, using a human in the loop, a real person, to proofread and edit text is a must. Perhaps the goal shouldn’t be to read the text for the sake of trying to determine whether it was written by an AI or a human. Rather the purpose of the human editor is to make sure the text is fitting, high quality, and appropriate for what it is intended for.
Detecting text generated by AI may be necessary. However, it can be challenging to determine whether some text is written by an AI or a human. So, instead of focusing on analyzing each word and the strings of words, it may be much more important to determine if the text is accurate, of high quality, and has all the other criteria that you require for the text. This, at the moment anyway, is best done by a human editor.
AI Educator and consultant