In-depth look into the world of AI detection in education

 

Dr Evans Sagomba
Everything AI

IN this week’s article, I will be answering several questions from our esteemed readers.

 

Since we started this column in December 2024, I have received several questions, and this week I took time to answer a question from two universities.

Can we distinguish with certainty, between content that has been created by human beings and content created by AI applications?

The question of certainty is quite challenging in this case.

 

This is a question that is currently topical in education at this point.

 

I have had the privilege of presenting at several conferences on this topic.

 

There is no doubt that in education, we need to ascertain the authenticity of the assignments we are receiving from our students.

At this point, several AI detection tools are flying all over the place, and they give the probability that the content has been generated by a human or AI (usually it is probably human or probably AI) — what is fun is the tools are also AI tools.

 

So, you have AI trying to dictate another AI (it is like sending a thief to catch another thief).

How does this detection work?

In AI detection, there are tools or software that were developed to identify and determine if a student’s assignments have been written using AI tools such as ChatGPT or if images or videos have been generated by an AI algorithm or a human being.

 

To achieve this, the software works by analysing several things. In particular, they analyse linguistic patterns and metadata.

 

This is what they mainly use to decide if the assignment is AI-generated or human-generated.

Are these tools accurate?

This is a big question, what I can say at this point is accuracy varies from tool to tool.

 

Some tools are accurate, while others have poor detection.

 

They mainly struggle with historical documents or documents that have gone through machine translations.

 

As we speak, these tools are being improved so that their reliability and accuracy are credible.

What influences the accuracy of these tools?

Well, several factors determine the accuracy of AI detection tools.

 

First, it is the type of content being analysed, some tools are good at analysing images and others analysing text.

Second, it is the quality of data that was used to train this tool.

 

Tools that were trained using diverse data sets tend to be more accurate as compared to some that were developed and trained with few data sets.

Third, it is the sophistication of the algorithms being used by these tools.

The tools that use advanced algorithms can detect AI content better because they are capable of recognising even subtle patterns in the content.

Can the tools detect all types of AI-generated content?

I can say significant progress has been made, however, there is still a long way to go.

 

In other words, the tools are not yet foolproof.

 

These days certain videos (deepfake videos) and pictures are very difficult to detect.

 

There is a need to continuously develop these tools so to keep up with the fast-evolving capabilities of AI.

Examples of successful AI detection?

Several universities have managed to detect AI in some essays and assignments.

 

However, it is important to note that there is a current race to develop non-detectable generative AI models.

This means that in the future, AI-generated content will not be detectable.

 

Currently, social media giants like X (Twitter) and Facebook are using AI tools to identify and remove AI-generated fake accounts and posts.

What are the limitations of these tools?

Despite the increase in technological advancement, AI detection tools are not 100 percent accurate.

 

Sometimes these tools can incorrectly identify 100 percent human essays and assignments, and give them a 100 percent AI-generated tag and the other way around.

In some cases, they are even 100 percent wrong altogether.

 

The reason is that they are not capable of continuously evolving at the same pace as the generative AI applications.

 

More so, some of the AI detection tools lack computational resources and expertise.

Can they be improved to be 100 percent?

AI detection tools can develop if their developers implement advanced machine learning techniques that enable the tools to learn from new data and improve from time to time.

 

Also, they are not as well-resourced in terms of financial resources as compared to the money being spent on developing the actual generative AI applications.

Do we need these AI detection tools?

My thinking is we need them, not as policing tools, but as tools of assessment.

 

I know this might sound controversial in education.

 

I will explain this in greater detail in the research papers I am publishing in the next few weeks on the role of AI in education.

As AI technology continues to get more and more complicated the education sector should move in tandem with these advancements.

 

Detection should only be used as an assessment tool and not as a tool to determine educational misconduct.

Bear in mind that the highest number of clients for generative AI applications are students.

 

So, the developers of generative AI applications will develop tools to protect their clients, that is developing non-detectable applications.

 

More so developers of AI detection tools need to develop robust applications that are reliable and can keep pace with the rapidly advancing generative AI tools.

What are my views on allowing students to use generative AI in writing assignments?

My thinking is that first, the developers of generative AI tools should collaborate with universities to develop education-specific tools that help to maintain educational integrity.

These tools should encourage students to research well and cite well.

 

That way there is not much need for academic cheating. On the side of the universities, we can encourage students to use AI and then assess them on how much they will use AI effectively to research and present their work.

If you have specific areas that you need to be addressed in the area of Artificial intelligence (AI), contact the editor or email the author directly and the issue will be addressed in the following week’s column.

Dr Evans Sagomba is a Doctor of Philosophy and, Chartered Marketer (CMktr, FCIM) with an MPhil and PhD. He specializes in AI, Ethics, and Policy Research, and is an AI Governance and Policy Consultant. His expertise extends to Ethics of War and Peace and Political Philosophy. Contact: [email protected]. ORCID: 0009-0007-0681-0329. /Social media handles; LinkedIn; @Dr. Evans Sagomba (MSc Marketing) (FCIM)(MPhil) (PhD)/X: @esagomba.