Researchers warn of discrimination through AI in schools

Artificial intelligence is already being used in many areas of daily life. But according to a new study, the use of AI in schools could lead to discrimination against students.

Artificial intelligence is also increasingly being used in the education sector. But a study carried out on behalf of the Dutch government now warns against discrimination.

According to the results, the use of algorithms in educational technology could lead to disadvantages for certain groups. The study points out that learning systems that use AI can disadvantage certain students, even though they are actually intended to offer them personalized educational content.

Disadvantages caused by AI in school

For their study, the researchers examined the use of algorithms in primary and higher education. Accordingly, many schools in the Netherlands use adaptive learning systems that automatically adapt their exercises to the students' performance.

These learning systems are designed to provide targeted support to students. However, if these systems are poorly designed, it is possible to reinforce existing prejudices.

“Unfortunately, prejudices against certain groups of students already play a role in education,” says the report from the Netherlands Institute for Human Rights NL Times quoted. Students from low-income families or whose parents or grandparents have immigrated to the Netherlands are therefore given lower school recommendations – regardless of their test results.

In order to solve this problem and not further increase inequalities, the systems must be tested “rigorously on different student populations”.

AI systems misjudge learning styles

It is also problematic to assess the learning styles of individual students if they deviate from the norm. The systems misclassify children with attention deficits or dyslexia and can thus hinder their progress.

“Every student has the right to develop and be treated equally in education,” the report says. It is particularly important to critically evaluate the implementation of AI-supported learning systems.

Schools must and can impose requirements on software providers in the areas of equal treatment, data protection, autonomy and transparency.

But it's not just schools that are in demand. The Netherlands Institute for Human Rights also holds politicians accountable. They must play a more active role in this process and set clear standards for developers of educational software.

Also interested: