School systems are using algorithms to evaluate students and make decisions about resource allocation.

• 🏆The Houston Federation of Teachers won an injunction against the Houston Independent School District to pause use of an opaque algorithm, called EVAAS, which helped make decisions about teacher pay and termination.
• In 2024, Nevada’s school system faced criticism for using an opaque machine learning system which diverted critical funding away from some students. The tool, created by a company called Infinite Campus, was meant to identify at-risk students but inexplicably excluded some low-income and even homeless students.

Artificial intelligence algorithms can help evaluate and provide feedback to students’ work, but these systems may be unfairly biased against some students.

• During the height of the COVID-19 pandemic in 2020, International Baccalaureate students felt cheated on their A-level exams when a seemingly-biased algorithm assigned their test grades, sometimes with serious future implications.

Predictive algorithms use past student data to rank students for admissions and recommend them for scholarships. However, this might leave out students who could still enroll even if they don't match those characteristics.

• An investigative report by Brookings.edu showed how automated tools made way for racial and socioeconomic bias in college admissions and financial aid distribution.

E-proctoring software uses AI to monitor students and prevent cheating. However, it has limited accuracy for some demographic groups and raises equity concerns.

• 🏆Facing numerous challenges with the rise of remote proctoring tools during the COVID-19 pandemic, students went online to share stories and push back against unfair practices.
• A reporter explored the impact of remote monitoring tools after a Florida highschooler was wrongly accused of cheating.
• A report by the CDT explores how using automated test proctoring tools during the pandemic created additional challenges for disabled students.

Schools collect personal and academic data about students through various apps. This raises concerns when data is shared or used to make decisions without students’ knowledge. 

• 🏆In 2023, the FTC sued Edmodo, an educational technology company, for using children’s personal information for advertising without proper consent.
• An investigative report by The74 dives into how schools used digital monitoring tools to discipline students for minor offenses rather than help them.

Teachers are seeking tools to tell if assignments are AI-generated or not. However, current AI detection software is unreliable, raising concerns about wrongly accusing students of cheating.

• 🏆Just six months after releasing it, OpenAI discontinued its AI text detection tool due to its “low rate of accuracy.”
• Parents of a U.S. high school student sued their son’s school after he was wrongly accused of cheating by an AI detection tool.