A team of faculty from the University of Missouri-Kansas City School of Computing and Engineering, Department of Communication Studies and Office of the Provost and Executive Vice Chancellor have embarked on a mission to leverage artificial intelligence and facial recognition technology to create inclusive environments in classroom settings.
Computer Science Professor Yugyung (Yugi) Lee, Associate Professor of Communication Studies Ye Wang and Associate Research Professor Alexis Petri are working to develop a technology system that will help instructors learn how classroom behaviors translate to student success.
The team of faculty has applied for funding to complete two projects using this technology.
1) Using artificial intelligence to help STEM educators teach challenging subject matter and to monitor growth mindset.
“It’s common for teachers to try and relate to students when teaching subjects like math and science by saying things like ‘it’s okay, I wasn’t good at math either,’ but that actually hinders students’ growth,” says Petri.
2) Launch a strategic initiative for UMKC professors to receive real time feedback on implicit bias.
“We want to recognize that as artificial intelligence learns, we are passing on our bias. Everything implicit to us gets passed on to AI. If we can teach AI to analyze human behavior more reliably than humans, it allows instructors to have a more confidential and genuine assessment,”
Using camera equipment to build facial recognition and gesture-recognition technology, the team will be able to monitor classroom settings and further understand the general sentiment of both students and the instructor in real time based on body language, speech and facial expressions.
“We built a Deep Learning model and taught it to detect faces with 8 different emotions (happiness, sadness, fear, anger, surprise, disgust, and neutral). We found that if you have 100 images, 80 of them were accurately labeled with one of these eight emotions by the model while 20 images are not properly labeled. These images may be from your own social network or different people,” said Lee who is working with computer science students to teach the technology to detect these things with 100 percent accuracy. Then, with Wang’s help, the team will be able to teach the technology to analyze the sentiment of every student in a classroom as it relates to content presentation and type without having to identify who the student is.
“Artificial intelligence isn’t about fixing the problem we have with implicit bias, but addressing it. This technology can help us overcome our biases by helping instructors see what’s actually happening in the classroom.”
“We want to recognize that as artificial intelligence learns, we are passing on our bias. Everything implicit to us gets passed on to AI. If we can teach AI to analyze human behavior more reliably than humans, it allows instructors to have a more confidential and genuine assessment,” said Petri.
Petri said if this project gets funded, Provost and Executive Vice Chancellor Barbara Bichelmeyer will conduct workshops on growth mindset.
Wang and Lee are working to establish a baseline with the help of the UMKC Theatre Department. Theatre students will get acting experience by role playing various classroom scenarios in a controlled setting to get exaggerated responses. Using camera footage from the classroom, Wang will create an animated report for instructors to visualize how speech and presentation correlate with students’ response.
To get an accurate report, students will be able to complete diversity information before entering the classroom and receive a QR code that will help the technology pay attention to the class’ racial makeup. For the students’ protection, the diversity information they complete will only include racial makeup and gender identity.
“This is a chance for humanities students to gain research experience, and work in labs when they don’t usually get to do so regularly. Collaboration between humanities and sciences is the future of research,”
This research will be useful for artificial intelligence research since the project will generate meaningful data for advanced research in artificial intelligence/deep learning. It also gives opportunities to three industries – computer science and technology and social sciences – the ability to understand one another better as there are not often a lot of research collaborations like this.
“Artificial intelligence isn’t about fixing the problem we have with implicit bias, but addressing it. This technology can help us overcome our biases by helping instructors see what’s actually happening in the classroom,” said Lee.
Lee has already began working with computer science students on testing and improving facial- recognition technology with the resources available in the School of Computing and Engineering. They are still waiting to hear the results from the three funding proposals they submitted to move forward with building the cloud application to analyze and translate what’s happening in the classroom.
“This is a chance for humanities students to gain research experience, and work in labs when they don’t usually get to do so regularly. Collaboration between humanities and sciences is the future of research,” said Wang.
Several community organizations are already interested in testing the technology when it is complete.
“In elementary education, it is important to support teachers as they set high standards for students and help students meet those standards. Developing math and science interests and capabilities early is important to broadening and increasing STEM majors in college. When teachers who struggle with math unintentionally give students the message that it is ok not to be good at math, it may have a lasting effect on what students pursue later,” Dr. Angelique Nedved, Director of School Quality, School Smart Kansas City.