An AI tutor raised math scores for middle school students, but only when it refused to hand over answers. That is the central finding from a new study out of the University of Toronto Mississauga, reported by Phys.org.
Economics professor Philip Oreopoulos led the research, which involved more than 6,000 middle school students in Tennessee. The study tested an AI tutor called NUMI, which was built to avoid giving direct math solutions and instead push students to reason through problems step by step.
"The technology now exists with the potential to make AI an exceptionally good tutor. The challenge, however, is that students want to get through their math assignments quickly and easily," says Oreopoulos. "Easier equals less effort and less learning ... The AI tutor has to be designed to teach students how to avoid using the tutor to take the easy route out in solving math problems."
NUMI was designed in collaboration with Michael Liut, an associate professor of mathematical and computational sciences at UTM. The tutor walked students through their mistakes and then required them to demonstrate the same skill correctly three times in a row before moving on. It focused on step-by-step reasoning rather than getting to an answer quickly.
Students assigned to NUMI scored three points higher on tests covering both practiced and unpracticed math problems compared with students receiving conventional computerized instruction. The study was published with the National Bureau of Economic Research as one of three recent working papers on AI and virtual tutoring.
"As a proof of concept, our study may be the first objective evidence to show that a well-designed AI tutor is more effective than instruction without AI," says Oreopoulos.
The result matters partly because of what AI tutoring tools often do wrong. Much of the existing evidence on AI in education suggests it tends to harm learning by instantly providing answers, bypassing the thinking process entirely.
"AI tutoring works by turning students' mistakes into productive learning moments," Oreopoulos explains.
The gap between NUMI and standard instruction was modest, but researchers say it is meaningful given how early this field of study is. Oreopoulos noted that the next step is building and testing a version with more interactive features to bring the experience closer to working with a human tutor.
