According to @emollick citing @hamsabastani, a 5-month randomized field experiment in Taipei high schools found that combining an LLM tutor with reinforcement learning for adaptive problem sequencing improved final exam performance by 0.15 standard deviations across 770 Python students, with larger gains for beginners. According to Hamsa Bastani’s thread, all students used the same AI tutor and course materials; only the sequencing differed (adaptive vs fixed), isolating the effect of the reinforcement learning policy on learning outcomes. As reported by the study author, the mechanism appears to be stronger engagement and more productive AI use, inferred from student–chatbot interaction signals and solution attempts. According to the author’s summary, the system personalizes the next problem using interaction data, suggesting a scalable path for edtech providers to enhance outcomes without changing core content. For businesses, according to the thread, this points to opportunities to layer RL-based curriculum sequencing atop existing LLM tutors to drive measurable, test-verified learning gains and target novice learners for outsized ROI.