Generative artificial intelligence is often presented as either a shortcut that threatens learning or a powerful new educational tool. A new South African study suggests that the design of the interaction may be more important than either description.
Researchers tested whether freely available generative AI platforms could be prompted to behave less like answer machines and more like tutors. Instead of simply supplying explanations, the systems were instructed to ask students questions, provide hints, respond to mistakes and progressively guide them through science content.
Across a 10-week intervention involving 42 student teachers, this flipped-interaction approach produced consistently high engagement. Students also reported stronger perceptions of science subject-matter learning from the AI tutor than from established interactive electronic worksheets used as a benchmark. The difference was statistically significant, although the study measured perceived learning rather than objective pre-to-post knowledge gains.
The findings, published on 30 September 2026 in the Canadian Journal of Science, Mathematics and Technology Education, offer a particularly relevant test of educational AI because the intervention took place in South Africa and relied on free platforms rather than a purpose-built commercial tutoring system.
Turning a chatbot into a question-asking tutor
The study focused on 42 third- and fourth-year Bachelor of Education students studying natural and physical sciences across two campuses of a South African university. All participants were Black South Africans under the age of 25 and were preparing to teach mathematics and science at middle- and high-school level.
Rather than building a dedicated intelligent tutoring system, the researchers gave students a roughly 600-word prompt designed to convert a free generative AI platform into what they call a Flipped-Interaction Intelligent Tutoring System, or FIITS.
The distinction is important. Conventional chatbot use often begins with a student asking a question and the AI producing an answer. In the flipped format, the AI takes a more active tutoring role. It asks questions, adjusts the level of challenge, gives feedback and encourages the learner to construct an answer rather than immediately revealing one.
Students alternated between the AI tutor and benchmark electronic worksheets. During odd-numbered weeks they pasted the provided prompt into a free AI platform of their choice and completed the tutoring activity. During even-numbered weeks they worked through interactive electronic science worksheets. The alternating design continued for 10 weeks, followed by a reflective task in week 11.
The researchers collected both quantitative and qualitative evidence. This included 290 written open-ended responses from all 42 students, reflections from 25 students and three group interviews involving 17 students. Weekly questionnaires measured affective, behavioural and cognitive engagement on four-point scales, alongside students’ perceptions of how much new science content they had learned.
For analyses of change over time, the researchers used 110 responses from the 22 students with complete AI-tutor data. Comparisons between the AI tutor and interactive worksheets used data from 32 students who had completed at least one activity of each type.
Engagement was high after an adjustment period
The results suggest that students needed time to adapt to the unfamiliar interaction. Behavioural engagement was comparatively weaker at first, and qualitative responses described uncertainty about how to work with the system. After this initial adjustment, however, engagement remained high.
Students repeatedly highlighted personalisation, relevance and conversational interaction as strengths. The AI could respond to a student’s answer, provide hints when they struggled and alter the difficulty of subsequent questions. Many participants described the experience in human terms, comparing the system with a guide, teacher or lecturer.
Career relevance also mattered. Nearly 60% of the open-ended questionnaire responses discussed how the activity related to students’ future work as science teachers. This suggests the technology’s appeal was not simply novelty. Students could connect the science content they were practising with knowledge they expected to need in their own classrooms.
That positive pattern was not unlimited. Workload pressure reduced engagement for some students, and the ability to select difficulty could be used strategically rather than educationally. Some participants admitted choosing easier levels when deadlines approached, allowing them to complete the task more quickly at the cost of challenge.
Students perceived more learning from AI than worksheets
When the researchers directly compared the two activity types, overall engagement was broadly similar. The clearer difference appeared in perceived subject-matter learning.
Among the 32 students included in the paired comparison, the mean perceived learning score was 1.89 for the AI tutor, with a standard deviation of 0.66, compared with 1.62 for the interactive worksheets, with a standard deviation of 0.83. The difference was statistically significant, t(31) = 2.37, p = 0.02.
This does not establish that students objectively learned more science from AI. The outcome measured what students believed they had learned. The authors explicitly recommend future research using pre- and post-intervention knowledge measures and a control group to test whether the perceived advantage translates into demonstrable subject knowledge.
Still, the comparison is useful because the benchmark was not passive reading. Students were comparing the AI tutor with established interactive worksheets designed for the South African context. The AI therefore had to compete with a structured digital learning activity rather than with no intervention at all.
The tutor was useful, but it was not always right
Accuracy remains one of the central risks of using generative AI for education, and the study documented errors rather than treating the technology as a reliable authority.
Students reported 51 interactions containing an error out of approximately 1,450 interactions, producing an estimated self-reported error rate of 3.5%. Of 23 comments that described errors in more detail, 13 concerned structural problems such as missing multiple-choice options or an incorrect activity setup, two were unclear and eight concerned content errors.
Five of the content-error reports involved incorrect feedback on calculation questions, sometimes because the AI and student used different rounding conventions. All reported errors occurred in ChatGPT interactions, although ChatGPT was also the most frequently chosen platform. Because students selected their own AI systems, the study cannot establish that one platform was intrinsically less accurate than another.
Interestingly, errors were not uniformly harmful. Some students said identifying an incorrect AI answer forced them to apply their own knowledge, check calculations and become more confident in challenging the system. That is potentially valuable, but it also depends on learners having enough prior knowledge to recognise when the AI is wrong.
Why free AI matters in resource-constrained education
The study’s practical significance lies partly in cost. Purpose-built intelligent tutoring systems can require substantial development, technical expertise and institutional infrastructure. A carefully designed prompt can instead repurpose a freely accessible general AI system into a more structured learning tool.
That does not remove barriers. Reliable internet access remains uneven, free AI platforms can change their capabilities or usage limits, and students still need sufficient digital literacy to use them effectively. Yet the approach lowers one important barrier by avoiding the need for institutions to develop a specialised tutoring platform from scratch.
For teacher education, the stakes extend beyond the university student using the tool. Strong subject-matter knowledge is a foundation for explaining concepts, diagnosing misconceptions and selecting useful examples in the classroom. If inexpensive AI tutoring can help future teachers practise that knowledge more actively, the benefits could eventually reach their learners as well.
Important limitations keep the findings preliminary
The study is exploratory and should not be interpreted as proof that free AI tutoring improves science achievement. The sample was small, drawn from a single university context and selected through convenience. Some quantitative analyses relied on only 22 or 32 students after filtering for complete or comparable data.
The central learning comparison was also based on self-reported perceptions rather than an objective knowledge test. Students knew which activity they were using, novelty may have influenced reactions, and the first author was involved in the teaching context. The researchers used mixed methods and triangulation to strengthen interpretation, but these design features still limit causal claims.
The reported 3.5% AI error rate should likewise be interpreted cautiously. It depended on students detecting and reporting errors. Mistakes that students failed to recognise would not appear in that estimate, while the unequal use of different AI platforms prevents meaningful platform-to-platform accuracy comparisons.
Even with these constraints, the study shifts the education debate in a useful direction. The question is not only whether students use generative AI. It is also what role the AI is instructed to play. A system designed to keep asking, probing and responding may create a very different learning experience from one used primarily to deliver finished answers.
Source Information
Study Title: Turning Free-to-Use Generative AI into Flipped-Interaction Intelligent Tutors: Exploring Student Engagement and Perceptions of Learning
Authors: Angela Elisabeth Stott and Stefanus Johannes Scheepers
Journal: Canadian Journal of Science, Mathematics and Technology Education
Published: 30 September 2026
DOI: 10.1007/s42330-026-00523-z
