A question beyond whether students use AI
When universities debate generative artificial intelligence, the discussion often centres on academic integrity, factual accuracy and whether students learn less when software can produce a fluent answer. A different question concerns students’ relationship with their classroom.
Can a carefully designed AI-supported preparation routine help students feel more connected to a course and more willing to engage with an instructor?
A 2026 study in Scientific Reports tested a structured instructional protocol among 220 undergraduates in six class sections at one Chinese university.
Its results suggest that the combined approach may strengthen classroom belonging and emotional engagement while reducing resistance to teacher guidance.
But the most important qualification is statistical: only six classes were randomized, and the study’s exact randomization tests did not reach conventional significance.
That makes the findings promising rather than decisive.
The work is useful because it investigates how AI might be embedded in teaching rather than treating access to a chatbot as an educational intervention in itself.
What the intervention included
The researchers, Renlin Wang, Jian Sun and Shu Wen, examined a sequence that combined AI exploration with reflection and instructor responsiveness.
Students were encouraged to explore material with generative AI, consider how confident they felt about their understanding, formulate questions that remained unresolved and receive responsive teaching from an instructor.
This design matters because each component could contribute to the outcome.
Generating a question may help a student recognise uncertainty.
Reflecting on confidence may reveal that a persuasive answer is not the same as genuine mastery.
Instructor responsiveness may communicate that uncertainty is welcome and that students are part of a learning community.
The study therefore does not isolate an effect of generative AI.
It evaluates a bundled teaching protocol in which software and human instruction operate together.
Any claim that a chatbot alone caused the observed changes would misrepresent the research design.
How the trial was organised
The study involved six intact undergraduate class sections, rather than assigning each individual student independently.
Following a baseline assessment, the sections were randomized equally between conditions.
Three sections, comprising 110 students, were assigned to each condition.
Assessments took place in Weeks 1, 6 and 12, and the intervention began in Week 2.
This is a cluster-randomized design: the class is the unit that receives the assigned approach.
It can be appropriate when teaching methods are delivered to whole groups and would be difficult to separate within a classroom.
Yet it creates a major analytical constraint.
Two students in the same section share an instructor, schedule and learning environment, so their responses cannot simply be counted as independent experimental replications.
Although 220 students contributed information, the trial randomized only six sections.
That difference is central to interpreting every statistical claim in the paper.
The reported difference in belonging
At Week 12, the authors reported a between-condition difference in change in classroom belonging of 0.361, with a 95% confidence interval from 0.145 to 0.576 and a Bonferroni-adjusted p value of 0.029 in their section-level parametric analysis.
In plain language, the intervention classes showed a more favourable change in the belonging measure than the comparison classes under that analytical model.
The number is a difference on the study’s measurement scale, not a percentage increase in belonging, and it should not be described as 36.1% more belonging.
The authors also reported corresponding differences of 0.282 for emotional engagement, −0.497 for reactance to teacher guidance and −15.24 points for a confidence–performance discrepancy index.
These outcomes address different dimensions of classroom experience.
Emotional engagement concerns students’ affective involvement, reactance concerns resistance to guidance, and the discrepancy index concerns the gap between confidence and performance.
They should not be combined into a single claim about improved academic achievement.
Why two statistical results tell different stories
The paper reported exact two-sided randomization p values of 0.100 for all primary contrasts.
This was the minimum attainable value with six balanced sections.
That finding deserves as much attention as the adjusted parametric p value for belonging.
With only six randomized clusters, the number of possible treatment assignments is small.
An exact randomization test evaluates evidence against the assignments that could actually have occurred, rather than leaning on a large-sample approximation.
A p value of 0.100 does not satisfy the familiar 0.05 threshold, although such a threshold is not itself a complete measure of scientific value.
The parametric section-level analysis used equally weighted section contrasts with four degrees of freedom and produced a more favourable significance result for belonging.
Both are reported by the researchers.
Together, they indicate that the direction of the findings is interesting but the inferential strength is limited by the trial’s small number of independently assigned groups.
What a confidence–performance gap means
The confidence–performance discrepancy index, abbreviated CPDI, is an attempt to capture whether students’ confidence aligns with how they actually perform.
This question is especially relevant to generative AI because an answer can sound complete and authoritative even when the user has misunderstood it.
Students may mistake fluency for mastery, or they may feel uncertain despite demonstrating solid understanding.
A reflective preparation routine could encourage them to check what they know and formulate better questions.
The reported difference of −15.24 points on the index is consistent with a more favourable discrepancy measure in the intervention condition, as interpreted by the authors.
However, the index was preliminary.
The study does not establish it as a universally validated measure of AI-related learning or as a diagnostic tool for individual students.
Nor does it show that students using AI necessarily become overconfident.
The index is one exploratory way of studying calibration in this particular educational setting.
The teacher remains part of the mechanism
A distinctive feature of the intervention is that unresolved questions were intended to feed back into instructor responsiveness.
This changes the role of AI from an answer-producing shortcut to one possible starting point for a human learning exchange.
Students may arrive at class with a clearer sense of what they do not understand.
Instructors can then respond to those uncertainties, potentially making the classroom feel more attentive and collaborative.
That explanation is plausible, but the study cannot establish which part of the package produced the measured changes. AI exploration, confidence reflection, question generation and teacher behaviour were implemented together. A future dismantling study would need separate conditions to estimate the contribution of each component.
Without that, the correct conclusion concerns the combined protocol, not a standalone AI feature or a confirmed psychological pathway.
Why belonging matters in education
Belonging is not interchangeable with exam scores.
It describes whether students feel accepted, included and connected within an educational environment.
Such feelings can shape whether learners ask questions, seek help or persist through difficult material, but measuring belonging does not by itself prove improvements in knowledge or long-term retention.
The current trial focused on belonging and related psychological measures over a 12-week period.
It did not demonstrate that students graduated at higher rates, learned more in every subject or maintained the observed differences over several years.
For universities, the practical interest lies in designing instruction that makes students’ uncertainty visible and invites constructive responses.
Research Today has separately covered an experiment on collaborative-learning messages and research on interrupted studying.
Those studies also illustrate that educational engagement depends on the surrounding learning conditions, not merely the presence of a particular technology.
The limits of one university and six classes
The trial took place at one Chinese university.
That limits how confidently the findings can be extended to other countries, disciplines, age groups and institutions. A large introductory lecture, a small postgraduate seminar and an online vocational course may respond differently to the same preparation routine.
The small number of randomized sections also makes the estimates sensitive to differences between classes that randomization may not balance well in such a limited sample.
The authors explicitly treated exploratory individual-level pathway and effect-modification analyses as hypothesis-generating because only six sections were observed and pathway estimates were sensitive to measurement timing.
That warning matters.
If an analysis suggests a pathway through confidence or instructor responsiveness, it should be investigated further rather than presented as an established explanation.
The researchers’ conclusions are accordingly cautious about mechanism and generalizability.
What a stronger follow-up would test
A next-stage trial could randomize substantially more class sections across multiple universities and subjects, preserving the classroom as the unit of assignment.
It could preregister a primary outcome, measure actual learning performance alongside belonging and report the costs and workload associated with instructor responsiveness.
Researchers might also compare several versions of the protocol: AI exploration alone, AI plus confidence reflection, structured question generation without AI, and the complete instructor-responsive approach.
That would help distinguish the value of the software from the value of the pedagogical structure around it.
Longer follow-up would reveal whether changes persist after the intervention ends.
Independent replication would be especially important because the present trial’s exact randomization evidence is constrained by the six-class design.
These improvements are not criticisms of the researchers for asking a small-scale question; they are the logical steps needed to establish a reliable educational practice.
A careful lesson for AI-enabled teaching
The study’s most useful message is not that generative AI has been proven to make students feel they belong.
It is that the design of an AI-supported learning routine may matter more than simple access to the tool.
The intervention connected exploration, reflection, question formation and human response.
In the observed classes, several measures moved in a favourable direction, including belonging, emotional engagement and resistance to teacher guidance.
Yet a small number of randomized sections and non-significant exact randomization tests prevent a definitive causal verdict.
Educators considering similar approaches can treat the protocol as an idea worth testing while monitoring students’ experiences and learning outcomes locally. AI may help students prepare questions; instructors still have to create an environment in which those questions are heard.
The evidence supports thoughtful experimentation, not a universal prescription.
Source Information
Study title: Structured generative AI preparation and classroom belonging in a six-class randomized trial
Authors: Renlin Wang, Jian Sun and Shu Wen
Journal: Scientific Reports
Publication date: 26 October 2026
Study design: Six-section cluster-randomized undergraduate classroom trial with repeated assessments
Sample or scope: 220 undergraduates in six class sections at one Chinese university; three sections and 110 students per condition
Key finding: Week-12 between-condition belonging-change difference 0.361 (95% CI 0.145–0.576) in a parametric analysis; exact randomization p=0.100 for primary contrasts
