The way AI feedback is arranged may matter
When students are given feedback on an essay, the challenge is not just understanding the comments.
They also have to decide which changes to make before their revision deadline.
A new classroom study examined whether presenting large language model feedback under the headings of an assessment rubric helped students revise their arguments more actively than a less structured list.
The results point to a promising design idea, but not a proven achievement gain.
Students whose AI comments were tied explicitly to rubric criteria made more changes on several measures.
Their adjusted final scores were slightly higher too, yet the main statistical comparison did not establish a significant difference among conditions.
A four-week experiment in real classes
Jing Wei studied 105 second-year Economics students in China during four weeks of classroom work.
The students were grouped into three existing classes, each assigned to one revision condition.
In one class, GPT-4o-generated comments were organised under rubric-aligned headings.
A second class received non-anchored AI feedback, and a third used a criteria-aligned self-revision checklist without the same feedback arrangement.
This was an intact-class quasi-experiment rather than individual random assignment.
That design made the activities realistic for a classroom but also meant class characteristics and instructor context could influence the outcome.
Because just one intact class represented each condition, the study could not confidently separate all differences attributable to feedback organisation from differences among classes.
What rubric anchoring actually means
A writing rubric describes the qualities used to assess a task, such as a clear claim, supporting evidence, reasoning and a counterargument.
In the rubric-anchored condition, the AI feedback was grouped according to such criteria.
The aim was not to make the model more knowledgeable but to make its comments easier for learners to connect with their assignment objectives.
A flat list of comments can contain useful advice yet place more work on students to decide what deserves attention.
Organised feedback could provide an implicit checklist for revision.
Importantly, however, the actual prompts used in the conditions differed, so the study does not isolate presentation layout alone as an independent causal ingredient.
Three different kinds of evidence
The researchers did not rely solely on the final score.
They gathered three kinds of information: rubric-scored argumentation performance, traces of changes students made between drafts, and evidence about whether students took up suggestions from the AI.
Those measures answer different questions.
A final score reflects how raters judged a piece of writing at a particular time.
Revision traces indicate the amount of textual change.
Suggestion uptake concerns whether proposed edits were acted on.
More words added or deleted is not inherently better writing; students can also make extensive edits that leave an argument weaker.
The value of collecting these measures together is that a small difference in final scores can be understood alongside what students actually did.
The writing assessments
Students completed two practice tasks involving draft, review and revision cycles.
The study also included a baseline assessment and a posttest completed without feedback.
The posttest was scored using the academic-argumentation rubric, with a maximum of 24 points.
The author compared covariate-adjusted posttest means across conditions to account for measured differences at the outset.
Descriptive adjusted means were 14.64 points for rubric-anchored feedback, 14.21 for non-anchored AI feedback and 14.05 for the self-revision comparison.
That ordering is interesting but narrow.
It cannot be expressed as a substantial demonstrated improvement in academic ability, and the study’s naive individual-level ANCOVA did not return a statistically significant condition effect.
What a non-significant comparison means
It would be incorrect to report that rubric-anchored feedback significantly raised student scores.
The adjusted scores differed by 0.43 points between the two AI conditions on a 24-point scale, while the highest and lowest conditions differed by 0.59 points.
Those numbers describe the sample rather than establish a stable effect that will generalise.
Because the three classes were the units assigned to condition, treating the 105 students as if each had been independently assigned can understate uncertainty.
The author explicitly identified the one-class-per-condition structure as a limitation.
This does not mean the observed ranking is meaningless.
It means a larger cluster-randomised test is needed before claiming that rubric headings cause higher assessed performance.
There was more activity in the revisions
Revision traces provided a second view.
Across the two practice tasks, the rubric-anchored group showed larger magnitudes of textual change than the alternatives on several indicators, including words added, deleted and net change.
The paper interprets these as descriptive process patterns rather than definitive proof that each change was useful.
In a time-limited classroom, a student might act on feedback more quickly when it appears next to an explicit criterion.
But the amount of editing can also reflect initial draft quality, personal strategy, familiarity with the tools or the difficulty of responding to comments.
A productive next study should distinguish surface-level rewriting from changes that genuinely strengthen the logic and evidence of an argument.
A closer look at suggestion uptake
The author also coded feedback uptake in an exploratory subsample of 24 students.
The coding focused on suggestions concerning claim, evidence, reasoning or counterargument, asking whether each suggestion was fully or partly used.
In that smaller analysis, the proportion of suggestions adopted or partially adopted was 75.0% with rubric-anchored feedback and 69.0% with non-anchored feedback.
That six-percentage-point descriptive difference is consistent with the idea that students can better connect organised feedback with their revision decisions.
Yet a 24-student subsample is not enough to claim a broadly reliable uptake effect.
It also does not show whether the adopted advice was always appropriate or whether revision quality improved as a direct result.
Why the active comparison matters
The study’s third class did not simply receive no task or no guidance.
Students in that condition undertook criteria-aligned self-revision.
This provides an active comparison with a form of self-directed support that instructors might already use.
The question was therefore not whether advice of any kind beats complete inactivity.
It was whether AI-generated suggestions arranged in different ways change classroom revision processes relative to a criteria-based self-review activity.
That makes the study more relevant to teaching practice, but it remains one local implementation of a particular model and set of classroom prompts.
A study using other subject areas or a different language model could produce another ranking.
How the study avoids overclaiming AI’s contribution
The research does not demonstrate that AI-generated suggestions were always accurate, that students learned to write independently over a long period, or that teachers can delegate assessment to a model.
It evaluated an approach to structuring comments while students worked on short cycles of academic argumentation.
An effective rubric can still depend on thoughtful human assessment and on whether students understand the disciplinary expectations behind each criterion.
Automatically generated comments may sometimes reward a formulaic argument rather than original reasoning.
The present paper does not establish the frequency of such errors; those are relevant questions for future evaluation, not findings to import into the current results.
A classroom can make feedback easier to act on
For instructors who already use AI-generated writing comments, the practical idea is straightforward: present feedback in a format that shows how it relates to stated learning objectives.
This may reduce the burden of sorting through a long list of suggestions, especially where learners have little time between draft and final submission.
But the study has not demonstrated that any one interface design will improve grades or transfer to new assignments.
Teachers could pilot different layouts, inspect the resulting edits and gather student feedback about which comments were useful.
A clear rubric also helps define what the model should address rather than treating polished wording as a substitute for well-supported reasoning.
Where educational fairness enters
A feedback tool that encourages more extensive revision could widen opportunities for some students while placing others at a disadvantage if they lack reliable access, confidence or the time needed to interpret suggestions.
The Chinese classroom study did not establish such distributional effects, and it should not be represented as evidence about unequal outcomes.
Still, equitable implementation deserves attention when a promising design feature moves beyond a trial.
Institutions should consider who can access the tool, whether comments remain understandable for learners with different language backgrounds, and how assessment rules distinguish acceptable assistance from unacknowledged authorship.
These choices depend on educational objectives and local policy, not solely on a model’s technical performance.
Related results are not direct replication
Research Today previously covered a study combining AI and peer feedback in English writing and research on automated essay scoring and its limits.
Those papers focus on different interventions and outcomes.
They should not be pooled informally with this 105-student classroom comparison.
Together they illustrate a broad methodological distinction: the ability of a tool to produce or organise comments is not the same as demonstrated learning improvement.
Researchers need both accurate measurement of the learning outcome and careful observation of students’ actual engagement with suggestions.
How to test the idea more convincingly
A stronger follow-up would randomly assign a larger number of classes to feedback formats while keeping the underlying advice and instructor procedures as comparable as possible.
It could measure the quality of changes rather than simply their volume, use independent raters unaware of condition and assess writing on a new task weeks later.
Where possible, analyses should account for the fact that students share a classroom environment.
Such studies would help determine whether rubric anchoring improves revision decisions, whether any effect persists and whether the design works in other subjects.
A separate experiment could manipulate only the layout of feedback while holding its wording constant, isolating the role of organisation more directly.
The interpretation to retain
This study offers a useful process clue, not a verdict that AI feedback improves essay scores.
Among 105 Economics students in three intact classes, rubric-anchored comments were linked to greater textual revision activity and higher descriptive uptake of argument-related suggestions.
Adjusted final scores followed the same descriptive ordering, but the condition comparison was not statistically significant and the design could not isolate rubric anchoring causally.
The reason for taking this result seriously is that feedback usefulness involves more than generating fluent advice: learners must be able to decide which suggestions address their actual assignment criteria.
Whether that design feature reliably improves learning remains an open empirical question.
Source Information
Study title: Rubric-anchored large language model feedback for academic argumentation revision
Authors: Jing Wei
Journal: Scientific Reports
Publication date: 10 November 2026
Study design: Four-week, three-intact-class quasi-experiment comparing rubric-anchored AI feedback, non-anchored AI feedback and criteria-based self-revision
Sample or scope: 105 second-year Economics students in China; exploratory feedback-uptake coding in a subsample of 24
Key finding: Adjusted posttest means were 14.64, 14.21 and 14.05 out of 24, without a significant Condition term; descriptive feedback uptake was 75.0% versus 69.0% in the exploratory subsample








