Teaching AI requires more than teaching buttons
A new experiment suggests that students learn to use generative artificial intelligence more effectively when instruction emphasises critical thinking, not just the mechanics of operating software.
Researchers randomly assigned 320 university students to four combinations of instructional style and creative task.
Students who received thinking-based instruction achieved higher scores on a practical AI literacy assessment than those taught procedural techniques.
The advantage appeared under both structured and open-ended tasks, and was larger when the assignment allowed creative freedom.
The findings come from a short experiment at one university, not a long-term trial of an entire curriculum.
What the experiment tested
The researchers asked whether the way AI is taught matters as much as the tool itself.
They compared skills-based instruction with thinking-based instruction.
They also compared a tightly specified image-generation assignment with one that allowed students to interpret a broad theme.
This created four experimental groups, with 80 students randomly assigned to each.
The design allowed the researchers to estimate the effects of instruction and task openness separately and together.
All students worked with the same image-generation platform under standardised settings.
The difference between two lessons
The skills-based lesson explained prompt structures, subject details, style controls and ways to revise outputs.
It focused on the procedures needed to produce an image using a generative AI tool.
The thinking-based lesson instead explored how AI image systems work, their limitations, bias, copyright, privacy and responsible human judgement.
Both lessons were delivered as short videos by the same narrator.
The skills-based video lasted six minutes and 28 seconds, while the thinking-based video lasted six minutes and 30 seconds.
Students could not skip or replay the initial presentation.
The researchers checked that students recognised the intended differences between the two lessons.
The two creative assignments
For the structured task, students had to generate an image containing solar-panel buildings, wind-power facilities, a clear river, sunshine and a photorealistic style.
That assignment specified what a successful result should include.
The open-ended task instead asked students to represent a future of harmonious coexistence between humans and nature.
Students had to decide what that idea meant and how to express it.
Both groups submitted the same types of evidence: a final prompt, a description of revisions and an explanation of why they selected their output.
The researchers could therefore assess the process behind each image, rather than judging the image alone.
A practical test rather than a confidence survey
The study’s primary outcome was an assessed practical task.
Two instructors who did not know the students’ assigned conditions independently scored the submitted materials.
The scoring rubric examined prompt complexity, iterative improvement, critical selection and justification, and ethical consideration.
Each category contributed one to three points, producing a total score between four and twelve.
Agreement between the two raters was high.
The final image itself was supporting evidence, not a separately scored visual-quality contest.
This makes the result more informative than asking students only whether they felt skilled.
Thinking-based instruction performed better
The experiment found a statistically significant advantage for thinking-based instruction.
That advantage held for both structured and open-ended assignments.
It was especially pronounced when students had to interpret the more flexible brief.
The researchers also found a statistically significant overall effect of task type.
The interaction indicated that the combination of teaching style and assignment design mattered.
This is not evidence that procedural teaching is useless.
Rather, a lesson encouraging evaluation and reasoning produced stronger scores on this study’s broader AI literacy rubric.
Why creative freedom changed the picture
An open-ended task asks learners to decide what they want to communicate before they can decide how to generate it.
That places greater demands on interpretation, planning and evaluation.
A procedural checklist may be less sufficient when there is no single predetermined image to reproduce.
The thinking-based lesson encouraged learners to question outputs and explain decisions.
The researchers interpreted the larger advantage under open-ended conditions as evidence of a useful match between instruction and task demands.
The result applies to this image-generation setting and should not automatically be extended to every AI activity.
Autonomy, competence and relatedness
The study drew on self-determination theory, which identifies autonomy, competence and relatedness as important psychological needs.
Autonomy concerns feeling able to make one’s own decisions.
Competence concerns feeling capable of handling a task and improving performance.
Relatedness concerns feeling connected to a wider learning community.
The researchers measured all three after students completed the assignment.
They then examined whether these measures helped explain differences in performance.
These analyses are informative, but the psychological measures themselves were not randomly assigned.
Competence mattered most
In the researchers’ structural model, perceived competence was the strongest psychological predictor of assessed AI literacy performance.
Autonomy was also positively associated with performance.
Relatedness did not have a statistically significant independent association after the other variables were included.
The thinking-based lesson was associated with higher autonomy, competence and relatedness scores.
Open-ended tasks were associated with greater autonomy and relatedness, but not a significant increase in competence.
This suggests that confidence in handling the task and ownership of decisions were particularly relevant in this experiment.
What the mediation results mean
The authors tested whether psychological needs statistically accounted for some of the relationship between the assigned lessons and performance.
For instructional style, the indirect associations ran primarily through autonomy and competence.
For task type, autonomy was the more prominent pathway.
Relatedness did not emerge as an independent mediator of the primary performance outcome.
These are model-based interpretations rather than direct experimental manipulations of the psychological needs.
Random assignment strengthens claims about the lesson and task conditions, but it does not prove every proposed psychological mechanism.
The role of prior AI experience
All 320 participants already had basic experience using AI image-generation tools.
Around 9 percent reported daily use, approximately 65 percent weekly use and 26 percent monthly use.
The distribution of reported usage frequency was similar across the four groups.
The researchers also conducted an analysis controlling for frequency of prior use.
The broad experimental pattern remained the same.
However, frequency is not a complete measure of skill.
The authors noted that future studies should measure baseline proficiency more thoroughly.
One platform and tightly controlled settings
Participants used the web-based Dreamina image-generation platform.
The researchers standardised the account configuration, output dimensions and number of permitted generations.
Students could request up to five rounds, with four candidate images generated per request.
Reference-image uploads and automatic prompt enhancement were prohibited.
These restrictions helped make conditions more comparable.
They also mean the experiment differs from ordinary classroom or workplace use, where students may switch tools and combine several methods.
Results could differ with other AI models or more complex projects.
What the experiment can establish
Because students were randomly assigned, the study offers evidence that instructional and task conditions affected performance during this experiment.
That is stronger for the main experimental comparisons than an observational survey of student preferences.
The study also measured performance using submitted work rather than relying exclusively on self-reported learning.
However, the experiment was short.
It did not measure whether the advantages lasted months later.
Nor did it test whether participants could transfer the skills to writing, coding, research or workplace decisions.
Those questions remain open.
A study-specific scoring rubric
The researchers developed a four-part rubric tailored to the image-generation activity.
The raters showed strong agreement when applying it.
Scores also correlated moderately with students’ perceived conceptual understanding and critical evaluation.
These findings provide useful support for the measure.
Nevertheless, the authors acknowledged that it lacked an external validation sample.
It should not be treated as a universal examination of AI literacy.
The relative advantage of one teaching approach could vary with the skills a different assessment emphasises.
Why relatedness may have mattered less
Students completed their practical assignments individually.
The researchers suggested that this may explain why relatedness did not independently predict performance once autonomy and competence were considered.
Feeling connected to classmates may matter more in projects requiring genuine collaboration.
This is a plausible interpretation, not a result directly tested through a group-work comparison.
A future study could assign students to collaborative and individual tasks.
It could then examine whether the social dimension of AI literacy becomes more important.
For now, the finding is specific to the measured individual activity.
What teachers can take from the findings
The results support including critical evaluation alongside technical demonstrations.
A lesson might teach students to write a prompt, then ask them to explain why they trust an output or why they reject it.
Students can also be asked to identify bias, privacy risks or misleading details.
An open-ended assignment may give learners more opportunities to exercise judgement.
But educators should not interpret the study as an instruction to abandon practical guidance.
Beginners may still need procedural support to operate unfamiliar tools.
The evidence suggests that technical fluency and critical reasoning serve different purposes.
How this connects to other AI education research
Research Today has previously reported that the way students use generative AI can matter more than usage frequency.
Another report examined how free AI tutors engaged South African student teachers while still making errors.
The new experiment addresses a related but distinct question: how educators should structure instruction before students use an AI tool.
Its randomised design provides evidence about a short-term teaching choice.
Together, the topics highlight why access to AI and effective education about AI are not interchangeable.
The limits of generalising the results
All participants came from one university in China.
The experiment involved undergraduates with previous experience of AI image generation.
The teaching videos were brief and delivered in Mandarin.
The activity involved a single platform and a specific image-creation rubric.
Different ages, disciplines, languages and levels of experience may produce different results.
The study did not include a group receiving no instruction.
It therefore compared two approaches, rather than proving either was better than learning without a lesson.
The authors called for longer studies across multiple institutions.
What future studies should test
Researchers could follow students over a semester to see whether performance differences persist.
They could test whether critical-thinking instruction improves fact checking and responsible use in text-based AI tools.
Another experiment could combine the two instructional styles rather than treating them as alternatives.
Larger multisite trials could examine students who have never used generative AI.
Researchers might also test group projects to understand relatedness.
Independent assessments would help establish whether the results reflect broad AI literacy.
Such work could turn a promising short experiment into stronger guidance for curriculum design.
The bottom line
In a randomised experiment involving 320 university students, a brief thinking-based AI lesson produced stronger practical AI literacy scores than a skills-focused lesson.
The difference was found with both structured and open-ended image-generation assignments.
It was larger when students had more freedom to interpret the task.
Autonomy and perceived competence helped explain performance patterns in statistical models.
The study was limited to one institution, one AI tool and a short activity.
It does not establish lasting learning gains or a universal best way to teach AI.
Its central lesson is that students may benefit from learning how to question and justify AI outputs, not only how to generate them.
Source Information
Original study: Designing for AI literacy: psychological mechanisms and self-determination.
Authors: Suqian Yu, Yuxi Lin and Chen Chen.
Journal: Frontiers in Psychology, published 14 September 2026.
Methods: Randomised 2 × 2 experiment with 320 university students, comparing skills-based and thinking-based lessons under structured and open-ended AI image-generation tasks.
