Category: Learning & Training

Universities are already experimenting with AI beyond the classroom. The useful question is not whether a chatbot can speed up a task, but which administrative work can be improved safely, measured fairly, and kept accountable to people.

AI in university administration is becoming a distinct area of institutional practice: staff are testing generative AI for routine communication, document handling, research support, reporting, and decision preparation. But the evidence does not justify assuming that every use saves money or improves services. Institutions need to separate plausible use cases from demonstrated outcomes—and low-risk assistance from decisions that can affect a student’s opportunities or rights.

What the latest evidence says

A September 2026 study from UNESCO’s International Institute for Higher Education in Latin America and the Caribbean (IESALC), prepared with the United Nations University Institute for the Advanced Study of Sustainability, surveyed 200 higher-education institutions in 19 Latin American and Caribbean countries. It found that 87% reported using AI in at least one area, while only 26% had a formal AI strategy. The report also found AI use in teaching and learning at 74% of institutions and in research at 57%.

These numbers describe the institutions surveyed, not every university worldwide. The study is a regional snapshot based on institutional responses, and it does not prove that AI adoption itself improves learning, efficiency, or service quality. Its most important signal for administrators is the gap between experimentation and formal strategy.

A peer-reviewed study of 127 administrative employees across three Greek public universities, published in Administrative Sciences in March 2026, found a generally supportive but conditional attitude toward AI. Respondents prioritized ethics and data protection, were more positive about automating specific tasks than about broad claims of organizational performance, and emphasized training and human oversight. Because the study was cross-sectional and based on self-reports, it measures perceptions—not verified productivity gains.

Together, these sources suggest a practical starting point: begin with bounded, repetitive tasks; protect sensitive data; involve the people who do the work; and measure results before scaling.

Six realistic uses of AI in university administration

1. Student-service information and routine enquiries

A carefully configured assistant can help students find published information about office hours, registration steps, campus services, deadlines, and where to direct a request. It can also help staff draft plain-language explanations or translate routine notices for review.

The system should retrieve approved information rather than invent policy. Every answer about deadlines, eligibility, fees, accommodations, or appeals should point to the authoritative page or office. If the question concerns an individual record or an exception, the assistant should route the student to a person instead of pretending it can resolve the case.

2. Drafting and classifying administrative documents

AI can propose first drafts of meeting summaries, internal announcements, standard replies, and procedural checklists. It may also help classify incoming requests so that staff can route them to the right team.

This is assistance, not automatic approval. A staff member should check names, dates, policy references, commitments, and any statement that could be interpreted as an official decision. Institutions should avoid sending confidential records to consumer tools without an approved data-protection and contractual basis.

3. Scheduling and resource coordination

Scheduling teams may use optimization software and AI-assisted analysis to compare room availability, timetable constraints, equipment needs, or service demand. The most appropriate systems are often narrow tools with explicit constraints, not a general chatbot asked to make decisions from incomplete information.

Any proposed schedule needs checks for accessibility, travel time between locations, staff workload, and conflicts that the source data may not capture. A recommendation should be explainable enough for staff to understand why it was made and to override it when local knowledge matters.

4. Research administration and grant support

Research offices may test AI for summarizing funding-call requirements, preparing internal checklists, identifying missing fields in applications, or drafting project-management updates. These uses can reduce repetitive reading, but the official call text and funder’s rules remain authoritative.

AI-generated summaries should link back to the original clause, deadline, or eligibility condition. Researchers and administrators should verify every requirement, especially when a mistake could make an application ineligible. Sensitive research data, unpublished results, and personal information require separate controls.

5. Quality assurance and reporting

AI can help staff organize feedback, summarize open-text survey responses, compare policy documents, or prepare a first draft of a quality-assurance report. It can also help identify questions that require further analysis.

However, a fluent summary can conceal minority views or overstate a pattern. Keep a traceable link to the underlying data, report how responses were selected and summarized, and let a responsible analyst inspect the evidence. Quantitative claims should be reproduced from the source data, not copied from an AI-generated narrative.

6. Decision support—not automated decisions about people

Analytics may help administrators explore service demand, identify bottlenecks, or test different resource-allocation scenarios. But a model’s prediction is not a fact about an individual student or employee. Historical records can encode unequal access, inconsistent past decisions, or missing data.

Do not let a general-purpose AI assistant independently decide admission, progression, discipline, disability accommodations, scholarships, employment, or access to essential services. For consequential decisions, the institution needs a lawful basis, validated methods, meaningful human review, an explanation appropriate to the context, and a route to challenge an error. In the European Union, certain AI uses in education and vocational training are specifically treated as high-risk under the AI Act; classification depends on the intended use, so institutions should consult the official guidance rather than rely on a vendor’s label.

Why efficiency claims need to be tested

AI vendors often demonstrate a task completed quickly. That is not the same as proving that an administrative service became faster, more accurate, fairer, or cheaper overall. Staff may spend the saved time checking hallucinations, repairing errors, handling escalations, or explaining decisions that the system made opaque.

A 2026 bibliometric review of 279 Scopus-indexed articles on AI in higher-education management found expanding scholarly attention and growing interest in generative AI, ethics, and governance. A separate systematic review of institutional approaches to generative AI found considerable variation in the methods and rigor of the studies it assessed. These reviews help map the field, but neither should be treated as a universal estimate of return on investment.

For each pilot, record a baseline before introducing the tool. Compare similar work before and after deployment, and measure both benefits and costs:

• Time: minutes to complete a case, including human checking and correction.
• Quality: factual errors, incomplete responses, rework, and complaints.
• Access: whether different language groups and disabled users can use the service successfully.
• Workload: whether the tool reduces repetitive work or simply moves it to another team.
• Trust: whether students and staff understand when AI is used and how to request human help.
• Total cost: licenses, integration, security review, training, maintenance, and exit costs.

Seven safeguards before a university deploys an AI tool

1. Define the task narrowly. Write down the user, intended benefit, data inputs, excluded uses, and what the system must never decide.
2. Classify the risk. Separate public-information drafting from uses involving personal records, student opportunities, employment, health, or other sensitive decisions.
3. Minimize data. Do not upload identifiable student or staff records unless the institution has approved the tool, purpose, access controls, retention terms, and legal basis.
4. Keep an accountable owner. Name the office responsible for quality, escalation, monitoring, and suspending the system when problems arise.
5. Test with real scenarios. Include ordinary cases, edge cases, multilingual queries, accessibility needs, and attempts to elicit unsupported policy answers.
6. Train users and publish guidance. Explain what the tool can and cannot do, how to verify outputs, how to report errors, and when human review is mandatory.
7. Plan for change or exit. Check data portability, vendor changes, audit access, incident reporting, and how the institution will continue the service if the tool is withdrawn.

The NIST Generative AI Profile offers a cross-sector risk-management resource for identifying and addressing generative-AI risks. It is not a university-specific legal standard, but its lifecycle approach can help teams document intended use, evaluation, oversight, and ongoing monitoring.

A practical 30–60–90 day pilot

Days 1–30: map and select

Interview frontline staff and student representatives. Map repetitive tasks, their current error rates, and the data they involve. Select one bounded, low-risk workflow—such as drafting replies from an approved FAQ—rather than launching a broad assistant across the institution.

Days 31–60: test in a controlled setting

Use synthetic or appropriately de-identified examples where possible. Compare AI-assisted work with the existing process. Have staff check every output, record failure types, and test whether users can reach a human. Do not scale simply because the demonstration looks convincing.

Days 61–90: evaluate and decide

Review time, quality, access, staff workload, incidents, and total cost with the people who use and receive the service. Publish a short decision note: continue, revise, expand cautiously, or stop. If the evidence is inconclusive, extend the pilot or choose a more measurable task rather than claiming success.

What this means for university leaders

The strongest case for AI in university administration is not a promise to replace professional judgment. It is the possibility of reducing avoidable routine work while making services clearer and helping staff focus on cases that require contextual knowledge, empathy, and accountability.

The latest UNESCO IESALC findings make the implementation gap visible: adoption can spread before formal strategies, governance, and evaluation catch up. Universities should respond by establishing clear ownership and shared rules—not by assuming that informal use is either automatically harmful or automatically beneficial.

For a broader institutional adoption process, see our guide to AI integration in education. For questions about accountability and academic work, read who should govern AI at universities. Teams evaluating a specific product can also use our evidence checklist for AI learning tools and adapt its measurement principles to administrative workflows.

Sources and further reading

• Valentini, A. (2026). AI Implementation in Higher Education in Latin America and the Caribbean. UNESCO IESALC and United Nations University Institute for the Advanced Study of Sustainability.
• Bousiou, O., Paraskevas, M., Kapoulas, V., & Liargovas, P. (2026). Prospects for Integrating Artificial Intelligence into the Administration of Higher Education in Greece. Administrative Sciences, 16(3), 131.
• Murni, S. (2026). Artificial intelligence in higher education management: a bibliometric analysis of challenges, opportunities and future research directions. Discover Education, 5, 276.
• Bacalso, F. D., et al. (2026). Institutional approaches to generative AI management in higher education: a systematic review. Frontiers in Education.
• National Institute of Standards and Technology. (2024, updated 2026). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.
• European Commission. Navigating the AI Act: FAQs and high-risk AI use cases.

Editorial note: Evidence about institutional adoption and staff attitudes is still developing. The workflow and safeguards proposed here are practical recommendations, not a claim that every listed use has already demonstrated causal improvements in university performance.

Generative AI can explain a difficult concept in seconds, generate practice questions, and adapt examples to a learner’s level. But a central question for educators is not simply whether AI makes study easier: does it help students remember, explain, and apply knowledge later, when the tool is no longer available?

A 2026 expert-consensus report from the University of Minnesota’s College of Education and Human Development asked 30 learning scientists and education researchers to assess 65 learning processes; consensus was reached on 42. The authors caution that much empirical research on generative AI and learning remains limited in quality, so the report maps expert judgments and evidence strength rather than proving that every AI use harms memory. The practical task for schools is to design activities that preserve the learner’s own effortful thinking while using AI where it adds value.

Why student memory matters when using AI

Learning involves more than producing a correct answer in the moment. Students need to retrieve ideas, connect them to prior knowledge, and use them in new contexts. If a tool routinely supplies explanations or completed solutions before learners attempt the task, students may finish faster without building the same ability to recall or transfer the knowledge independently.

This risk is often discussed as cognitive offloading: using an external tool to carry out mental work. Offloading is not inherently bad. Calculators, dictionaries, and accessibility tools can free attention for more demanding work. The key question is whether the offloaded task is a prerequisite skill the student is expected to learn, or a burden that prevents access to the learning goal.

Claims about AI and memory should be made carefully. Outcomes depend on age, subject, prior knowledge, task design, the quality of feedback, and whether students later practice without assistance. Correlation between AI use and a learning outcome does not by itself establish causation.

Practice 1: Ask students to attempt retrieval before opening AI

Begin a lesson with a brief no-notes question: ask learners to recall a definition, explain a process, solve a problem, or sketch the main steps from memory. Only after this first attempt should they consult an AI assistant.

This sequence gives the teacher evidence of what students can currently retrieve and makes the AI response a resource for comparison rather than a substitute for an initial attempt. The prompt can be simple: “Here is my explanation. Identify what is accurate, what is missing, and what I should verify. Do not rewrite it for me.”

For younger learners, use short oral responses, diagrams, or worked examples. For advanced learners, ask them to state assumptions and justify each step. Retrieval attempts should be low stakes; the goal is practice and diagnosis, not punishment for forgetting.

Practice 2: Use AI for hints, not immediate solutions

Configure classroom prompts so the tool gives one hint at a time. A useful tutor asks a guiding question, points to a relevant concept, or highlights the next decision without completing the whole task.

For example, rather than asking a chatbot to solve an algebra problem, a student could ask: “Ask me one question that helps me decide which operation to use next. Wait for my answer before giving another hint.” Teachers should test these prompts because a language model may still reveal the solution or provide inaccurate guidance.

Hints should fade as competence grows. If a learner is repeatedly stuck, a worked example may be appropriate—but it should be followed by a similar problem completed independently.

Practice 3: Require explanation and verification

After using AI, ask students to explain the answer in their own words and identify the evidence supporting it. In science, they might distinguish observation from inference. In history, they might check a claim against a primary source. In mathematics, they might substitute a solution back into the original equation.

A practical “explain and verify” routine has three parts:

• Explain: State the reasoning without copying the AI response.
• Verify: Check important claims against the textbook, teacher-provided materials, calculations, or credible primary sources.
• Revise: Identify one change made after verification and explain why it was necessary.

This approach also teaches AI literacy: fluent language is not proof of accuracy, and a confident answer may contain fabricated details, omitted assumptions, or outdated information.

Practice 4: Schedule an independent follow-up

A student may understand an explanation while reading it but struggle to reproduce the idea later. Build a short follow-up into the lesson or the next class: ask learners to answer a parallel question without AI, notes, or copied text.

Use more than one time scale where practical. A brief exit ticket can check immediate understanding; a short retrieval question a few days later can help reveal whether knowledge remains accessible. Keep the questions aligned with the intended learning objective rather than testing trivia unrelated to the lesson.

Teachers should interpret results in context. One quiz is not a complete measure of learning, and performance can be affected by anxiety, language demands, disability, or unfamiliar question formats. Combine short checks with discussion, projects, and other appropriate assessments.

Practice 5: Make the learner—not the chatbot—responsible for the final product

When AI is allowed, define which parts of the process students must perform themselves. They might use AI to brainstorm counterarguments but independently choose and defend their position; ask for feedback on a draft but write the revision rationale themselves; or generate practice questions and then answer them without assistance.

Ask students to submit a brief process note describing how they used the tool, what they accepted or rejected, and which claims they checked. The note should be proportionate to the task and should not become a surveillance exercise. Explain expectations before students begin, including privacy rules and any restrictions on entering personal or confidential information.

For related guidance, see Prowad’s articles on AI feedback and student learning and evidence on adaptive learning in primary mathematics.

How teachers can evaluate whether AI is helping

Before adopting a tool widely, run a small, time-limited pilot tied to specific learning goals. Compare student work completed with AI access to later work completed independently, where that comparison is appropriate and fair. Record the task, the support provided, and the criteria used to assess performance.

Useful indicators include:

• Independent retrieval: Can learners recall the key ideas without the tool?
• Reasoning quality: Can they explain why a solution works and identify assumptions?
• Transfer: Can they apply the concept to a new problem rather than repeat the example?
• Verification: Can they identify and correct unsupported or inaccurate AI claims?
• Equity and access: Do students with different language, disability, device, or connectivity needs receive appropriate support?
• Teacher workload: Does the approach improve feedback and learning enough to justify the time required to supervise it?

Do not treat these indicators as a validated universal scale. They are practical questions for local evaluation. If the pilot shows faster task completion but weaker independent explanations, revise the activity before expanding it.

What the evidence does—and does not—show

Research on retrieval practice and feedback provides a strong rationale for asking students to recall and explain knowledge, while research on generative AI in education is developing rapidly. An expert-consensus report can help identify plausible risks and priorities, but it is not the same as a randomized trial demonstrating that a particular classroom routine improves long-term retention.

Schools should therefore avoid two extremes: assuming AI inevitably damages learning, or assuming that more convenient answers automatically mean better learning. The strongest local decisions come from clearly defined learning goals, transparent tool rules, teacher oversight, and evidence from students’ independent work.

A practical lesson sequence

1. Start with a two-minute retrieval question without AI.
2. Let students use AI for a limited hint or feedback on their attempt.
3. Require them to verify at least one important claim or step.
4. Ask for a short explanation in their own words.
5. Finish with an independent exit question and revisit the concept later.

This sequence does not require every school to purchase a new platform. It can be implemented with existing classroom materials and an approved tool, or adapted to a no-AI lesson where student privacy or access is a concern.

Conclusion

The educational question is not whether students should always avoid AI or always use it. It is whether the design of the activity leaves learners doing the mental work that the lesson is meant to develop. Retrieval before assistance, hints instead of instant solutions, explanation and verification, delayed independent practice, and clear learner responsibility provide a cautious starting point.

These practices should be treated as research-informed design principles to test—not as a guarantee of improved memory. Schools should monitor independent learning outcomes, review evidence as it develops, and adjust their approach when the results do not support the intended benefit.

Sources and further reading

• Roediger & Karpicke (2006), “Test-Enhanced Learning” — experimental evidence on retrieval practice and delayed retention.
• Dunlosky et al. (2013), “Improving Students’ Learning With Effective Learning Techniques” — research review of practice testing and distributed practice.
• What Works Clearinghouse (U.S. Institute of Education Sciences) — evidence reviews of educational interventions.
• UNESCO: Digital Education — guidance on technology, equity, and education systems.
• OECD: Education — international research and policy resources on learning and skills.

As generative AI becomes a routine part of study, the central question for universities is no longer simply whether students can use it. It is whether an assessment still provides valid evidence of what a student knows, can explain, and can do. AI assessment design should therefore start with the learning outcome—not with a detector or a blanket ban.

The timing is important. On 22 September 2026, the European Commission published a summary of two new reports on digital education. Its review of generative AI says that better task performance does not automatically mean better learning, and that benefits depend partly on whether students actively engage with the content. Separately, the UK Quality Assurance Agency (QAA) reported in July that inconsistent AI rules and practices can create confusion and unequal student experiences. UNESCO’s Institute for Information Technologies in Education is holding a webinar on assessment in higher education on 27 October 2026, reflecting how urgently institutions are rethinking the issue.

This guide turns that evidence into seven practical changes. It does not claim that one assessment model suits every subject; the right design depends on the skill being assessed, student needs, available staff time, and the consequences of an incorrect judgement.

Why AI assessment design needs to change

Take-home essays, coding assignments, reports, and short-answer tasks may still be useful. The problem arises when a task can be completed by a general-purpose AI system but the grade is meant to certify that the student independently possesses the underlying skill. In that case, the submitted product may reveal more about the tool than the learner.

A 2026 systematic review published in Assessment & Evaluation in Higher Education distinguishes “AI-resistant” assessment—which tries to restrict AI access—from “AI-resilient” assessment, which continues to measure meaningful human capability even when AI is available. The review synthesised 47 peer-reviewed studies alongside policy documents and sector reports, and cautioned against treating AI detection as the primary response. That is a useful design lens, not proof that every suggested method works equally well in every discipline.

The QAA’s report, The Perfect Storm: AI, assessment and a sector under pressure, identifies risks involving assessment validity, parity of student experience, trust, foundational skills, and institutional capacity. Its recommendations include substantive training for staff and students, involving student voices in policy, and improving consistency. A clear assignment brief is therefore part of assessment quality—not just an administrative extra.

What recent evidence does—and does not—tell us

The European Commission’s 22 September 2026 summary reports that four in ten EU young people aged 16–24 used generative AI for formal education in 2025. It also notes that only 29.7% of EU teachers surveyed in TALIS 2024 had taken professional development related to AI for teaching and learning. These figures describe use and training in a particular region; they should not be treated as global estimates.

The Commission’s review makes an important distinction: AI may help students perform a task, yet the resulting performance does not necessarily demonstrate durable learning. That finding supports assessments that include explanation, independent transfer, and opportunities to check what learners can do without assistance when unaided competence is itself a course outcome.

There are limits. Research on fast-changing AI tools is developing, and findings from one discipline or institution may not transfer directly to another. A July 2026 workshop report from computing educators documents practical strategies such as oral and interactive assessment, process evidence, rubric redesign, and evaluating AI-produced work. The authors explicitly describe it as a report of professional discussion rather than a controlled research study. Use such examples as ideas to pilot, not as proof of impact.

Seven evidence-informed changes to assessment design

1. Start with the capability the grade is meant to certify

Write the learning outcome in observable terms before deciding whether AI should be permitted. Is the student expected to recall core knowledge, interpret evidence, solve a problem, communicate a recommendation, or collaborate effectively with AI? These are different capabilities and may need different evidence.

For each outcome, state what counts as acceptable evidence. If students must learn to calculate a result by hand, include a suitable unaided task. If the goal is to evaluate AI-generated analysis, then tool use can be part of the assessment—but students should still demonstrate that they can judge its quality.

2. Make the rules specific to each task

A single course-wide sentence such as “AI is allowed” or “AI is prohibited” leaves too much open to interpretation. Tell students which uses are allowed for each assessment: brainstorming, language editing, coding assistance, generating a first draft, or producing the final answer. State whether students must disclose use, retain prompts or outputs, cite generated material, or submit a brief process note.

Explain the educational reason behind the rule. A no-AI task may be appropriate when independent performance is the outcome; an AI-permitted task may be better when professional practice includes tool use. Consistency does not mean identical rules for every assignment. It means that differences are deliberate, visible, and connected to learning goals.

3. Collect a small amount of process evidence

Instead of grading only the final submission, ask for a proportionate trace of how the work developed: an initial plan, selected sources, a draft, a revision note, test results, or a short explanation of a major decision. Choose evidence that helps you judge the learning outcome, not a large paperwork burden.

Process evidence is not a perfect authorship detector. Drafts and notes can also be generated or edited with AI. Their value is that they create useful moments for feedback and make the student’s decisions discussable. Tell learners in advance what to save and how it will be assessed.

4. Add a short oral or interactive check where it matters

A brief conversation can reveal whether a student understands a claim, method, calculation, design choice, or limitation in their submitted work. Ask the student to explain one decision, interpret a changed scenario, or identify what would make their conclusion unreliable.

Oral checks need careful implementation. Extemporaneous speaking can disadvantage students with language, disability, or anxiety-related needs, and individual interviews may not scale to large cohorts. Offer accessible alternatives—such as recorded explanations, structured written responses, demonstrations, or small-group questioning—and use a common rubric. The goal is to check understanding, not to reward confidence or accent.

5. Ask students to critique and improve AI output

In suitable subjects, provide a deliberately imperfect AI-generated answer, explanation, code sample, lesson plan, or evidence summary. Ask learners to identify errors, verify claims against sources, test the output, explain why a proposed solution fails, and produce a corrected version.

This can assess judgement that matters in real work. But students must have enough subject knowledge to recognise a plausible mistake; otherwise the task rewards guesswork. Require references to course concepts, data, tests, or authoritative sources rather than accepting “the AI was wrong” as an explanation.

6. Use authentic tasks with meaningful choices and constraints

Give students a real or realistic problem with a specific audience, context, dataset, case, or constraint. Ask them to justify the question they chose, explain trade-offs, and adapt their work when a condition changes. A local case study or unfamiliar dataset can make the reasoning more visible than a generic prompt.

Authenticity alone does not make an assignment AI-proof. Models can generate plausible case analyses and project plans. The stronger design asks students to make consequential decisions, support them with evidence, and respond to follow-up questions. Keep the task achievable and provide enough scaffolding so that openness does not simply increase confusion.

7. Grade reasoning, verification, and transfer—not polish alone

Rubrics should distinguish the quality of the final product from the quality of the thinking that supports it. Depending on the discipline, criteria might include accuracy, evidence selection, reasoning, testing, acknowledgement of uncertainty, revision in response to feedback, and the ability to apply a principle to a new case.

Do not make “AI use” a vague penalty or bonus. If using AI is allowed, assess whether the student chose an appropriate use, checked the result, and took responsibility for the final work. If AI is not allowed for a particular task, explain the conditions and the evidence that will be used to assess performance. Share examples of strong work and borderline cases so that students can interpret the rubric.

A practical assessment blueprint

For a course built around a research report, for example, an instructor could combine:

• Early diagnostic: a short low-stakes task to check essential concepts before students begin the project.
• Milestone: a research question, source log, or draft argument with feedback.
• Final product: the report, solution, or artefact, assessed against transparent subject-specific criteria.
• Verification: a brief oral explanation, demonstration, or structured response to a changed scenario.
• Reflection: a short note identifying a consequential revision, a limitation, and how evidence was checked.

This is an illustrative sequence, not a validated universal formula. Adjust the components and weighting to the course outcomes, cohort size, accessibility needs, and workload. Avoid adding every component to every task; one well-chosen checkpoint may be more useful than a burdensome portfolio.

How to pilot changes without overwhelming staff

1. Choose one high-stakes assignment. Identify where the submitted product may no longer provide convincing evidence of the intended capability.
2. Map the outcome to evidence. Decide what students must demonstrate and which parts, if any, may use AI.
3. Rewrite the brief and rubric together. Add examples, disclosure rules, permitted uses, and accessible alternatives.
4. Run a small pilot. Ask a colleague to review the instructions; collect student questions and note where grading becomes difficult.
5. Review learning evidence. Compare the quality of reasoning, performance on follow-up tasks, student experience, marking time, and differences in access to tools. Do not treat satisfaction or faster completion alone as proof of learning.

Institutions can build on their existing AI evaluation work. See Prowad’s evidence checklist for evaluating AI learning tools and its five-step framework for institutional AI integration. For the learning side of the problem, the article on AI chatbots and higher-order thinking in STEM explores how prompts can support explanation and reflection rather than simply deliver answers.

What institutions should avoid

• Relying on AI detectors as decisive proof. Detection is not a substitute for valid assessment design, transparent rules, and fair procedures.
• Assuming every student has equal access. Paid tools, device quality, internet access, disability accommodations, and familiarity with prompting can affect performance.
• Confusing a polished submission with deep learning. Include evidence that matches the capability being certified.
• Making every task an oral exam. Use multiple, accessible forms of evidence and consider staff workload.
• Changing rules without student input or training. Explain the purpose of the assessment and provide a consistent course-level approach.

Conclusion: design for valid evidence, not a race against AI

The most durable response to generative AI is not to keep inventing tasks that a model cannot complete today. It is to ask what knowledge and judgement learners must develop, then gather evidence that those capabilities are genuinely present. Some outcomes require unaided practice; others benefit from transparent AI collaboration. Good assessment makes that distinction explicit, combines appropriate evidence, and remains fair to students with different needs and resources.

As the European Commission, QAA, and UNESCO discussions indicate, this is a pedagogical and institutional challenge as much as a technical one. Start with one assignment, make the intended learning visible, and evaluate whether the redesign improves the quality of evidence—not merely whether it makes AI use harder.

Further reading and sources

• European Commission (22 September 2026): new reports on AI and digital education in Europe.
• Quality Assurance Agency (9 July 2026): research on AI, assessment, and the student experience.
• Amnie (2026): “AI-resistant and AI-resilient assessment in higher education,” Assessment & Evaluation in Higher Education.
• SIGCSE Virtual Working Group (2026): workshop report on computing assessment in the age of generative AI.
• UNESCO IITE: webinar on assessment in higher education in the era of AI, scheduled for 27 October 2026.

Workplace training platforms increasingly promise to personalise learning: diagnose a skills gap, serve the next activity, and adapt the path as an employee progresses. But does adaptive delivery improve learning, or does it simply make training feel more modern? A 2026 field experiment involving 2,574 utility employees offers a useful, qualified answer. The results are encouraging, but they also show why organisations should measure knowledge and transfer—not just completion rates or user satisfaction.

This article examines the study, explains what it can and cannot establish, and proposes a practical evaluation plan for organisations considering AI training at work. For a complementary perspective on the difference between completing unfamiliar tasks with AI and acquiring transferable skills, see our analysis of generative AI skill acquisition and the 487-worker trial.

What the 2026 workplace-training study tested

Tang and colleagues reported a three-wave, matched-pair cluster-randomised field experiment in six county-level power-supply companies within one Chinese provincial electric utility. Of 3,000 employees invited, 2,574 were included in the analysis. Participants received either knowledge-graph-driven adaptive training or traditional uniform training covering the same syllabus.

The adaptive system used learner-related information to route participants through training content. The researchers assessed training self-efficacy, knowledge and intention to transfer what was learned to work. Knowledge and transfer intention were measured again at a six-week follow-up. The study appeared in Frontiers in Psychology in 2026.

Main finding: the adaptive-training group showed small advantages in training self-efficacy, intention to transfer learning, and knowledge. The reported standardised effects were approximately 0.26 for self-efficacy, 0.25 for transfer intention, and 0.29 for knowledge at the second measurement. At six weeks, the knowledge advantage remained around 0.26 standard deviations.

These are modest effects, not evidence that AI transforms every training programme. In practical terms, the results suggest that a well-designed adaptive system may improve some learning outcomes, but organisations should not promise large productivity gains on the basis of this single study.

Why the findings matter for AI training at work

Workplace learning differs from a school lesson. Employees need to remember procedures, diagnose problems, make decisions under real constraints, and apply skills after the course ends. A training system that personalises the next lesson is useful only if the learner gains knowledge that can be used safely and reliably on the job.

The study is notable because it examined employees in a real utility context rather than relying solely on student samples or laboratory tasks. Its follow-up also moves beyond immediate reactions to a training session. However, the measured outcome was knowledge at six weeks—not a direct audit of workplace performance, error rates, safety incidents or organisational productivity.

For organisations, the most defensible interpretation is therefore narrow: this particular adaptive training package was associated with small, persistent knowledge advantages in the studied setting. Whether the same result would occur in retail, healthcare, manufacturing, public administration or remote knowledge work remains an empirical question. For a related look at how AI can help apprentices attempt tasks beyond their formal curriculum, see our analysis of AI task expansion in apprenticeships. For a broader view of how AI may support workforce skills systems, see our guide to AI in vocational education and training.

What may explain the benefit?

1. Less unnecessary cognitive load

The researchers found that the adaptive condition was associated with lower extraneous cognitive load, with a reported effect of about −0.34 standard deviations. Extraneous load refers to mental effort caused by avoidable features of instruction—for example, confusing navigation, irrelevant material or a poor sequence of activities. Reducing this burden can leave more capacity for the skill being learned.

But cognitive load is not one simple quantity. The study also found a higher intrinsic-load measure in the adaptive condition. That is a reminder that demanding content may still require effort; the goal is not to make every task effortless, but to remove avoidable friction while preserving the thinking needed to master the skill.

2. A better fit between content and the learner

Perceived personalisation fit was also associated with favourable outcomes. Employees may engage more when the content feels relevant to their current capability and job needs. Yet a system should not confuse personalisation with continually giving a learner easier work. A useful pathway can revisit foundational skills, introduce appropriately difficult scenarios, and require the employee to demonstrate competence before moving on.

3. A structured learning pathway

The intervention combined adaptive routing with a particular delivery system. The authors explicitly note that the comparison cannot isolate the effect of the algorithm alone: the adaptive and uniform conditions differed in delivery medium as well as routing. The observed result belongs to the package as tested, not to personalisation as an abstract feature.

Six questions to ask before buying an adaptive training platform

1. What skill is the system meant to improve? Define the job task and the observable standard before choosing a platform. “AI-powered learning” is not a learning objective.
2. Does it diagnose a meaningful gap? Ask what evidence drives the next activity: a knowledge check, demonstrated performance, self-report, or an opaque prediction. Weak inputs can produce confidently personalised but unsuitable pathways.
3. Does it preserve productive difficulty? The system should provide scaffolding without removing the practice, retrieval and decision-making that the employee must later perform independently.
4. Can learners and trainers understand the pathway? Employees should know why an activity was assigned and how to request a review. Trainers need a way to correct mistaken recommendations.
5. What happens to learner data? Establish what is collected, who can see it, how long it is retained, whether it is used in employment decisions, and how employees can challenge inaccurate records.
6. Is there evidence beyond completion and satisfaction? Require a baseline measure, a comparable assessment, a delayed follow-up and—where feasible—an independent measure of job performance.

These checks complement Prowad’s broader evidence checklist for evaluating AI learning tools. The setting differs, but the central principle is the same: assess the learning outcome and the risks, not merely the feature list.

A practical evaluation plan for employers and training teams

Organisations do not need to begin with a large, expensive deployment. A small pilot can test whether the proposed system helps a defined group learn a specific task.

Step 1: Define a job-relevant outcome

Choose one capability, such as troubleshooting a routine fault, applying a compliance procedure, or handling a customer scenario. Set a clear performance rubric and record the starting level. If the target is safety-critical, involve the relevant technical and safety specialists in defining the standard.

Step 2: Compare like with like

Where feasible, compare adaptive training with the current programme using the same learning objectives, time allowance and assessment standard. Random assignment is stronger than comparing volunteers with non-volunteers, because the groups may differ before training begins. If randomisation is impractical, document the differences and avoid treating the comparison as definitive causal evidence.

Step 3: Measure three stages

• Immediate learning: Can participants answer questions or demonstrate the skill after training?
• Retention: Can they still retrieve and apply the knowledge several weeks later without the platform supplying the answer?
• Transfer: Can they perform the relevant task in a realistic work setting, with appropriate supervision and safety controls?

Completion, time-on-task and satisfaction can help explain participation, but they are not substitutes for these outcomes. For a related discussion of feedback design, see why AI feedback is not the same as learning.

Step 4: Check who benefits—and who does not

Average results can conceal unequal experiences. Examine outcomes by role, prior experience, language needs, disability-related access requirements and starting skill level where it is lawful, ethical and statistically appropriate to do so. The study reported exploratory differences by experience, with larger benefits among more experienced employees. That finding should be treated as a hypothesis for replication, not a universal rule for designing training.

Step 5: Set a decision rule before the pilot

Agree in advance what level of improvement would justify the cost and what safety, privacy or fairness problems would stop the deployment. Include trainer time, integration, maintenance, accessibility and employee support in the cost calculation. A system that produces a small test-score gain but adds substantial administrative burden may not be the best option.

What the evidence does not yet prove

Several limitations matter. First, the trial was conducted in six companies within one provincial utility in China. It does not establish that the same effects will generalise to other sectors, countries, languages or organisational cultures. Second, the adaptive and comparison conditions differed in delivery medium as well as routing, so the study cannot attribute the entire effect to the adaptive algorithm. Third, the reported knowledge outcome at six weeks is not the same as independently observed job performance or long-term organisational impact.

More broadly, the 2026 scoping review by Pham and colleagues mapped 59 studies of AI and data-supported workplace learning. It describes uses such as skill-gap analysis, personalised training, feedback and recommendations, while highlighting fragmented systems, data-governance concerns and the need for more longitudinal and experimental evidence. The literature is developing, but it is not yet a licence to treat every AI-enabled learning feature as proven.

For context on how AI intersects with formal skills development, the OECD’s 2026 report on AI in vocational education and training examines the implications for curricula, qualifications and occupational standards. Organisations should connect their own training experiments to those wider workforce and qualification requirements rather than treating software adoption as a strategy in itself.

Conclusion: buy evidence, not the word “adaptive”

The 2,574-employee field experiment provides a useful signal: a particular knowledge-graph-driven adaptive training package produced small advantages in knowledge and related training outcomes, with the knowledge difference still present at six weeks. Its real value is not a blanket endorsement of AI training, but a clearer set of questions about personalisation, cognitive load and evaluation.

For employers, the practical next step is to pilot one job-relevant skill, compare outcomes fairly, check retention and transfer, protect employee data, and scale only when the benefits justify the cost. AI can help organise practice; the organisation remains responsible for ensuring that employees can perform the work.

References and further reading

• Tang, C. et al. (2026). AI-driven adaptive training for utility employees: a three-wave cluster-randomized field experiment on cognitive-load and personalization-fit pathways. Frontiers in Psychology. DOI: 10.3389/fpsyg.2026.1944626.
• Pham, P. et al. (2026). The adoption of AI technologies and data to support workplace learning in the corporate sector: A scoping review. Review of Education, 14, e70198.
• OECD (2026). Developing Vocational Education and Training with Artificial Intelligence.

Research update — 23 October 2026. Adaptive learning in primary maths is often promoted as a way to personalise practice for every child. A research update from Radboud University on 2 October reported a large, multi-year study of 7,885 Dutch primary-school pupils: children using an adaptive learning system showed slightly more positive growth in mathematics performance than peers in classes that did not use it. The result is encouraging, but its limits matter just as much as the headline.

The key lesson is not that software can replace a teacher. It is that adaptive practice may help teachers respond to differences between learners—when it is connected to a clear curriculum, interpreted by educators, and evaluated through learning rather than screen activity.

What did the 7,885-pupil study find?

According to Radboud University’s research summary, researchers examined the development of numeracy skills among 7,885 Dutch primary-school pupils over a three-year period. The data covered the years 2014–2017, when classes using an adaptive learning system were compared with classes teaching mathematics without that system.

The software adjusted the difficulty of exercises in response to pupils’ performance: learners doing well received more challenging tasks, while those struggling received simpler ones. The university reported a small, generally positive trend in maths growth. The effects appeared more evident in larger schools and in schools serving more socio-economically disadvantaged pupils, where differences in prior attainment may be harder for one teacher to address simultaneously.

The study is reported as Arithmetic Development: A Longitudinal Observational Study into the Use of Adaptive Technology, listed by the university as forthcoming in Learning and Instruction. The important distinction is methodological: this was a longitudinal observational study, not a randomised trial. It can identify a pattern across a large sample, but it cannot by itself establish that the software caused every difference observed.

Why “adaptive learning” is not the same as a chatbot

Searches for AI learning tools often group very different technologies together. An adaptive maths platform typically selects or adjusts practice tasks using a learner’s answers and progress data. A generative AI chatbot produces new explanations or responses in natural language. Some products combine both, but the evidence for one type should not automatically be treated as evidence for the other.

The Dutch study examined adaptive learning technology used in 2014–2017—well before today’s generative AI chatbots became widely available. It therefore offers useful evidence about personalised practice and learner progression, but it does not demonstrate that a conversational AI tutor improves maths learning. For a separate discussion of how to judge education tools, see our guide to evaluating AI learning tools in schools.

What the findings mean for primary-school teachers

1. Use software to target practice, not to define the curriculum

Adaptive systems can vary the pace and difficulty of practice, but teachers still need to decide which mathematical ideas pupils should understand. A platform’s next-task recommendation is not a substitute for curriculum sequencing, classroom discussion, or professional judgement. Before adopting a tool, map its activities to the learning objectives and identify which concepts require direct explanation or concrete materials.

2. Ask pupils to explain their reasoning

Correct answers alone do not reveal whether a child understands place value, has memorised a procedure, or guessed successfully. Pair digital practice with brief explanations: “How did you work it out?”, “Why does this answer make sense?”, or “Can you show another method?” Teachers can use recurring errors as prompts for a small-group lesson instead of assuming that a progress dashboard tells the whole story.

3. Check transfer away from the platform

Improvement inside an app is not necessarily the same as durable learning. Periodically give pupils a short, curriculum-aligned task on paper or at the board, without hints from the software. Include a new problem that requires the same underlying idea in a different form. If performance drops sharply when the tool is removed, the next step may be more explanation and guided practice—not simply more app time.

4. Keep the teacher in the feedback loop

The Radboud report explicitly stresses the continuing role of teachers. A useful workflow is to review the platform’s error patterns, compare them with classwork, select one misconception to address, and then check whether pupils can apply the correction independently. Automated feedback can make practice more responsive, but it may not reveal language difficulties, anxiety, gaps in prior knowledge, or the reason a child chose an answer.

5. Test whether the tool is equitable in your setting

The reported stronger pattern in larger and more disadvantaged schools is worth investigating, not a guarantee that every under-resourced school will benefit. Schools should check device availability, connectivity, accessibility, language support, and whether the tasks reflect pupils’ curriculum and context. Ask which learner data are collected, how long they are retained, who can access them, and whether teachers can export or delete them under the school’s policies.

How schools can evaluate an adaptive maths tool

Before expanding a platform, schools can run a small, time-limited evaluation. Agree on the learning objective first, record a baseline measure, and decide what evidence would count as improvement. Where feasible, compare results with a similar class or with pupils’ performance before implementation, while acknowledging that simple before-and-after comparisons cannot rule out other explanations.

• Learning: Do pupils improve on curriculum-aligned questions, including questions they have not practised in the platform?
• Retention: Can they still solve comparable problems after a delay?
• Reasoning: Can they explain a method and detect an implausible answer?
• Equity: Are gains and access similar across groups, or are some pupils receiving less useful practice?
• Teaching: Does the system give teachers actionable information and free time for instruction, or add monitoring work?
• Privacy and reliability: Are data use, security, accessibility and failure procedures documented?

These checks complement, rather than replace, the research literature. A 2025 open-access meta-analysis of 34 empirical studies on learning-analytics interventions reported an overall positive effect on learning outcomes, while also finding substantial variation across studies. That broader evidence concerns learning analytics interventions generally—not only AI systems or primary mathematics—so it supports careful evaluation rather than a universal claim of effectiveness. See the published meta-analysis for methods and limitations.

What the evidence does not yet tell us

Three cautions should shape decision-making. First, a small average benefit can conceal variation between pupils, schools, and implementation models. Second, data from one national context and a particular generation of adaptive software may not transfer unchanged to schools with different curricula, languages, resources, or teaching practices. Third, evidence about adaptive exercise selection should not be used to make stronger claims about generative AI tutors, which introduce different questions about accuracy, explanation quality, privacy, and student dependence.

Research on AI-supported learning more broadly similarly points to implementation as a central issue: a recent review in the Italian Economic Journal argues that the impact of computer-assisted learning and AI-guided tutors depends on curriculum alignment, teacher preparation, interactive design, and sustained human support. Read the open-access review of AI-guided tutors and computer-assisted learning for a broader synthesis.

A practical conclusion for schools

The 7,885-pupil study offers a useful reminder that adaptive learning in primary maths should be judged by the quality of learning it supports, not by the novelty of the technology. Its reported gains were small, the design was observational, and the data predate the current generative-AI wave. The most defensible response is neither blanket adoption nor blanket rejection: choose tools that fit the curriculum, use their data to inform—not automate—teacher decisions, and verify progress with independent tasks and transparent evaluation.

For a related framework on implementation planning, read our article on a five-step framework for integrating AI in education. For subject-specific design principles, see our guide to AI chatbots and higher-order thinking in STEM.

Further reading

• Radboud University: Learning maths goes (slightly) better with AI, but teacher plays a key role (2 October 2026).
• The Effectiveness of Learning Analytics-Based Interventions in Enhancing Students’ Learning Effect (2025).
• Artificial Intelligence in Education: Computer-Assisted Learning and AI-guided Tutors (published online 2025).

Generative AI is often introduced into STEM lessons as a faster way to explain a concept, solve a problem, or produce an answer. A different design goal is emerging: an AI chatbot can be built to make students explain, compare, and justify their own reasoning instead of doing the thinking for them.

A study published online on 14 October 2026 in the Journal of Microbiology & Biology Education offers a timely proof of concept. Researchers developed the UCI Math ThinkBot to prompt fifth- and sixth-grade students to reflect on proportional-reasoning strategies. The study is promising, but it is not evidence that chatbots automatically improve learning. Its value lies in the instructional design choices it makes visible.

This article examines what the study found, what it cannot yet establish, and how teachers and learning designers can apply the principles responsibly.

What the October 2026 study actually tested

Richland and colleagues studied 33 students in grades five and six. After a short lesson on ways to compare ratios, students completed an assessment and then spoke with Math ThinkBot about how they had solved the problems. Rather than delivering a worked solution, the chatbot asked learners to describe their strategy, explain why it worked, compare alternative methods, and reflect on which approach was useful in different circumstances.

The researchers examined the dialogue for “higher-order thinking talk” (HOTT): language that signals comparison, relationships, inference, justification, or generalisation. Their qualitative and frequency-based analyses found HOTT cues in both the chatbot’s prompts and students’ responses. The amount of HOTT language produced by the bot and by children was positively associated.

That is a useful finding about the interaction: carefully designed dialogue can create more opportunities for students to articulate reasoning. But the study was a proof of concept with a small sample, and the chatbot conversation occurred after the lesson and assessment. The researchers did not measure whether the chatbot caused learning gains. Cue-word counts are also only a rough proxy for the quality of thinking; saying “compare” is not the same as making a sound comparison.

Primary source: Richland et al. (2026), “Developing a generative AI chatbot to support students’ higher-order thinking in STEM education,” Journal of Microbiology & Biology Education.

Why the design matters more than the chatbot label

“AI tutor” can describe very different experiences. One system may provide a complete answer after a short prompt; another may ask the learner to explain a hypothesis, inspect evidence, compare strategies, and revise an argument. Both use generative AI, but they create different demands on the learner.

Math ThinkBot was designed to avoid simply supplying the answer. Its prompts were grounded in the relevant mathematics, sample dialogues, common misconceptions, and alternative solution strategies. The researchers also constrained the chatbot to use short, student-friendly turns. These are not minor interface details: they define the pedagogical role the model is being asked to play.

The distinction connects with broader teacher-development work. A 2026 UNESCO IITE analysis of 36 national, institutional, and academic teacher AI competency frameworks stresses that AI competence should be pedagogically grounded and aligned with curriculum, implementation, and assessment—not reduced to knowing how to operate a tool. See the UNESCO IITE report on Teacher AI Competency Frameworks.

Six design principles for AI chatbots that support higher-order thinking

1. Ask for reasoning before offering an explanation

Start with prompts such as “How did you approach the problem?”, “What makes you think that?”, or “Which evidence supports your claim?” The aim is not to withhold help indefinitely. It is to make the learner’s current thinking visible before the system supplies additional information.

For example, when a student solves a ratio problem, the chatbot might ask the student to describe the relationship between the two quantities before showing another method. Teachers can then distinguish a correct answer supported by understanding from one reached by guessing or copying.

2. Make comparison a deliberate learning move

Comparison can reveal the structure of a concept. Ask students to compare two solution methods, models, scientific explanations, or interpretations. Useful prompts include “What is similar?”, “Where do the methods differ?”, “Which approach is more efficient here, and why?”, and “Would your reasoning still work if the numbers changed?”

These questions encourage learners to connect procedures with underlying relationships. They also help teachers see whether a student can transfer an idea beyond the exact example that was practised.

3. Use scaffolding without taking over

A productive chatbot should respond to the learner’s answer with a next step matched to the apparent difficulty. If a response is vague, it can request one specific reason. If the learner is stuck, it can offer a hint, a representation, or a smaller subproblem. It should not immediately replace the learner’s attempt with a polished solution.

Scaffolding needs boundaries. A model can misread a student’s intent, provide incorrect feedback, or keep asking questions that do not fit the curriculum. Teachers should be able to inspect the prompts, correct misconceptions, and intervene when the dialogue stops helping.

4. Ground the chatbot in the curriculum and likely misconceptions

General-purpose chatbots may generate plausible but irrelevant explanations. A subject-specific learning design should specify the target concept, grade level, permitted methods, common misconceptions, expected representations, and the kinds of help the system must not provide. These materials should be reviewed by a teacher or subject expert.

Grounding does not eliminate error. The chatbot should be tested against a set of realistic student responses, including incomplete reasoning, incorrect answers, ambiguous language, and attempts to get the final answer without explanation.

5. Keep each turn short and cognitively focused

Long, multi-part responses can overload learners and make it difficult to know what to do next. One focused question at a time is easier to answer and easier for a teacher to review. A useful turn should serve a clear purpose: elicit a reason, invite a comparison, test a generalisation, or help the learner reflect.

Age, reading ability, language proficiency, disability, and accessibility needs matter. Plain language, flexible response formats, and a way to request a hint can make dialogue more usable without lowering the intellectual goal.

6. End with independent transfer, not a chatbot transcript

A strong interaction should lead to a task the student completes without AI assistance. Ask the learner to solve a new problem, explain a concept to a peer, critique a new claim, or apply the same principle in a different context. This makes it possible to test whether the learner can use the idea independently.

The chatbot conversation itself is not proof of learning. Students may echo the system’s vocabulary, and a fluent explanation produced with support may not be reproducible later. Independent work is therefore essential when the educational goal is durable understanding.

A practical classroom workflow

1. Set the learning objective. Name the concept or reasoning practice students should demonstrate, such as explaining proportional relationships or evaluating evidence for a scientific claim.
2. Collect an initial attempt. Have students work independently first. Preserve their reasoning, not only the final answer.
3. Run a bounded dialogue. Use a teacher-reviewed chatbot prompt that asks students to explain, compare, and reflect. Make clear that the system is a thinking partner, not an answer key.
4. Review the interaction. Sample transcripts for conceptual errors, unhelpful loops, overconfident claims, and whether students did the reasoning themselves.
5. Assess transfer without AI. Give students a parallel problem or short explanation task. Compare the quality of reasoning with the initial attempt using a consistent rubric.
6. Revise before scaling. Discuss results with students and teachers, adjust the prompt or task, and repeat on a limited scale before broader adoption.

How to evaluate whether the chatbot is helping

Schools should measure learning outcomes, not merely how often students use a tool or how satisfied they report feeling. A small pilot can track several complementary indicators:

• Reasoning quality: Does the student justify claims, connect representations, identify assumptions, or compare methods accurately?
• Independent transfer: Can the student solve a new task without the chatbot?
• Misconception handling: Does the system surface and address errors without reinforcing them?
• Participation and access: Who can use the system effectively, and who encounters language, disability, device, or connectivity barriers?
• Teacher workload: Does the design save time overall, or does reviewing transcripts and correcting errors add more work?
• Safety and privacy: Are student data minimised, access controlled, and retention practices explained to families and staff?

Where feasible, compare the AI-supported activity with a credible alternative, such as teacher-led reflection or peer discussion. Use the same learning objective and comparable assessments. If sample sizes are small or conditions differ, report results as exploratory rather than as causal proof.

For a wider institutional checklist, see Prowad’s guides to evaluating AI learning tools with evidence and building an AI integration framework for education. They address adoption and governance; this article focuses on the interaction design that can support students’ reasoning.

Risks and open research questions

Several questions remain unresolved. The Math ThinkBot study involved 33 children, one mathematical context, and reflective dialogue after instruction. It did not establish durable learning gains, effectiveness across other STEM subjects, or superiority over teacher-led discussion. Future studies need more diverse samples, direct measures of learning, independent transfer tasks, and comparisons with alternative instructional approaches.

There is also a measurement challenge. Counting words such as “why,” “same,” or “different” can indicate that a conversation contains reasoning prompts, but it cannot fully capture whether the reasoning is correct, original, or conceptually deep. Human review or validated semantic coding is needed to judge the substance of students’ explanations.

Finally, schools should avoid assuming that more AI interaction is always better. Prowad has previously examined evidence that an AI study assistant can backfire when its design changes how students engage with learning. The relevant question is not whether AI is present, but whether the task preserves meaningful student effort and produces evidence of independent understanding.

Conclusion

AI chatbots for higher-order thinking should be designed to elicit reasoning rather than replace it. The October 2026 Math ThinkBot study offers a concrete example: ask students to explain strategies, compare alternatives, and reflect on conceptual relationships. The result is promising evidence about the potential of structured dialogue, not proof that a chatbot improves achievement.

For teachers and institutions, the most defensible next step is a bounded, curriculum-aligned pilot with teacher oversight and an independent transfer assessment. If students can explain and apply the concept after the chatbot is gone, the design may be supporting learning. If the main output is simply a fluent conversation, more evidence is needed.

References

• Richland, L. E., et al. (2026). Developing a generative AI chatbot to support students’ higher-order thinking in STEM education. Journal of Microbiology & Biology Education. Published online 14 October 2026.
• UNESCO Institute for Information Technologies in Education. (2026). Analytical Report: Teacher AI Competency Frameworks.
• Yang, Y., Chen, L., Sun, D., Zhu, W., & Pang, S. (2026). Fostering preservice teachers’ design thinking by integrating generative AI and analytics-supported reflective assessment in a collaborative inquiry environment. Computers & Education, 249, 105602.