AI and Independent Learning: What a 1,222-Person Study Found

Research · Education · Practice

An AI assistant can help someone solve a problem correctly in seconds. But does that mean the person has learned how to solve it? A set of three randomized experiments involving 1,222 US-based adults suggests that the answer depends on what the AI does for the learner—and what happens when the help disappears.

The paper, “AI Assistance Reduces Persistence and Hurts Independent Performance” by Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey, was first submitted to arXiv on 6 April 2026 and last revised on 3 October 2026. It is a research preprint, so its findings should be treated as important but not yet as a settled verdict about AI-assisted education.

The central distinction: performance is not the same as learning

When a learner uses AI, the immediate task score can improve because the system supplies an explanation, a hint or even the answer. That score alone cannot tell us whether the learner has built knowledge they can retrieve independently later. To test this distinction, the researchers measured performance both while AI was available and after it was removed.

The paper reports three randomized experiments: two using fraction problems and one using reading-comprehension questions. In total, 1,222 people were recruited through the online research platform Prolific. Participants were adults in the United States—not a sample of schoolchildren—and each task session lasted roughly 10–15 minutes.

What the experiments found

In the first experiment, participants worked on fraction problems with or without an AI assistant. The AI condition used GPT-5 in a sidebar, with the system pre-prompted with each problem and its solution so that participants could request direct answers. After the practice phase, the assistant was removed and participants attempted three further problems independently.

• While AI was available: participants with AI solved more problems correctly and skipped fewer questions.
• After AI was removed: the analysed AI group solved a lower share of the final fraction problems than the control group—57% versus 73%. The difference was statistically significant, with a reported Cohen’s d of −0.42.
• Persistence also mattered: in that first experiment, the AI group skipped more of the final questions—20% versus 11%. This difference was statistically significant, though the effect was smaller than the performance difference.

The researchers then repeated the fraction experiment with methodological changes intended to address a potential confound in the first experiment. The independent solve rate was again lower in the AI condition: 71% versus 77%, a smaller but statistically significant difference (Cohen’s d = −0.19). The overall difference in skipping questions was not statistically significant in this second experiment.

A third experiment tested reading comprehension. After the AI assistant was removed, the analysed AI group answered fewer of the final questions correctly than the control group—76% versus 89%—and skipped more questions. Both differences were statistically significant in this experiment.

These are not proof that AI permanently damages skills. They are evidence that, in these short experimental tasks, access to a system that readily supplies answers was followed by weaker performance on a small set of unaided questions.

Why the type of AI help may matter

One of the paper’s most educationally relevant analyses examined how participants used the assistant. In the second fraction experiment, 61% of AI-condition participants said they mainly used it to obtain direct answers, 27% said they mainly sought hints or clarifications, and 12% reported not using it. The authors found that poorer independent performance and some of the persistence differences were concentrated among those who sought direct solutions.

This comparison needs careful interpretation: participants were not randomly assigned to “answer-seeking” or “hint-seeking” styles, so this part of the analysis cannot establish that one prompt style caused the difference. People who chose different styles may also have differed in other ways. Nevertheless, it offers a useful hypothesis for educators to test: AI that completes the cognitive work may have different consequences from AI that prompts learners to do the work themselves.

How strong is the evidence—and what are its limits?

The study has several strengths. It uses randomized comparisons, includes a replication of the fraction task with improved controls, and tests a second domain—reading comprehension. The findings therefore go beyond a survey asking people whether they feel more or less capable after using AI.

But the boundaries of the evidence are just as important:

• Short exposure and short follow-up: the experiments measured performance immediately after brief AI-assisted sessions. They do not establish whether any difference persists over weeks or months.
• Adult online participants: the findings cannot be transferred directly to children, teenagers, classroom teaching or every educational subject.
• A specific form of assistance: the AI was configured to make direct answers readily available. This is not a test of every AI tutor, guided practice system or classroom implementation.
• Small final test sets: the unaided outcome was measured using only a few questions after the assistance phase, so the size and generality of the effect require further study.
• Not every result was consistent: the overall persistence difference was not statistically significant in the second fraction experiment, even though the performance difference was.

The right conclusion is therefore narrower than “AI makes students less intelligent.” The study suggests a risk: when a learner repeatedly receives solutions instead of practising the reasoning, immediate success may not translate into independent performance once the support is gone. A language-learning example is explored in our guide to AI vocabulary learning and independent word use.

How this fits with the wider research

The result is consistent with a more directly educational field experiment published in Proceedings of the National Academy of Sciences in 2025. In a study involving nearly 1,000 high-school mathematics students, Bastani and colleagues compared an unrestricted GPT-4 interface with a tutor designed with learning safeguards. Access to AI improved performance during practice, but students using the unrestricted version performed worse on a subsequent unaided test than students who had not received that assistance. The scaffolded tutor produced a different pattern, underscoring that the design of the help matters. Read the PNAS study.

The OECD’s Digital Education Outlook 2026 makes a related point: general-purpose generative AI can improve task performance without necessarily producing learning gains when it is used without pedagogical guidance. Taken together, these sources do not show that AI is inherently harmful. They point to a more practical question: is the tool designed to finish the task, or to help the learner become able to do it?

For a broader overview of cognitive offloading and critical thinking, see Prowad’s earlier analysis of AI and critical thinking. For classroom routines that protect retrieval and independent recall, see Prowad’s guide to AI and student memory. For the product-design side of this question, read our analysis of AI tutors and study agents. This article focuses more narrowly on the evidence from randomized tests of what happens after assistance is removed.

What educators can do now

The evidence supports cautious design principles rather than a blanket ban on AI. Teachers and learning designers can test the following approaches:

1. Ask for an attempt before revealing the answer. Have learners write down a first step, prediction or explanation before asking AI for help.
2. Prefer graduated hints to complete solutions. Start with a question or small clue, and reveal more only when the learner is genuinely stuck.
3. Fade the assistance. After a worked example, give learners a similar problem to solve without AI. This checks whether they can transfer the method independently.
4. Make reasoning visible. Ask learners to explain why a step works, identify a possible error, or compare two solution paths—not simply submit the final answer.
5. Assess learning without the tool as well as with it. Use short retrieval questions, oral explanations, or a brief unaided task alongside AI-assisted work. The goal is to measure both supported performance and independent competence.
6. Evaluate the intervention, not just satisfaction. If a school introduces an AI tutor, compare learning outcomes and retention against an appropriate alternative, rather than relying only on usage, speed or student enthusiasm.

These are design implications informed by the evidence, not interventions all directly tested by the 1,222-person study. Their effectiveness should be evaluated in the relevant classroom, subject and age group.

The takeaway: design AI around the learner’s next independent step

The most useful question is not whether AI can produce a correct answer. It is whether the interaction leaves the learner better prepared to reason when the answer is no longer available. The new experiments provide a warning about immediate-answer assistance, while the wider evidence suggests that guided, pedagogically purposeful use may lead to different outcomes.

For schools and universities, this means judging educational AI not only by what students accomplish with it, but also by what they can explain, remember and do independently afterwards.

Sources and further reading

• Liu, G., Christian, B., Dumbalska, T., Bakker, M. A., & Dubey, R. (2026). AI Assistance Reduces Persistence and Hurts Independent Performance. arXiv preprint, submitted 6 April 2026; version 5 revised 3 October 2026. https://arxiv.org/abs/2604.04721
• Bastani, H., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
• OECD (2026). OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education. https://doi.org/10.1787/062a7394-en

Editorial note: The lead study is an arXiv preprint. Its latest version is dated 3 October 2026; the article does not treat it as a newly published journal paper or as definitive evidence of long-term effects.