Publié le par — Laisser un commentaire

Generative AI in Programming Education: What 46 Studies Reveal

Computer science students learning programming on laptops in a university classroom

Generative AI can now explain a programming concept, suggest code, identify a likely bug, and produce a working example in seconds. That changes the practical conditions under which students learn to program. It does not, by itself, answer the educational question: can a learner design, explain, test, and repair a program without the assistant?

A systematic review published in September 2026 mapped 46 peer-reviewed empirical studies of generative AI in higher-education programming courses. A second, more recent field evaluation followed 357 learners using a teacher-governed adaptive large language model (LLM) tutor. Read together, these sources point to a useful distinction: AI can make practice and feedback more available, but stronger formative activity is not automatically the same as stronger independent achievement.

This guide explains what the research can and cannot establish, then turns it into a practical design for instructors teaching Python, Java, or introductory computing. The goal is not to ban coding assistants or let them complete every task. It is to make AI support visible, bounded, testable, and gradually less necessary.

What the 46-study review reveals about generative AI in programming education

Yalçın, Bakır-Yalçın, İslamoğlu, and Ursavaş published Mapping Research on Generative Artificial Intelligence for Programming Education: A Systematic Review in the Journal of Computer Assisted Learning on 21 September 2026. Following PRISMA guidelines, the authors analysed 46 peer-reviewed empirical studies in higher education.

Several patterns matter to course designers:

  • ChatGPT dominated the reviewed research. It appeared in 76% of the studies. This describes the literature reviewed, not the share of all students or institutions using ChatGPT.
  • Java and Python were the most frequently studied languages. The evidence therefore speaks most directly to common university programming contexts, not every language, age group, or vocational setting.
  • The research was geographically broad but still exploratory. Studies came from 28 countries, while nearly 70% appeared in conference proceedings.
  • More than half of the studies lacked an explicit pedagogical framework. This is a central weakness: tool use is easier to describe than the learning theory and instructional sequence that should guide it.
  • Reported benefits included engagement, feedback, and learning support. The review also identified concerns about over-reliance, weaker conceptual understanding, and academic integrity.

The review does not justify a universal claim that AI improves programming achievement. It maps a growing and varied research field. Different studies use different tasks, tools, measures, course designs, and comparison conditions. A student finishing a coding exercise faster is not necessarily learning the concepts more deeply.

Why faster code completion is not the same as learning to program

Programming competence combines several capabilities: understanding a problem, decomposing it into steps, choosing data structures, writing syntax, predicting behaviour, testing edge cases, locating faults, and explaining design decisions. A generative model can help with some of these activities, but it can also conceal which parts the student can perform independently.

Consider a beginner who asks an assistant to write a loop. The returned code may run correctly. Yet the student may not understand the loop condition, why the variable changes, what happens at the boundary, or how to diagnose an off-by-one error. The completed task is evidence of a working program; it is not sufficient evidence of the learner’s mental model. A June 2026 scoping review of generative AI for computational-thinking education identified the same tension: beginners may offload too much thinking, while advanced learners may underuse AI in complex projects. Its authors proposed seven design guidelines rather than a one-size-fits-all tool rule (Ouaazki et al., 2026).

There is an opposite risk too. If a course prohibits every AI interaction, students may lose opportunities to compare explanations, receive timely debugging hints, or learn how to evaluate generated code responsibly. The instructional choice is therefore not simply use versus no use. It is which assistance is permitted at which stage, for what learning purpose, and with what evidence of independent understanding.

What a field evaluation of 357 learners adds

A study by Pavel Bartoš, Adaptive large language model scaffolding supports online programming education through learning analytics and knowledge tracing, was published in Discover Education on 17 September 2026. It evaluated a low-stakes design combining a map of programming knowledge components, learning analytics, teacher-authored recommendations, a constrained LLM tutor, and explainable records.

The field evaluation included 357 consenting learners, 8,568 pre- and post-assessment item records, 6,532 formative events, and 1,168 assistant sessions. The study compared adaptive and comparison conditions using the same core materials and assessments. The authors note that missing archival details required the findings to be interpreted quasi-experimentally rather than as a definitive randomised causal test.

The adjusted difference on the primary post-test outcome was 3.07 percentage points, with a 95% confidence interval from -0.87 to 7.02 and p = .127. That result was not statistically significant. Adaptive learners showed a small raw-gain advantage (Hedges g = 0.21) and stronger formative and survey indicators, but course completion was unchanged.

This is a useful corrective to exaggerated claims. Teacher-governed AI scaffolding may help learners practise, seek help, and feel supported. In this study, however, those process signals did not translate into a statistically significant improvement in the primary final assessment. The authors recommend keeping such systems transparent, auditable, and low-stakes—not treating them as authorities for grading or certifying mastery.

A new programming pedagogy model: useful proposal, not yet a proven intervention

On 28 October 2026, Bringula and Heintz published a perspective article in Frontiers in Artificial Intelligence, Large language model-assisted modern programming pedagogy model: a perspective on teaching programming in the era of LLMs. It proposes the LLM-Assisted Modern Programming Pedagogy (LAMPP) model.

The model connects four parts of programming pedagogy—teaching, lesson design, learner engagement, and assessment—with ethics as a cross-cutting concern. It draws on constructivism, social constructivism, cognitive apprenticeship, and self-regulated learning. Its central idea is that the educational value of an LLM depends on how teachers orchestrate the activity, not on the presence of the tool alone.

Because this is a perspective paper proposing a framework, its mechanisms should be treated as design propositions for educators and researchers to test, not as experimentally confirmed effects. It is nevertheless consistent with the systematic review’s finding that many studies have not made their pedagogical assumptions explicit.

Six ways to use AI coding assistants without outsourcing the learning

1. Require a first attempt before the assistant responds

Ask students to write a short plan, pseudocode, flowchart, or initial solution before opening an AI assistant. The first attempt gives the learner something to compare with the generated suggestion and gives the teacher evidence of prior reasoning. For a beginner’s Python exercise, the student might identify the inputs, expected output, and sequence of operations before requesting a hint.

2. Ask for a hint or explanation, not a complete solution

Teach students to request one next step: explain an error message, define a concept, suggest a test case, or point to the relevant line without rewriting the whole program. Instructors can provide a set of approved prompt patterns. The aim is to keep the learner responsible for deciding what to change and why.

3. Make debugging a reasoning task

When a program fails, students should state what they expected, what actually happened, and which evidence narrows down the cause. An assistant may suggest possible faults, but the student should verify each suggestion by inspecting the code, running a test, and explaining the result. This turns debugging from copying a patch into a repeatable problem-solving process.

4. Compare generated code against tests and requirements

Ask students to review AI-generated code for correctness, readability, edge cases, security, and alignment with the assignment. Require at least one test that the generated solution initially fails or one change the student can justify. A plausible answer is only a hypothesis until it has been tested against the specification.

5. Fade assistance as competence develops

Early practice may permit conceptual explanations and small hints. Later tasks can restrict AI to documentation lookup or debugging questions, followed by an independent exercise. The amount of support should respond to the learner’s demonstrated understanding, not simply to a fixed number of weeks or prompts.

6. Assess the process as well as the final program

Use short code walkthroughs, live modifications, oral explanations, debugging demonstrations, and project reflections alongside submitted files. Ask a student to change one requirement and explain which parts of the program must change. These methods do not need to become high-pressure oral examinations; short, low-stakes checks can reveal whether the student understands the code they submitted.

A practical lesson sequence for an introductory programming class

  1. Set one observable objective. For example: students will use a loop to process a list and explain how the loop terminates.
  2. Model the concept without AI first. Demonstrate a small example and ask learners to predict the output before running it.
  3. Give students an independent attempt. Collect their pseudocode or first version so that later changes can be interpreted.
  4. Allow bounded AI support. Students may ask for a hint, explanation, or test case, but not request a complete submission. They record what they asked and what they accepted or rejected.
  5. Require verification. Students run normal, boundary, and invalid-input tests where relevant, then explain at least one decision they made.
  6. Finish with an unaided transfer task. Change the input, add a condition, or ask students to trace a similar program without the assistant.
  7. Review and adjust. Compare the independent work with the initial attempt, discuss common errors, and refine the AI-use rules for the next lesson.

How to assess programming learning when AI is available

A balanced assessment should separate the quality of the program from the learner’s ability to explain and adapt it. One practical rubric can score each dimension from 0 (not yet demonstrated) to 2 (demonstrated independently):

  • Problem decomposition: Can the student translate the task into steps and identify relevant inputs and outputs?
  • Conceptual explanation: Can the student explain why the algorithm and key constructs work?
  • Testing and debugging: Can the student design a useful test, interpret an error, and justify a repair?
  • Adaptation: Can the student modify the program when a requirement changes?
  • Responsible AI use: Can the student disclose assistance, check generated suggestions, and identify code they do not understand?

This is a suggested classroom rubric, not a validated scale from the cited studies. Teachers should adapt it to the course outcomes and test whether different assessors can apply it consistently. A code detector should not replace direct evidence of competence or a fair opportunity for students to explain their work.

What institutions should monitor before scaling AI in programming courses

Start with a limited pilot rather than a campus-wide mandate. Record the model and version used, permitted functions, prompt guidance, data handling, and any changes made during the course. Keep student code and conversation logs only when necessary, explain the purpose, restrict access, and follow institutional privacy requirements.

Measure more than tool activity. Useful indicators include independent test performance, the ability to explain and modify code, quality of debugging, completion of formative practice, student help-seeking, teacher workload, and differences in access or outcomes across learner groups. Compare the AI-supported sequence with a credible alternative where possible. If the design is not randomised or the sample is small, report associations and limitations rather than causal conclusions.

Teachers need professional development to inspect AI outputs, design tasks that preserve student reasoning, recognise common failure modes, and make transparent decisions about acceptable assistance. Students also need to evaluate AI outputs critically; see Prowad’s guide to what AI literacy means for a broader framework. For broader institutional evaluation, see Prowad’s evidence checks for AI learning tools. For a related example of chatbot design that elicits reasoning rather than simply supplying answers, read AI chatbots for higher-order thinking in STEM.

Conclusion: teach students to work with AI—and without it

The 46-study review shows that generative AI has become a significant research topic in programming education, but the evidence base remains uneven and often lacks an explicit pedagogical framework. The 357-learner field evaluation suggests that teacher-governed scaffolding can strengthen formative practice and perceived support without necessarily improving the primary final assessment. The recent LAMPP perspective offers a coherent way to think about teaching, lesson design, engagement, assessment, and ethics, but its proposals still need testing.

For instructors, the practical direction is clear: make an independent attempt visible, use AI for bounded explanations and debugging support, require verification, gradually reduce assistance, and assess transfer without the tool. The measure of success is not how much code an assistant can generate. It is whether students become more capable of understanding, testing, adapting, and defending the programs they create.

References

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *