About 15% used UMD’s AI study assistant. Classes offered access logged less Canvas activity

The trial assigned access, not individual use

The randomized trial covered 2,379 undergraduates, 30 instructors and 34 course sections during fall 2025. The courses ranged from biology and computer science to communication, English, family science, public policy and mathematics.

Randomization happened at the instructor level within 13 research blocks. Nine blocks matched different sections of the same course. Some instructors received UMD’s GPT-4o-based assistant inside the university’s Canvas system, where it could draw on course materials selected by the instructor. The others continued without it.

Students were free to ignore the assistant. About 15% of those offered access opened it at least once, and users averaged roughly four sessions. The experiment therefore estimates what happened when a class received the option. It does not isolate the effect of an individual conversation or compare users with otherwise identical nonusers.

Canvas recorded the clearest change

In the matched same-course sections, access reduced recorded page views by 0.37 standard deviations, active days by 0.51 and participation by 0.90. Participation included discussion responses, assignments and quizzes.

Submission stayed steady. The researchers found no detectable change in the share of homework turned in or submitted on time. The larger difference appeared in other activity that Canvas could record.

The grade result needs the same sample split. Across the full mix of courses, the final-grade estimate did not meet the paper’s conventional 5% significance threshold. Among 1,353 students in matched sections of the same course, the adjusted model estimated final grades 4.12 percentage points lower in classes offered access, equal to 0.37 standard deviations. On the course GPA scale, the estimate was 0.11 points lower.

The two results occurred together, but the paper does not show that lower Canvas activity caused lower grades. Platform logs also miss reading, handwritten practice, office hours, study groups and work done in another app.

Nearly three-quarters of requests sought information or solutions

Students who opened the assistant sent 3,460 requests. Researchers classified 73.8% as requests for facts, explanations, solutions, generated text or code. Practice and test preparation made up 11.3%, course navigation and logistics 7.6%, and other or unclear interactions 6.6%. Just 0.7% asked for feedback on work a student had already attempted.

Most instructors kept the assistant in its default direct-instruction mode instead of switching to a guided tutor mode. Among treatment-group instructors who answered an implementation survey, none said they had built the assistant into an assignment.

This was not a test of a tightly scripted tutor that required students to show their reasoning before receiving help. It tested optional access to a mostly direct-answer tool placed beside the rest of a course.

The next version would be a different experiment

The study is an EdWorkingPaper and has not been peer-reviewed. Its academic outcome was the grade assigned inside each course, not a common independent test. Sections of the same course could still differ in grading practice or in how much work Canvas made visible.

Students in both groups could use other university-supported AI tools, including ChatGPT and Gemini, and the researchers could not observe all of that use. Offering UMD’s assistant may also have changed what students thought was permitted even if they never opened it. Three instructors who did not activate the tool were removed after random assignment. That post-randomization exclusion means the reported intention-to-treat estimates do not include every instructor originally assigned.

UMD now advises instructors to define a specific learning purpose, require an attempt before assistance, use tutor mode and watch course engagement. Those are recommendations, not results from this trial. The trial did not test whether they improve grades or prevent the same drop in recorded activity.

A more deliberately designed assistant may produce a different result. Whether it can help without coinciding with the same decline in recorded course activity is the next question, not an answer supplied by this study.