Does AI Improve the "Cost-Effectiveness of Studying"? "Achieving the Same Scores with Less Effort" — Insights from AI-Enabled Classes on Learning Efficiency

Does AI Improve the "Cost-Effectiveness of Studying"? "Achieving the Same Scores with Less Effort" — Insights from AI-Enabled Classes on Learning Efficiency

2025年10月30日 01:21

The University of Massachusetts Amherst (UMass Amherst) has published a longitudinal semester experiment comparing a "class permitting AI use" and a "class prohibiting AI use" under the same syllabus and exams. The conclusion is provocative—there was no significant difference in exam scores and final grades. However, simultaneously, class participation, self-efficacy, satisfaction, and learning efficiency consistently improved in the AI-permitted class. The study is available as a preprint on SSRN and has been reported by the university's release and Phys.org. SSRNumass.edu


Key Points of the Study: What and How Was Compared

  • The subjects were two consecutive sessions (including the afternoon) of an advanced antitrust economics class. The lectures, assignments, and grading were identical, with one group (n=29) being structurally permitted to use generative AI (with guidelines and disclosure requirements specified), while the other group (n=28) was prohibited from using AI. Exams were conducted with paper and pencil, and notes or AI were not allowed. As a conservative design, the AI-permitted group was assigned to the afternoon slot, which historically had slightly lower grades. Phys.org

  • Results: There was no detectable difference in proctored exam scores or final grades. On the other hand, class participation, confidence and efficiency in learning, and satisfaction were higher in the AI-permitted class. Additionally, an increase in AI-related career aspirations and growth in metacognitive behaviors such as editing AI outputs, error detection, and prioritizing one's own answers were observed. SSRN

  • From the educators' perspective, the message "If permitted, provide 'scaffolding' and disclosure rules" is emphasized.umass.edu

  • Limitations: The researchers themselves caution that the sample size is small and includes self-reported metrics.Phys.org

What Does "Scores Don't Improve, but Satisfaction Does" Mean?

AI changes the "feel of learning" rather than the "amount of learning"—this is how the findings can be interpreted. Students utilized AI in focused sessions of 15 to 30 minutes, and through verification and editing of outputs, they strengthened their preference for their own answers. This is a change in attitude that tends to occur when AI functions as a **copilot** rather than a "solution generator." SSRN


The fact that the evaluation was centered on paper tests is also significant. The design philosophy of returning evaluations to a manual and face-to-face focus while allowing AI use in the learning process is widely supported in engineering communities.Hacker News


Alignment with Other Studies and Recent Cases

  • In the field of grading, there are reports that GPT-4 demonstrated performance equivalent to human graders in ranking descriptive answers. AI's applicability as "evaluation assistance" is expanding.Phys.org

  • On the other hand, the operational risks of AI grading are also real. Approximately 1,400 essays in the Massachusetts state standard test (MCAS) were misgraded due to technical issues with AI grading, leading to a subsequent rescore. Human oversight and mature design are essential. Boston.com

  • Issues of distrust and false detection regarding AI misconduct in universities have become major points of discussion in English newspapers and communities since last year. The uncertainty of detectors, student harm due to misidentification, and the need to review evaluation methods continue to be debated. The Guardian

Reactions on Social Media (Initial Observations)

Due to the recent release, the volume of individual posts is limited, but the following points have resurfaced in related news and existing discussion threads.

  • "AI is allowed for learning, evaluations should be face-to-face"—voices seeing a "realistic solution" in this design. It aligns with the argument that the focus of class design should be on exams and oral examinations. Hacker News

  • "AI grading is convenient but risky"—citing the misgrading incident of MCAS, there is a cautious view that AI implementation should be premised on "human redundancy". Boston.com

  • "Improvement in student experience cannot be overlooked"—local reports in Boston emphasize the point that **"the class experience improves with the same scores". The phrase **"the same for less"** is spreading. CBS News

  • "Concerns about AI misconduct and false detection"—doubts about the reliability of AI detectors, and voices from students forced to "prove their innocence." A demand for a refresh in evaluation design. The Guardian

※The social media observations in this article are summaries of "initial trends" based on public posts and existing large threads related to the theme, and not intended to comprehensively cover individual reply quotes to the study.


Implications for Practice: "If permitted, design it"

UMass researchers recommend the three-point set of "permission + scaffolding + disclosure". They suggest formalizing the role, scope, procedures, and disclosure format of AI in classes, and measuring learning achievement with evaluations less affected by AI, such as paper exams, oral examinations, and demonstrations. The key is the two-layer design of **"learning with AI, proving without AI."** umass.edu


Specific Measures (Checklist)

  1. Specify AI policies in the syllabus (permitted tasks/prohibited tasks/disclosure templates).

  2. Provide prompts and attach rationale (what, why, and how instructions were given to AI).

  3. Submit editing logs/self-assessments (traces of verification, correction, and examination of outputs).

  4. Center evaluations in the classroom (paper exams, oral questioning, whiteboard coding, mini-presentations). This is strongly supported by the community. Hacker News

  5. Use "dual assessment" when employing AI for grading (AI + human). Reflecting the lessons from the MCAS incident. Boston.com

In the Context of Japan

The Ministry of Education, Culture, Sports, Science and Technology (MEXT) has been advancing generative AI training for teachers from October 2024 to March 2025, developing insights into class design, material creation, and evaluation support. The suggestion of "if permitted, design it" aligns well with the direction of Japan's training and guideline development. Ministry of Education, Culture, Sports, Science and Technology


What Is "Still" Unknown

  • Generalizability: It remains unverified whether the results obtained under the control of upper-level undergraduate, economics, and the same instructor can be extended to other academic fields, grades, and large classes.

  • Longitudinal Effects Beyond a Semester: Retention of basic knowledge, transfer, collaborative skills, and other long-term outcomes need to be tracked.

  • ##HTML_TAG_502

← Back to Article List