Why Was Contract Law Tested?
Contract law was chosen precisely because it resists the answer key. The 40 questions used in the study—the kind a student might raise after class or in office hours—demanded synthesis of competing arguments and a defensible conclusion rather than rote recall, testing whether a model could reason where there is no single right answer.
Read more Oz Says Trump Gets Frequent Medical Checkups Because ‘He Likes The Results’
Key Background
The paper was authored by Nyarko with liftlab researcher Alejandro Salinas as first author, alongside colleagues from Yale, New York University, the University of Chicago and other institutions. Participants wrote their own answers before grading anyone else’s, evaluations ran blind through multiple scoring methods and the AI outputs were calibrated to match the length and structure of human responses. The team tested a range of systems, including commercial tutoring tools and Google’s NotebookLM, and found performance varied. Even where the models were hampered by limited context, evaluators frequently still favored them over their human peers. The findings land in the middle of an unresolved debate inside legal education, where some schools are racing to integrate AI while others warn of hallucinations, student overreliance and the slow erosion of the critical-thinking skills a legal education exists to build.
Read more Amazon Prime Day 2026 Moves To June—This Time With Alexa AI Powering The Cart
What To Watch For
The authors are emphatic that quality and deployment are separate questions, and they have only addressed the first. Nyarko said the conversation should now move from whether AI can produce accurate, high-quality legal answers to how it can best benefit students.
Read more Trump Signs Highly Anticipated AI Executive Order: Here’s What It Does