Harvard’s CS 2881R, a graduate-level AI Safety course taught by Boaz Barak, has publicly exposed its Fall 2026 syllabus, offering a rare, granular look at how top-tier institutions are structuring technical AI safety education. The course, which runs Thursdays from 3:45pm to 6:30pm Eastern Time in Allston, Massachusetts, moves beyond abstract ethics to mandate hands-on experiments and deep dives into specific LLM failure modes. With a heavy prerequisite load involving probability, information theory, and Python-based neural network training, the curriculum signals that AI safety is increasingly treated as a rigorous engineering discipline rather than a philosophical sidebar.
Infrastructure and Model Internals
The syllabus prioritizes a deep understanding of the underlying machinery, with lectures dedicated to modern LLM training and inference mechanics. Students are required to read Horace He’s 'Making Deep Learning Go Brrrr' and Austin et al.’s transformer math guides, focusing on KV-cache fragmentation and gradient checkpointing. This technical grounding is paired with practical experiments, such as analyzing how filler tokens affect reasoning and investigating the 'Hawthorne effect' in evaluation awareness. The course explicitly encourages the use of agentic AI for study and assignments, acknowledging that safety researchers must be proficient users of the tools they seek to constrain.
Alignment and Reward Hacking
A significant portion of the curriculum is dedicated to the mechanics of alignment, specifically focusing on reward hacking and grading systems. Guest lectures from industry heavyweights like John Schulman (Anthropic) and Nathan Lambert (AI2) anchor sessions on RLHF and open-source model policies. Required readings include Gao, Schulman & Hilton’s scaling laws for reward model overoptimization and recent Anthropic blogs on grader awareness. Students must engage with specific technical reports, such as the Xiaomi MiMo-V2.6 audit and OpenAI’s Model Spec, to understand how misalignment emerges from the very metrics used to train models.
Industry Integration and Conflicts
The course structure reflects the tight coupling between academic safety research and frontier lab development. Boaz Barak, who is also a member of the technical staff at OpenAI, has included a transparency note regarding this potential conflict of interest. Despite this, the syllabus integrates perspectives from multiple providers, including Anthropic and Thinking Machines Lab, ensuring a multi-vendor view of safety challenges. The inclusion of guest lecturers like Sam Altman and Ajeya Cotra (METR) demonstrates a direct pipeline between academic theory and frontier lab practice, providing students with access to the individuals actively shaping AI governance and capability trajectories.
Key Takeaways
- AI safety education is now deeply technical, requiring proficiency in transformer internals and RLHF mechanics.
- The curriculum mandates reading specific, recent technical reports from OpenAI, Anthropic, and Xiaomi rather than relying on general overviews.
- Hands-on experimentation, such as testing for evaluation awareness and reward hacking, is a core component of grading.
- There is a significant overlap between academic safety researchers and frontier lab staff, with transparency notes addressing potential conflicts of interest.
The Bottom Line
Harvard’s CS 2881R confirms that AI safety has matured from a philosophical debate into a rigorous engineering discipline, where understanding the math behind reward hacking is now as critical as understanding the ethics of AGI.