Pablo Zavala · AI Safety Evaluation · Research Engineering
A Governance Lab Should Run the Same Way Twice
Teaching AI governance through Colab notebooks where Run all works deterministically on free hardware, with fixed seeds, dated snapshots, sample caps, and provider-agnostic fallbacks that let students audit every claim, and a structured mini-brief that turns each lab into an argument from question to method to limitation to recommendation.
A governance lab should run the same way twice.
Generative AI courses often teach governance through slides and readings, yet a slide leaves a claim hard to check. For CMU Heinz 94-816, Generative AI: Applications, Implications, and Governance, I built a Colab notebook pack around a stricter idea: teach governance, engineering, and prompting through notebooks where "Run all" finishes the same way twice on free hardware, so a student can inspect every step that led to a conclusion.
A Course Owns Its Assessment
Attribution comes first. 94-816 is a CMU Heinz course, and the teaching team owns its syllabus, rubrics, and grading. I contributed the notebook pack as a teaching assistant, so this essay describes my design and pedagogy while the assessment materials, grading keys, and exam content remain the course's. The design deserves description precisely because the method travels even when the graded specifics stay behind the course.
Three Labs Share One Method
The pack ships three independent notebooks: one for AI governance, one for AI engineering built around retrieval-augmented generation, and one for prompt engineering. Each runs standalone, each opens with a short study-design frame (a question, a hypothesis, and stated interpretation limits), and each closes with the same structured brief. The governance notebook probes the policy landscape, risk classification, and a fairness check; the engineering notebook walks ingestion, chunking, retrieval, and a local answer step so a student can inspect where a pipeline succeeds or misses; the prompting notebook contrasts weak and improved prompts and runs a controlled red-team-and-defense exercise. A shared method across three domains lets a student carry one discipline (frame, run, inspect, brief) from governance into engineering into prompting.
Determinism Makes Governance Teachable
A governance discussion collapses when two students see two different outputs and argue about the tool rather than the question. The pack fixes that by exposing four reproducibility controls in every notebook: a fixed SEED, a MAX_SAMPLES cap, a MAX_TOKENS budget, and an AS_OF_DATE. Because a seed pins randomness and a sample cap bounds the workload, the controls are designed so one student's run should match another's, and a teaching assistant can grade against a stable result. Determinism turns a lab into shared evidence: everyone argues from the same numbers, so the conversation moves toward governance judgment rather than runtime luck.
Free Hardware Keeps Every Student In
Cost quietly decides who participates. A pack that demanded a paid GPU or a personal API key would exclude the students least able to pay, so the design defaults to free Colab CPU and completes without any API key. Optional provider sections support OpenAI, Anthropic, and Google keys for comparison, yet blank keys skip those cells gracefully and the notebook still finishes. Provider-agnostic fallbacks matter for governance specifically: a lesson about accountable AI should avoid making a single vendor a prerequisite, and a student should reach the learning outcome whether a provider responds or stays silent.
Currency Needs a Dated Snapshot
Governance facts move, so a notebook that hard-codes a legal claim ages badly. The pack answers with a live-check-plus-fallback pattern: each notebook tries a live source, and a failed check falls back to a dated snapshot rather than breaking the run. An AS_OF_DATE field stamps that snapshot, so a reader knows exactly when a claim was current. The build snapshot from 2026-02-23 tracks moving targets like the EU AI Act's phased obligations and Colorado's SB24-205 as amended by SB25B-004, and the dated frame teaches an honest habit: cite the law as of a date, then check the source again.
A Mini-Brief Turns a Lab Into an Argument
A run produces output; an argument produces accountability. Each notebook closes with a structured mini-brief that asks the student to record a research question, a method, findings, limitations, a recommendation, and sources. Those six fields convert a lab from a pile of cell outputs into an auditable line of reasoning: the question frames the inquiry, the method shows how the student got there, the limitation states what the evidence fails to support, and the recommendation commits to a judgment a reviewer can contest. An optional JSON export packages the brief as a submission artifact, so the reasoning travels in a form another reader can inspect.
Governance Literacy Grows From Verifiable Practice
Lecture alone teaches governance as vocabulary; a rerunnable lab teaches governance as practice. When a student changes a prompt, watches a retrieval miss, sees an injection attempt blocked, and then writes down the limitation, the student learns to tie a claim to an observed output rather than to a confident summary. That habit, evidence first, boundary stated, judgment owned, runs straight through my portfolio. My LectureForge course-book work applies the same discipline to teaching material: show the sources, record what the workflow checked, and keep human authority over release. A governance curriculum built the same way produces graduates who ask for the seed, the date, and the limitation before they trust an output, which is the whole point of teaching governance at all.
Boundaries
- This essay describes the pack's design and pedagogy rather than publishing its assessment materials; the rubrics, grading keys, and exam content stay with the course.
- 94-816 is a CMU Heinz course; I contributed the notebook pack as a teaching assistant, and course ownership remains with the Heinz teaching team.
- Claims about reproducibility describe the pack's design intent: a fixed seed, a dated snapshot, and sample caps aim at deterministic reruns rather than certifying bit-for-bit identical output across every environment.
- Governance and legal references reflect the build snapshot's stated currency date rather than independent legal verification.
Sources
- CMU Heinz College public course listing for 94-816, Generative AI: Applications, Implications, and Governance (public course description).
- My teaching-assistant role on 94-816 and authorship of the reproducibility-first Colab notebook pack.
- My essay LectureForge course-book workflow.
Non-public course files (notebooks, instructor guide, and rubrics) informed this description and stay uncited and unquoted.