Schedule

The first part is a refresher on deep learning: training, optimisation and generalisation, and what each of them is doing. The second part introduces the transformer: its architecture and the mathematics under it.

Two combined challenges: choosing an alignment target, our vision for how systems should behave, and solving the technical problem of aligning systems with it. Frameworks for decomposing the technical problem (training stories, outer and inner alignment, inductive biases) and the critiques of them; how goal-directedness raises the stakes; and a survey of how hard and how severe people take the problem to be, alongside the spread of solution approaches the field pursues.

Distributed representations and codes, with superposition as the case that matters. The attempts at a mathematical theory of superposition; the theory of when sparsity recovers the feature directions that matter; bilinear MLPs, and whether components can be replaced with more linear ones; and architectures designed to be interpretable rather than interpreted after the fact.

Pretraining, post-training, deployment: what affordances each phase gives us for alignment, illustrated with the state-of-the-art methods at the major labs, plus the empirical results worth carrying around about how LLMs actually behave. The deployment section takes a higher-level view and covers the non-technical parts as well: responsible scaling policies, safety cases, the economic impact of AI, and governance.

Reinforcement learning motivated from preferences: what the von Neumann-Morgenstern axioms assume, how preferences become a utility function, and how that becomes the reward, return and policy vocabulary of a Markov decision process. Then goal misgeneralization: a policy that keeps its capabilities off the training distribution while pursuing a goal other than the intended one. Afternoon: AIXI, the Bayes-optimal agent built on Solomonoff induction (slides and module notes D.3.1 and D.3.2).

Steganography is the hiding of messages inside innocent-looking outputs. The day covers how it works, what current models can already do, why it is hard to detect or prevent, and which defences, such as paraphrasing, destroy hidden channels. The second half turns to backdoors in language models: how a hidden trigger is planted, and how hard it is to find one.

The results that make deep learning strange. For each one, the day states the phenomenon, describes the experiment that shows it, and weighs the candidate explanations against it. By the end you should be able to say what a theory of deep learning has to explain.

Field-theoretic treatment of wide networks: perturbative expansion in the inverse width, the infinite-width Gaussian limit, the partition function correspondence, renormalisation and relevant operators, and why the depth-to-width ratio matters.

Within an architecture, certain weight vectors correspond to structurally simpler networks. These degeneracies complicate the map from parameter space to function space, and make learning in neural networks substantially richer than learning in classical statistical models. Qualitative definitions of degeneracy through the parameter–function map, the Fisher information matrix and the curvature of the loss landscape; then the central quantitative definition, the local learning coefficient, from volume scaling asymptotics; then Watanabe’s free energy formula for Bayesian inference as a case study.

Morning, 10:00: Introduction to singular learning theory, Shaowei Lin (Beneficial AI Foundation). Beginning with Bayesian statistical learning, the lecture introduces the basic concepts of singular learning theory and outlines how the theory could explain some mysterious aspects of deep learning.

Afternoon, 13:30: lecture and exercises building on the morning, Dmitry Vaintrob.

Implicit regularisation: how the training process itself biases toward simple solutions, via loss-landscape geometry, the edge of stability, simplicity bias, the neural tangent kernel, and the lazy versus rich regimes in deep linear networks as a toy model of deep learning. Emergence: grokking, induction heads and silent alignment, read through phase transitions (grokking as a transition from the lazy to the rich regime) and what that perspective buys us for detecting emergent capabilities early, which is the safety payoff.

Afternoon, 13:30: The mechanics of learning: symmetry, features, and implicit bias, Daniel Kunin (UC Berkeley). Building on the morning's introduction, the lecture explores how symmetry, initialisation and gradient noise shape what neural networks learn and when they learn it. Through tractable models spanning linear networks and sequential group composition, it connects conservation laws, stagewise feature learning and stochastic collapse to the emergence of structured representations and simpler subnetworks.

Starting from hidden Markov models, the module motivates generalised HMMs through minimality and uniqueness, then develops two views of Bayesian inference over emissions: belief states geometrically, and the mixed state presentation algorithmically.

World models are what let an advanced agent plan and weigh counterfactuals without acting. Three units: a general introduction to world models and how they are used in modern AI; how they can be formalised in a reinforcement learning setting; and how agents use them to build abstractions.

Pick a research area and write a proposal in it. The areas on the table by now: AIXI, steganography and backdoors, physics for deep learning, singular learning theory, training dynamics, computational mechanics and world models.

The focus moves from weight space to training data, framed as the counterfactual impact of reweighting individual data points. Three frameworks each read the data-to-model map differently: influence functions, as an implicit function of data weights at a unique minimum; Bayesian influence functions, as a posterior distribution over parameters; and unrolling, as a concrete optimisation trajectory. They turn out to be closely connected (influence functions emerge as a limiting case of both alternatives) and the degeneracy phenomena from the SLT day reappear exactly where the classical theory breaks down.

Most of the month studies one AI system and one principal. This day asks what happens when many systems, built by different developers and acting for different people, interact through markets, negotiations and shared infrastructure. It follows the taxonomy of Multi-Agent Risks from Advanced AI: miscoordination, conflict and collusion, and the risk factors behind them, such as information asymmetries, selection pressures and commitment problems. Then multipolar failure, where every agent does roughly what its owner wants and the outcome is still one nobody chose. The day closes with cooperative AI, multi-agent evaluations and infrastructure for agents, and a design challenge: a pricing game in which two agents try to collude past a monitoring regulator.

Morning, 11:30: stochastic dynamics of multi-agent reinforcement learning, James Sykes (Warwick), remote.

Afternoon, 13:30: when can we trust untrusted monitoring?, Morgan Sinclaire (FAR). A safety case across collusion strategies.

In purely syntactic domains such as formal mathematics, correctness can in principle be verified by checking a proof step by step, but that becomes infeasible for exponentially long computations, and does not transfer cleanly to semantic, real-world tasks that rest on costly human judgement. Debate has two systems argue against each other, reducing hard global verification to sequences of small local checks: logarithmic-depth verification in idealised settings, and constant-size local consistency checks under cross-examination. Proof checkers, the complexity-theoretic power of debate and cross-examination, the practical limits of finite debaters such as obfuscated arguments, the prover–estimator approach, and the UK AISI debate-based safety case with its open problems.

A central difficulty in alignment is that we have to reason about the behaviour of systems that do not exist yet, and cannot learn from our mistakes with them. This module develops formal tools for doing so across several agendas: coherence arguments and the complete class theorem, Löb's theorem and the Löbian obstacle to safe self-modification, tiling agents and Vingean reflection, logical induction and reasoning under logical uncertainty, functional and updateless decision theory, and the thermodynamics of optimisation.

Afternoon, 13:30: an introduction to condensation, Sam Eisenstat (Resolution).

Morning, 10:00: Mechanistic estimation and its applications to AI alignment, Victor Lecomte (ARC). The lecture presents ARC's agenda of mechanistic estimation, how it would fit into a full solution for AI alignment, and the main technical and conceptual challenges still open. It assumes no previous exposure to ARC's ideas.

Afternoon, 13:30: practical session building on the morning, Aden Power.

Sparse autoencoders are the most-used tool for turning activations into something a person can read, and also the most argued about. This day covers how they are trained, what a feature recovered by one is and is not evidence of, and the geometric picture underneath: superposition, the linear representation hypothesis, and where each of them is known to break.

Morning, 10:00: ambitious mechanistic interpretability, Andrew Mack (Principles of Intelligence).

Afternoon, 14:00: sparse autoencoders and activation geometry, Eric Michaud.

The second proposal day. The areas added since the first: data attribution, multi-agent dynamics, debate, agent foundations, heuristic arguments and verification and SAEs and activation geometry.