Papers

Large language models acquire their values from data. Post-hoc alignment methods such as RLHF modify only the first few output tokens, creating a shallow safety veneer that fine-tuning on as few as ten adversarial examples can remove. This paper argues that alignment must be reconceived as a data-centric endeavour, one that begins with the pretraining corpus itself. Drawing on a systematic review of 248 papers, we formalise the concept of corpus-level alignment and provide a comprehensive taxonomy of data-centric alignment techniques spanning pretraining data filtering, intrinsic alignment signal extraction (ISAAC), constitutional and self-alignment, instruction-tuning dataset construction, synthetic data generation, and preference optimisation. We synthesise evidence showing that corpus composition is the primary determinant of model alignment: pretraining data interventions reduce misalignment rates from 45% to 9% (Tice et al., 2026), safety pretraining frameworks lower attack success rates from 38.8% to 8.4%, and corpus-based methods achieve safety improvements comparable to supervised approaches without human labels. We identify six open research gaps (including scaling laws for alignment data, multilingual corpus-level alignment, and principled corpus composition) and argue that the path forward requires not only technical advances but also normative frameworks for deciding what values a training corpus should encode.

Post-hoc alignment methods (reinforcement learning from human feedback (RLHF), direct preference optimisation (DPO), and their variants) have become the standard approach to making large language models safe, and their practical advantages in cost, speed, and iterability are substantial. Yet a growing body of evidence shows that these methods modify only a thin, geometrically fragile shell of model behaviour: safety concentrates in a low-rank subspace that fine-tuning on as few as ten examples can disrupt. This paper synthesises evidence from over 370 recent studies across six research frontiers to argue that alignment should begin with the pretraining corpus, not end with post-hoc correction. We synthesise the geometric foundations of alignment fragility, characterise scaling laws for alignment data, demonstrate a depth-robustness gradient where pretraining-level interventions survive orders of magnitude more adversarial pressure than post-training defenses, map the multilingual safety gap, survey corpus auditing infrastructure, and review principled approaches to data composition. For each frontier, we identify concrete missing experiments (with hypotheses, methods, falsification criteria, and compute estimates) that would advance the field toward a principled, data-centric alignment paradigm. We present this work as an agenda paper with theoretical grounding: while the evidence strongly favours corpus-level alignment, its central geometric predictions remain to be tested directly, and we identify the specific experiments needed to confirm or refute them.