Transformer Architectures as Quantum Event Horizons: Information Scrambling, Page Curves, and Island Formation in Deep Neural Networks

Abstract

Chur Chin

This study reinterprets the information flow within Transformer neural network architectures through the lens of quantum black hole thermodynamics, specifically the Event Horizon model and the Page Curve framework. We propose that each Transformer layer constitutes a dynamical system whose stability is governed by a Lyapunov exponent λ, analogous to Hawking radiation scrambling near a black hole event horizon. Numerical experiments using a BERT-base model demonstrate that early layers (Steps 0-6) exhibit λ > 0 (Lipschitz constant L ≈ 3.05), corresponding to a Fast Scrambling phase in which information is mixed violently across feature dimensions. At Step 7, a critical phase transition is observed: λ inverts to -0.1017, signaling the spontaneous formation of an Information Island: a stable, contracting attractor region analogous to the quantum gravity Island that resolves the black hole information paradox. Beyond Step 7, λ deepens monotonically to -0.4704 at Step 20, while the von Neumann entropy of the radiation subsystem stabilizes at S = 0.6858, consistent with a unitary Page Curve. We further derive an analytic expression for the Page Time as t_page = τ ln[(λ_chaos + κ)/κ] ≈ 5.86 layer-steps, and demonstrate that the Island boundary is determined by extremization of a generalized entropy functional incorporating the Jacobian-derived Area term, log det(JTJ). These findings suggest that well-trained Transformer models are not merely function approximators but implement a geometric information compression mechanism mathematically equivalent to black hole evaporation dynamics.

PDF