Structural Isomorphism Between LLM Embedding Spaces and Quantum Mechanical Systems

Introduction

Large language models (LLMs) encode semantic information as high-dimensional embedding vectors, with cosine similarity serving as the standard measure of semantic relatedness. Despite their empirical power, LLM embedding spaces currently lack a rigorous theoretical framework comparable to, for instance, statistical mechanics for thermodynamics. Embeddings are often treated as points in �N with no deeper mathematical structure beyond the dot product. This paper addresses this gap by establishing a precise structural isomorphism between LLM embedding spaces and quantum mechanical systems.

The primary goals of this work are multifaceted. We first provide a complete and rigorous derivation of both the LLM embedding classical system and its quantum mechanical counterpart, detailing aspects previously presented in abbreviated form. Crucially, we offer a clear interpretation of the quantum mechanical time parameter and complex phases within this context, resolving ambiguities from prior work. Central to our contribution, we prove the exact structural isomorphism between these classical and quantum formulations, demonstrating their mathematical equivalence in describing semantic similarity. This isomorphism not only unifies these descriptions but also reveals new mathematical structures, such as a local U(1) gauge symmetry, genuinely absent from classical accounts, and clarifies the profound implications of this correspondence. We also present a concrete quantum circuit implementation that leverages this isomorphism, providing an experimental verification of the theoretical framework by showing that classical cosine similarity can be precisely recovered as a quantum measurement probability. This isomorphism is structural, not physical: it highlights that the underlying mathematical principles governing LLM embeddings—vector spaces, inner products, projections, and squared norms—are precisely those of quantum mechanics, and that this correspondence is exact.

The paper is organized as follows. Section II reviews background and related work. Section III derives the LLM embedding classical system from the geometry of the unit sphere. Section IV constructs the quantum system and proves its equivalence to the classical system through the Schrödinger equation. Section V establishes the commutative diagram and characterizes the structural isomorphism. Section VI resolves the interpretation of time as a gauge parameter and explains the role of the complex state vector. Section VII presents the quantum circuit implementation and its derivation from the Hamiltonian structure. Section VIII reports the experimental validation through quantum circuit simulations. Section IX discusses the implications of the framework, the additional quantum structure it provides, and directions for future work.

Background and Related Work

The mathematical framework developed in this paper draws on three distinct lines of research: the geometry of embedding spaces in natural language processing, the application of quantum formalism to meaning and cognition, and the practical intersection of quantum computing with machine learning.

The idea that meaning can be represented geometrically dates to the distributional hypothesis—the observation that words appearing in similar contexts tend to have similar meanings [1,2]. This hypothesis underlies the construction of word embedding spaces, where semantic relationships are encoded as geometric relationships between vectors. The development of contextual embeddings through transformer architectures and sentence-level representations extended this geometric picture to richer linguistic units, with cosine similarity between embedding vectors serving as the standard measure of semantic relatedness [3-6]. The present work takes this geometric structure as its starting point and shows that it admits a reformulation in the language of Hamiltonian mechanics and, subsequently, quantum mechanics.

The application of quantum-mechanical formalism to language and cognition has a separate history. Widdows [7] explored connections between vector space models of meaning and quantum logic, exploiting the shared mathematical structure of subspaces, projections, and orthogonality. Coecke, Sadrzadeh, and Clark [8] developed a compositional distributional model of meaning grounded in category theory, using the tensor product structure of quantum mechanics to compose word meanings into sentence meanings. In the cognitive sciences, quantum probability theory has been applied to model phenomena in human judgment and decision-making that violate classical probability axioms, such as order effects in sequential judgments and the conjunction fallacy. These approaches share with the present work the recognition that quantum-mechanical mathematics can describe systems that are not themselves quantum physical, but they differ in scope and method: they typically postulate quantum-like structures to model specific linguistic or cognitive phenomena, whereas the present work derives the quantum structure from the existing geometry of LLM embeddings without additional postulates.

On the computational side, quantum computing offers a distinct set of primitives—superposition, entanglement, and interference—that have been applied to machine learning tasks including classification, clustering, and kernel methods [9,10]. Quantum algorithms for search and optimization demonstrate that quantum resources can provide advantages for specific computational problems [11,12]. The quantum circuit constructed in this paper belongs to this computational tradition: it encodes classical embedding data into quantum states and extracts cosine similarity as a measurement probability. However, the circuit is derived from the structural isomorphism rather than designed as a standalone algorithm, and its primary purpose is to provide an experimental verification of the theoretical framework rather than a computational speedup.

The present work builds directly on a series of earlier papers by the author [13–19]. In Ref. [13], the concept of a semantic wave function was introduced, associating quantum state vectors with LLM embeddings. Ref. [14] developed the quantum LLM framework, modeling semantic spaces with quantum principles. Ref. [15] demonstrated the first quantum circuit implementation on IBM quantum hardware, computing cosine similarity as a measurement probability on a 128-qubit processor. Ref. [16] introduced the Hamiltonian formulation and the discrete semantic state structure that forms the classical foundation of the present work. Refs. [17] and [18] extended the framework to partition function similarity and hierarchical models of linear and nonlinear dynamics in embedding spaces, and a local U(1) gauge symmetry. Ref [19] shows how to implement quantum algorithms for Large Language Models on noisy intermediate Scale quantum computers. The present paper consolidates and extends these results by providing complete derivations of both the classical and quantum systems, proving their structural isomorphism through a commutative diagram, resolving the interpretation of time and complex phases, and presenting a systematic experimental validation, which were previously given in abbreviated form.

The LLM Embedding Classical System

In this section we derive the classical framework that forms the foundation of the quantum mapping. Starting from the standard definition of cosine similarity between two L2-normalized embedding vectors, we arrive at a Hamiltonian formulation with a universal eigenstructure through a direct geometric construction. The key result is that the transformed cosine similarity S′C equals the square

Cosine Similarity and the Embedding Vector Parameterization

The Fundamental Scale and L2 Normalization

Geometric Derivation of S'C

The Rank-1 Hamiltonian and Its Eigenstructure

Derivation of S′C with

Diagonalization and the Unitary Transformation

The Degenerate Case a = −b

The degenerate case therefore does not affect the validity of the framework; it simply requires a different choice of vˆ, and the result S′C = 0 is recovered correctly for any such choice.

The LLM Embedding Quantum System

We now show that the classical system admits an exact quantum mechanical representation. The transition from the classical to the quantum formulation requires two conceptual steps: the promotion of real vector components to complex, time-dependent amplitudes, and the promotion of the Hamiltonian matrix to an operator acting on a Hilbert space. We show that these steps preserve all physically observable quantities—in particular, the transformed cosine similarity S′C—and that the quantum formulation is not an approximation but a structurally exact equivalent of the classical system.

The Classical-to-Quantum Mapping and State Construction

There are two fundamental characteristics that distinguish a quantum mechanical system from a classical one: the state vector is complex-valued, which introduces the concept of phase, and the state vector is time-dependent, with its evolution governed by the Schrödinger equation. We now construct the quantum analogue of the classical embedding system by introducing both of these features through a canonical mapping.

The Canonical Mapping

Construction of the Quantum State in the Eigenbasis

The Role of the Complex Phase and Normalization

The Time-Dependent Schrödinger Equation and Its Solutions

We now demonstrate that the time evolution defined by the coefficients ψ˜n(t) in Eq. (26) satisfies the time-dependent Schrödinger equation in both the eigenbasis and the computational basis. This establishes that the quantum embedding system obeys the fundamental dynamical law of quantum mechanics.

Verification of the Schrödinger Equation in the Eigenbasis

Consistency with the Schrödinger Equation in the Computational Basis

Matrix Representations

Equivalence with the Classical System and Observables

Both expectation values are identical to the classical transformed cosine similarity at all times. This confirms that the Schrödinger equation, despite introducing complex phases and time dependence, preserves the physical content of the classical system.

Geometric Interpretation

The Commutative Diagram and Structural Isomorphism

in which the first two equalities are classical and the last two are quantum mechanical.

Nature of the Isomorphism

where the first term is a classical squared projection, the second is a quantum squared amplitude, and the third is a measurement probability on a quantum computer. The left-hand side is computed classically from LLM embeddings; the right-hand side is measured on quantum hardware. Their equality is confirmed experimentally in Section VIII.

Additional Structure in the Quantum Formulation

The classical system is embedded within the quantum system as the special case where all phases are zero and measurements are performed in the eigenbasis. The quantum formulation is therefore strictly richer than the classical one, while remaining fully consistent with it at the level of all diagonal observables.

Interpretation of Time and the Complex State Vector

The quantum formulation introduces two features absent from the classical description: a complex state vector with phase information, and a time parameter t governing its evolution. Since the embedding vectors a and b are static objects produced by the LLM encoder, the natural question is: what does t mean in a context where there is no obvious physical time?

The Time Evolution Operator and Its Role

The Gauge Parameter Interpretation

The Hadamard Gate as Schrödinger Time Evolution

To make the gauge parameter interpretation concrete, we work through a complete example using the Hadamard gate which is a well-known gate in quantum computing. We find a Hamiltonian and a time t such that e^{−iHt/â?ÂÃÃÂ??ÂÃÂ?} exactly reproduces the Hadamard gate, demonstrating explicitly that the time parameter is absorbed into the gate definition and does not appear as an independent observable. This example also illustrates the rank-1 Hamiltonian structure shared with the LLM embedding system.

The Hadamard Gate and its Eigenstructure

Constructing the Hamiltonian

Verification

Connection to the LLM Embedding System

The Hadamard example illustrates the general principle governing the LLM embedding quantum system. In both cases:

• A unitary transformation is generated by a rank-1 Hamiltonian acting for some time t.

• The time parameter is a gauge variable: only the product t/h is physically meaningful, and this product is fixed by the unitary.

• The Hamiltonian has a single nonzero eigenvalue, so time evolution consists of a single phase rotation in a one-dimensional subspace.

• The observable output is a real-valued probability obtained by squaring the relevant amplitude, in which the phase cancels.

The difference is one of context, not mathematical structure. In a quantum circuit, the Hamiltonian and time are determined by the physical implementation of the gate. In the LLM embedding system, the Hamiltonian is determined by the embedding vectors through Hˆ ′ = vˆvˆT , and t is the gauge freedom that does not correspond to any physical duration. The absence of an explicit time variable does not invalidate the quantum description; quantum circuits demonstrate that fully quantum mechanical systems can be described, implemented, and measured without ever referencing time directly.

The Complex State Vector as a Theoretical Construct

In the LLM embedding quantum system, the complex state vector (whether |Ψ(t)⟩ in the computational basis or |ψ˜(t)⟩ in the eigenbasis) serves three essential purposes. First, it ensures unitary evolution: the Schrödinger equation requires complex amplitudes to generate the phase factors e^−iEnt/h that make the time evolution operator unitary, since real-valued evolution equations do not in general preserve the norm under non-trivial Hamiltonians. Second, it encodes phase relationships: the relative phases between components of ψ˜(t)⟩ carry information about the Hamiltonian eigenstructure and become observable when the system is measured in a rotated basis, as occurs in quantum circuit implementations and in the gauge field extension. Third, it enables quantum circuit realization: quantum computers operate natively on complex state vectors, and the state preparation, unitary transformation, and measurement steps all rely on the complex Hilbert space structure. The complex state vector is therefore not an arbitrary mathematical decoration but the minimal structure required to simultaneously satisfy the Schrödinger equation, preserve unitarity, and enable quantum circuit implementation.

Interpretation of h as Semantic Resolution

h as the Semantic Resolution Scale

Connection to the U (1) Gauge Structure

The semantic resolution scale connects naturally to the local U(1) gauge symmetry discussed in Section V and developed in Ref. [18]. Under the local gauge transformation

Structural Interpretation

As with the time parameter t, the interpretation of h as h_sem is structural, not physical. We do not claim that LLM embedding spaces possess a physical analogue of Planck’s constant, nor that the discreteness of the token vocabulary arises from quantum mechanical quantization. The claim is precisely that the mathematical role of h in the formalism—as the scale setting the ratio θs = t/h that governs phase evolution—is identical to the mathematical role of the minimum angular separation h_sem in the geometry of the embedding space. The correspondence is exact at the level of mathematical structure, and it provides a canonical gauge fixing that connects the abstract parameter h to a directly computable property of any LLM embedding model.

Quantum Circuit Implementation

The structural isomorphism established in Sections III–IV shows that the transformed cosine similarity can be expressed as a quantum mechanical measurement probability. This section details a quantum circuit that computes S'C from LLM embedding vectors, realizing the path from B to D in Figure 1. We first initialize the quantum circuit with the values of the classical embedding vector and then use a quantum computer to perform the diagonalization. As a result, we should be able to observe the same classical result S'C .

The circuit operates on n = ⌈log2 N⌉ qubits, where N is the embedding dimension (padded to the nearest power of 2), and consists of three stages: state preparation, unitary transformation, and measurement.

State Preparation

Unitary Transformation

Measurement and Recovery of Cosine Similarity

The Complete Circuit

Experimental Validation

The theoretical framework predicts that the probability of measuring the all-zeros state in the quantum circuit, P(|0...0⟩), equals the classically computed transformed cosine similarity S'C . We validate this using quantum circuit simulations on the Qiskit Aer qasm simulator, which emulates ideal quantum hardware. This approach allows us to focus solely on the theoretical framework’s prediction, as validation on real quantum hardware would introduce a different research question centered on characterizing hardware noise and fidelity.

Embedding Model and Preprocessing

Results and Convergence

The quantum circuit was executed at various shot counts (N_shots). The measured probability P(|0...0⟩) consistently converged towards the theoretical S'C value of 0.8862. For instance, at N_shots = 8192, the measured P(|0...0⟩) was 0.8842, with a deviation of 0.0020 from the theoretical value

Consistency and Validation

Conclusions

This paper has established a structural isomorphism between LLM embedding spaces and quantum mechanical systems, providing a rigorous mathematical framework that connects classical cosine similarity to quantum measurement probability. The central result is that the transformed cosine similarity S'C = (1 + a • b)/2 between two LLM embedding vectors admits four equivalent formulations connected by a commutative diagram (Figure 1)

This equivalence chain demonstrates a formal identity between classical linear algebra, eigenspace decomposition, quantum amplitudes, and Born rule probabilities. The isomorphism is structural, not physical: it asserts that the mathematical structures governing LLM embeddings are formally identical to those of quantum mechanics, preserving algebraic, spectral, and observable properties. It does not imply that LLMs are physical quantum systems, nor does it suggest a computational speedup for pairwise similarity.

The framework rests on several key theoretical results. We provided a complete derivation of the LLM embedding classical system, showing how S′C equals the squared projection onto an angle bisector, leading to a rank-1 Hamiltonian with a universal eigenstructure. The quantum system promotes this classical structure to complex, time-dependent state vectors governed by the Schrödinger equation.

We clarified that the time parameter is a gauge variable, not physical time, and that complex phases, while crucial for unitary evolution, cancel out in diagonal observables. The classical-to-quantum mapping is exact, preserving all observable quantities.

Symmetrically, the parameter h admits interpretation as the semantic resolution scale h_sem = mini_≠j arccos(a_i • a_j), the minimum angular separation between distinct token embeddings in the model’s vocabulary. Only the ratio θ_s = t/h enters observable predictions, so both parameters are gauge variables and the physically meaningful object is the semantic phase angle θ_s alone. This interpretation connects the abstract quantum parameter h to a directly computable property of any LLM embedding model, and yields the testable prediction that h_sem decreases monotonically with embedding dimension N.

The utility of this quantum mechanical language, despite its structural nature, is profound. It reframes cosine similarity as a measurement probability, connecting it to quantum measurement theory. It introduces complex phases, allowing for the definition of a semantic noise profile and, most significantly, reveals a local U(1) gauge symmetry [14]. This symmetry provides a mathematically precise model for contextual influence on embeddings, analogous to electromagnetism, with a conserved semantic charge [18]. Furthermore, the isomorphism enables direct implementation on quantum hardware, compressing N-dimensional embeddings onto ⌈log2 N⌉ qubits, opening avenues for quantum algorithms operating natively on semantic representations.

The empirical content of this isomorphism is validated by quantum circuit simulations. The circuit, consisting of state preparation, unitary transformation, and measurement, produces P(|0...0⟩) which converges to the classically computed S′C with deviations consistent with statistical shot noise. This confirms the physical realizability of the framework and provides a falsifiable prediction for any pair of LLM embedding vectors. The U(1) gauge structure offers further testable predictions regarding contextual perturbations.

In conclusion, this work provides a robust mathematical foundation for understanding LLM embedding spaces through the lens of quantum mechanics. It offers new conceptual tools and a richer framework for analyzing semantic representations, modeling contextual influences, and exploring quantum computing applications. The path forward involves leveraging these insights to develop new theoretical models and practical quantum algorithms for natural language processing.

References