Quantum-Native Transformer Architecture: Interference, Partition Functions, and Nonlinear Schrodinger Dynamics

Introduction

The transformer architecture has revolutionized natural language processing, achieving state-of-the-art results across a wide range of tasks. Its core innovation, the scaled dot-product attention mechanism, computes relationships between tokens using linear projections, a softmax function, and weighted averaging. While transformers have demonstrated remarkable capabilities, they remain fundamentally classical systems processing information as real-valued vectors.

This paper addresses a fundamental question: if the initial and final embedding states of a transformer can be represented as quantum states, what quantum mechanical processes and structures are necessary to maintain a valid Hilbert space throughout the intermediate computations? We hypothesize that classical transformer architectures operate within a limited subspace, or submanifold, of a higher-dimensional quantum-native semantic superspace. The classical transformer is not the full quantum system but its real-valued shadow: the projection of a richer quantum computation onto the subspace accessible to real-valued vectors. This perspective motivates a systematic mapping of each transformer component to its most natural quantum mechanical counterpart.

This paper introduces the Quantum Semantic Circuit (QSC), a quantum-native transformer architecture grounded in this mapping. The correspondences are classified by their logical status. Some are exact mathematical identities: L2-normalized embeddings satisfy the quantum state normalization axiom, softmax attention weights are the Boltzmann distribution by definition, and shifted cosine similarity equals a Born rule probability exactly. Central to the QSC is the use of quantum interference, replacing the classical dot-product with a richer mechanism for computing relationships between token states. One correspondence is derived from axioms: the cubic nonlinear Schrödinger equation is the unique minimal norm-preserving nonlinearity consistent with Hilbert-space structure. Others are derivational approximations: classical sinusoidal position encoding is the first-order real projection of an exact quantum phase rotation, and the classical residual connection is the leading-order Taylor expansion of exact unitary evolution. The remaining correspondences are representational or functional. The central contribution is to establish and classify these correspondences precisely, distinguishing exact identities from approximations and analogies.

This work does not claim that classical transformers are secretly quantum mechanical. The framework identifies which parts of the transformer are already quantum mechanical in structure, which are classical approximations of quantum operations in the semantic superspace, and which require genuinely new quantum primitives. Beyond the direct correspondences, the quantum formalism offers explanations for empirical design choices in the classical architecture that have no theoretical account within the classical framework itself: the dimensional expansion d → 4d → d of the feed-forward network and the asymmetry between its weight matrices can be understood as shadows of quantum symmetry breaking and restoration, with the Higgs mechanism providing a qualitative account of large multi-head models. These are explanations of empirical observations rather than strict predictions, since the classical FFN structure is itself an empirical design choice. The Mexican hat potential is proposed as the natural next approximation beyond the cubic NLSE in a hierarchy of increasingly complex nonlinearities, chosen by the same criterion as the cubic: it is the simplest well-studied potential at the next level of complexity with an extensive existing literature. We validate the framework through a quantum simulator pilot on a lexical disambiguation task, confirming architectural self-consistency and that the NLSE nonlinearity does measurable computational work beyond linear evolution.

Background and Related Work

The transformer architecture, introduced by Vaswani et al. [1], has become a dominant paradigm in natural language processing, achieving state-of-the-art results on various tasks, including machine translation [2], text summarization [3], and language modeling [4]. However, the computational cost of training and deploying large transformer models remains a significant challenge, motivating research into more efficient architectures and training techniques [5]. The attention mechanism, a core innovation, allows the model to selectively focus on different parts of the input sequence [2], computing pairwise relationships between token embeddings through linear projections, softmax normalization, and weighted aggregation. The success of transformers has also led to research into more efficient and scalable training methods, including techniques like LoRA [6] and knowledge distillation [7].

The application of quantum mechanics to machine learning and artificial intelligence is a growing field of research, known as quantum machine learning (QML) [8]. Quantum machine learning algorithms, such as quantum support vector machines [9] and quantum neural networks [10], have the potential to offer significant computational advantages compared to their classical counterparts [11]. However, the development of practical quantum machine learning algorithms is still in its early stages, and significant challenges remain in terms of hardware limitations and algorithm design [12]. Despite these challenges, the potential for exponential gains in computational efficiency for certain tasks motivates continued research into new quantum algorithms and hybrid quantum-classical approaches. QML research explores not only improved computational efficiency but also fundamentally different approaches to learning and representation, potentially leading to new insights in both machine learning and quantum physics.

Quantum computing and quantum circuits provide the fundamental building blocks for implementing quantum algorithms [13]. Quantum computers leverage the principles of superposition and entanglement to perform computations in a fundamentally different way than classical computers. Quantum circuits are composed of quantum gates, which are unitary transformations that operate on qubits, the basic unit of quantum information [14]. While current quantum hardware is still limited in terms of qubit count and coherence time, significant progress is being made in the development of more powerful and reliable quantum computers [15]. These advancements are crucial for realizing the potential of quantum algorithms in various fields, including machine learning and natural language processing. The development of fault-tolerant quantum computers will be essential to overcome current limitations and unlock the full computational power of quantum algorithms.

Several recent works have explored the connections between quantum mechanics and natural language processing, giving rise to the field of quantum natural language processing (QNLP) [16]. For example, some researchers have used quantum-inspired models to represent word embeddings and semantic relationships [17–19]. These models often leverage the principles of superposition and entanglement to create richer representations of semantic relationships, sometimes achieving improved performance on tasks such as semantic similarity and text classification. The work by Laine [20–27] has also explored the use of quantum formalism for understanding LLM representations. These works provide a foundation for the present paper, which aims to establish a more direct correspondence between the transformer architecture and quantum mechanics.

This paper builds upon the existing literature by exploring specific correspondences between transformer components and quantum mechanical operations. It categorizes these correspondences based on how closely the classical and quantum elements align. These correspondences are organized into four types. An exact correspondence holds when the classical and quantum objects are mathematically identical: no approximation, projection, or structural argument is involved. A derivational correspondence holds when the classical operation is the leading-order approximation of the quantum operation, obtained by linearization, real projection, or truncation of an exact quantum expression; the quantum operation is the primary object and the classical operation is its shadow. A representational correspondence holds when a classical object can be mapped to a quantum object preserving the relevant geometry, with the two objects describing the same mathematical content in different languages. A functional correspondence holds when a quantum procedure performs the same architectural role as a classical transformer subroutine, with the internal implementations differing and neither being an approximation of the other. Table 1 summarizes the status of the main components considered here.

Table 1: Complete mapping between transformer components and quantum mechanical operations, with correspondence status. Exact: mathematical identity, no approximation. Exact up to rescaling: identical up to a rank-preserving parameter rescaling. Derivational: the classical operation is the leading-order approximation of the quantum operation, obtained by linearization, real projection, or truncation. Representational: classical and quantum objects describe the same geometric content in different mathematical languages. Functional: classical and quantum procedures serve the same architectural role; neither approximates the other.

Quantum Encoding of Token Embeddings

This section establishes the foundation for the quantum-native transformer, outlining how token embeddings can be represented as quantum states. This encoding leverages key results from the structural isomorphism detailed in [27], which demonstrates a deep connection between classical embedding spaces and quantum mechanical systems. The classical embedding vector is first lifted into a general quantum semantic superspace and subsequently mapped to a specific quantum qubit representation for implementation.

Embeddings as Quantum States

where ak ≥ 0 are real amplitudes and φk are phases. Real-valued embeddings correspond to φ_k = 0. These phases provide additional degrees of freedom that are crucial for the interference patterns produced by unitary operations.

For GPT-scale models with d = 12288, the quantum representation requires n = 14 qubits, spanning a 2¹⁴ = 16384dimensional Hilbert space. The potential advantage of this representation lies in the inductive bias of parameterized quantum circuits: circuits with (poly(n)) parameters can generate states with specific entanglement structures that may capture linguistically relevant geometric relationships more naturally than unconstrained real vectors.

Key Results from the Structural Isomorphism

The structural isomorphism established in [27] demonstrates a fundamental link between L2-normalized embedding vectors and quantum states. The complete equivalence chain connecting classical, quantum mechanical, and circuit representations is

where S′C = (1 + a · b)/2 ∈ [0, 1] is the shifted cosine similarity, H′ is the Gram matrix of the embedding pair, a′1 is the first component of the embedding vector a in the eigenbasis of H′ (so that a′ 2 is the projection onto the dominant eigendirection), and ψ˜1 is the corresponding quantum amplitude in that basis. The first two equalities are classical, the middle two are quantum mechanical expectation values, and the last is a Born rule measurement probability on a quantum circuit of ⌈log₂ d⌉ qubits. Each equality is exact with no approximation at any step; the full derivation is given in [27].

This justifies the quantum encoding: token embeddings can be represented as quantum states, and their pairwise similarities are exactly recoverable as Born rule probabilities. Note that the isomorphism connects the squared modulus |⟨ψ_a |ψ_b⟩|2 to S_C′ ; the attention mechanism uses instead Re⟨ψ_q |ψk_j⟩, which is the direct quantum generalization of the classical dot product and is discussed in Section 6.

Position Encoding as Quantum Phase

Classical transformers use additive sinusoidal position encodings that alter both the magnitude and direction of token embeddings, disrupting the semantic geometry. The correct question is not how to replicate this additive encoding in the quantum framework, but what position encoding should be at the level of abstract structure. Position encoding must accomplish three things:

i. Distinguish tokens at different positions that have identical embeddings.

ii. Preserve the semantic content already encoded in the embedding.

iii. Modify the relational structure between tokens (how they attend to each other) in a position-dependent way.

Requirements (ii) and (iii) are in direct tension in the classical framework: any additive modification ej → ej + pj simultaneously changes both the token’s intrinsic content and its relational geometry. There is no way to satisfy (ii) and (iii) independently in â?d, because content and relational structure are encoded in the same real numbers.

In Hilbert space, this tension dissolves. Amplitudes |ψ_k|² and phases φ_k are orthogonal degrees of freedom: semantic content is carried by the amplitudes, while relational structure is carried by the phases. Position information therefore belongs naturally in the phases, where it modifies relational structure without affecting semantic content. In the quantum framework, position information is encoded as a unitary phase rotation

Interference-Based Projection

The learned projection matrices WQ , WK , and WV in standard attention mechanisms can be understood as parameterized unitary operators implemented through quantum interference. This section details how these projections can be realized in a quantum system, highlighting the advantages and differences compared to classical projections. Interference is a central concept, as it provides a fundamentally different way to compute relationships between semantic states compared to classical dot products [25].

Semantic Superposition, Projection, and the Interference Mechanism

Each output component is a sum of constructive and destructive interference contributions from all input dimensions, weighted by the gate parameters U_jk (θ) and the input phases e^iφk . This is formally similar to the classical projection [W x]_j = _k W_jkx_k, but the quantum implementation offers key differences: the phases e^iφk provide additional degrees of freedom, and the parameterization imposes a different inductive bias. The attention mechanism in transformers leverages this interference to compute relationships between tokens. The query, key, and value projections can be seen as shaping the interference patterns between token states, allowing the model to selectively attend to relevant information.

Inductive Bias and Parameter Efficiency

Understanding the differences in inductive bias is crucial for assessing whether quantum circuits can efficiently capture complex linguistic relationships. While classical projection matrices learn patterns entry by entry, quantum circuits leverage trigonometric functions and circuit topology to achieve structured cancellation patterns more easily. The unitary nature of quantum projections provides automatic regularization but defers information selection to the measurement stage. The central question is whether this inductive bias aligns well with the transformations that language models need to learn.

Although Eq. (11) is formally analogous to the classical projection [W x]_j = _k W_jkx_k, the two parameterizations impose qualitatively different inductive biases. These differences arise from the parameterization family: the set of transformations easily represented by a given parameterization. While any specific unitary or real matrix can be approximated by sufficiently complex classical or quantum circuits, respectively, the key distinction lies in what each parameterization makes easy to achieve with few parameters, and therefore the prior each imposes on the learned transformation.

Phase Structure and the Geometry of Cancellation

To understand how the quantum parameterization’s inductive bias differs from the classical one, consider encoding the signed linear combination x₁ + x₂ − x₃ into the amplitude of a designated output basis state.

Entanglement Topology as a Structured Prior on Correlations

The role of circuit topology is best understood by analogy with convolutional neural networks (CNNs). A fully connected layer connects every input dimension to every output dimension with independent weights, lacking any prior about relevant connections. A CNN, by contrast, connects each output only to a local neighborhood of inputs with shared weights, encoding the prior that nearby pixels are correlated. This prior allows CNNs to learn faster and generalize better than fully connected networks for image data. Similarly, quantum circuits can exploit entanglement locality to encode a structured prior about which qubit pairs carry important correlations.

The CNOT connectivity graph of a quantum circuit acts as this structured prior. When designing a circuit, one chooses which qubits are connected via CNOT gates, determining which correlations are easy to generate (adjacent qubits) and which require more circuit depth (distant qubits), as illustrated in Fig. 2. By choosing a specific topology, one encodes a hypothesis about the important correlations in the data.

To see this concretely, consider 4 qubits in a linear chain. Generating correlations between adjacent pairs requires a single CNOT layer, while distant pairs require multiple layers. This encodes the prior that important correlations are between adjacent dimensions. The topology is therefore a design hypothesis about the structure of the data, analogous to the local receptive field of a CNN. Table 2 summarizes the precise analogy.

	CNN	Quantum circuit
Prior	Nearby pixels are correlated	Certain qubit pairs are correlated
Mechanism	Local receptive field	CNOT connectivity graph
Restriction	Translation equivariance	Entanglement locality
Benefit	Fewer parameters, better generalization	Fewer parameters, structured compression
Risk	Wrong if data is not spatially local	Wrong if data correlations do not match topology

Table 2: Analogy between the inductive bias of a convolutional neural network and the entanglement topology of a quantum circuit. Both architectures encode a structured prior over which correlations are important: the CNN through a local receptive field that enforces translation equivariance, and the quantum circuit through a CNOT connectivity graph that enforces entanglement locality. In both cases the prior reduces the number of free parameters and improves generalization when the hypothesis matches the data, but permanently constrains the model when it does not.

A classical projection matrix has no such prior. The quantum circuit, by contrast, encodes a hypothesis about which inter-dimensional correlations are important and learns within that constrained family. If the data has the assumed correlation structure, the circuit learns efficiently. If the hypothesis is wrong, the circuit is permanently constrained.

Whether the correlations that trained transformers learn are compatible with a natural circuit topology is an open empirical question. The low intrinsic rank of weight updates in LoRA [6] and the specialization of attention heads suggest that learned correlations are not uniformly distributed, but neither constitutes direct evidence that the correlation structure matches a specific circuit topology.

The CNOT gate is used throughout as the standard entangling primitive, but any two-qubit entangling gate fulfills the same architectural role. On superconducting hardware, the CZ gate is often native and requires no decomposition. On trapped-ion hardware, the RZZ(θ) gate is native and offers the additional advantage of a continuously tunable entanglement strength, making the correlation prior itself a learnable parameter rather than a fixed binary connection. The entanglement topology argument of Fig. 2 applies to any choice of two-qubit entangling gate; what matters is the connectivity graph, not the specific gate implementing each edge.

Unitarity: Norm Preservation with a Cost

The quantum projection is unitary, meaning it preserves the norm of every state. This has two key consequences: an advantage for training stability and a constraint on information flow.

The advantage is automatic regularization of the projection step. Classical projection matrices can in principle have poorly conditioned singular values; a unitary matrix has all singular values equal to 1 by construction, so the projection step cannot introduce exploding or vanishing gradient pathologies. Classical transformers address gradient pathologies through techniques such as gradient clipping and layer normalization, which are applied globally; the quantum projection eliminates this specific source of ill-conditioning at the projection step without requiring additional mechanisms.

relevant after further processing. The potential disadvantage is that early projection acts as a form of regularization, preventing noisy directions from contaminating the attention scores. The quantum architecture carries all directions, including noisy ones, through the entire computation, relying on the POVM measurement to perform the selection at the end. This is not a technical inconvenience but a fundamental consequence of unitarity. The classical and quantum architectures are solving the same problem with qualitatively different strategies. Whether deferred information selection is advantageous or costly for language modeling is an open question that cannot be resolved by structural arguments alone and is identified as a direction for future empirical work.

Hadamard Gate and S-Gate Extension: Exploring Real and Imaginary Components

General Parameterized Projection

From Interference to Attention

This implies that the attention score depends on the relative phases, φk − φj,l, between query and key tokens. Consider two key tokens with identical amplitude profiles but different phases. In a classical transformer, these tokens produce identical attention scores because cosine similarity depends only on amplitudes. In the quantum architecture, the attention scores differ because the interference pattern depends on the phases. The quantum attention mechanism therefore distinguishes semantic states that are indistinguishable classically, enabling a strictly richer scoring function due to the complex-phase structure of quantum embeddings. The three properties identified above structured phase cancellation, entanglement topology prior, and norm-preserving isometry all contribute to this richer scoring function.

Attention as Partition Function

This section establishes two distinct correspondences. The first is a classical mathematical identity: the softmax attention weights are exactly a Boltzmann distribution, a fact that holds independently of any quantum mechanical interpretation. The second is specifically quantum: the Boltzmann weights can be realized as Born rule probabilities on a quantum circuit, up to a rank-preserving rescaling of the inverse temperature [24]. The quantum content of this section lies entirely in the Born rule bridge, not in the Boltzmann identity itself. This distinction motivates the use of density matrices for value aggregation.

Softmax is a Boltzmann Distribution

The Born Rule Bridge

Complete Quantum Attention

The full quantum attention mechanism proceeds in four steps: project via interference

The NLSE as Quantum Activation Function

The sections from here through Section 9 are ordered by theoretical dependency rather than by position in the layer stack. The NLSE section precedes the FFN section because the FFN analysis depends on NLSE machinery derived here.

In classical transformer architectures, activation functions such as ReLU and GELU serve one essential architectural purpose: they introduce nonlinearity into the computation. Without nonlinearity, the entire network collapses to a single linear map regardless of depth. As seen in classical LLM experiments, the specific functional form of the activation is secondary to this architectural role.

This motivates the question: what is the minimal way to introduce nonlinearity into a quantum mechanical system while preserving the Hilbert-space structure established by the encoding? Intermediate semantic processing must preserve Hilbert-space structure, ruling out classical activation functions applied pointwise to amplitude vectors, since such operations do not respect the normalization constraint ⟨ψ | ψ⟩ = 1 and break the Born rule interpretation of squared amplitudes as probabilities.

Derivation of the Cubic NLSE

The cubic NLSE can be derived from three requirements:

Properties of the Cubic NLSE

Alternative Nonlinearities and Semantic Bias Selection

The cubic NLSE is the canonical starting point and the minimal norm-preserving nonlinearity consistent with Hilbert-space structure. Its value is twofold: it is the simplest possible nonlinearity by construction, and it has an extensive theoretical literature with well-understood properties including soliton solutions, integrability in one dimension, and stability analysis. These properties make it the natural first choice for theoretical work, not because it is necessarily the correct model for any specific regime, but because it is the tractable one from which rigorous results can be derived.

The principal nonlinearity families, ordered by complexity, are listed below. The cubic and Mexican hat cases are discussed in detail in the following subsections; the remaining two are included for completeness.

The Mexican Hat Potential

The Mexican hat potential is the natural next step beyond the cubic: it is the simplest well-studied potential that introduces a qualitatively new feature absent from the cubic, namely spontaneous symmetry breaking. A particularly structured example is obtained from the quartic potential [20]

The reason to study this potential is that the Higgs mechanism, Goldstone boson absorption, and spontaneous symmetry breaking are well-understood consequences of it, and these theoretical results can be imported directly into the transformer context. The Mexican hat potential is the entry point to this body of theory, just as the cubic NLSE is the entry point to soliton theory and integrability results. The consequences for the relationship between W1 and W2 are developed in Section 9.1.3.

Solitons as Stable Semantic Representations

A stable solution of the NLSE in a self-reinforcing mode is a soliton [25]. In this state, the nonlinear self-phase modulation exactly balances dispersion, maintaining the shape of the state indefinitely. This is the functional analog of a fixed point of a classical activation function, where repeated application of the nonlinearity leaves the representation unchanged. The soliton is one of the theoretical results that the cubic nonlinearity makes available; the Mexican hat potential brings a different class of stable solutions, namely the minima of the potential, which correspond to semantically committed states rather than propagating wavepackets.

Hardware Implications

The NLSE cannot be implemented on current gate-based quantum hardware, which supports only unitary (linear) operations. This is a fundamental constraint, not an engineering limitation, and it constitutes a concrete hardware requirement implied by the quantum-native transformer framework. Any quantum system intended to replicate transformer-class computation must support nonlinear dynamics. Possible paths to physical realization include: (i) approximating the NLSE nonlinearity through ancilla-based measurement-and-feedback schemes; (ii) using photonic quantum hardware, which naturally supports nonlinear optical interactions (Kerr effect); (iii) developing LLM-specific quantum hardware designed from the outset to support nonlinear dynamics. The Mexican hat nonlinearity of the large-model regime has an additional natural physical realization in systems with double-well potentials, which are wellstudied in nonlinear optics and superconducting circuit QED, providing a concrete hardware path for implementing the symmetry-enhanced regime independently of the minimal cubic case. These are identified as separate research questions.

The present paper validates the NLSE step in simulation. Quantum simulators are invaluable tools for investigating QSC circuits, allowing for the straightforward implementation of nonlinearities and the generation of noise-free results. The utility of these simulated results then motivates the development of corresponding real quantum hardware capable of supporting such nonlinear dynamics.

Extension to Mixed States: The Nonlinear von Neumann Equation

Residual Connections as Hamiltonian Evolution

Feed-Forward Network as Continuous Nonlinear Evolution

This section explores the feed-forward network (FFN) within the transformer architecture, contrasting the classical and quantum approaches. We propose that the dimensional expansion (d → 4d → d) in classical FFNs arises from the limitations of real-valued computation, specifically the absence of phase information. The quantum FFN, operating in the quantum semantic superspace, leverages both amplitude and phase degrees of freedom, enabling complex nonlinear transformations without dimensional expansion.

The Classical FFN and Its Dimensional Structure

The classical FFN applies a two-stage transformation: FFN(x) = W2 σ(W1x + b1) + b2, where W1 expands the representation to a higher-dimensional space (dff = 4d), σ is a pointwise nonlinearity, and W2 projects back to the model dimension. The expansion factor, often 4, is empirically determined and lacks a theoretical derivation within the classical framework. The quantum framework provides a consistency argument for this factor.

In the quantum semantic superspace, a quantum state |ψ⟩ ∈ C²ⁿ carries both amplitude and phase information, degrees of freedom absent in a real vector x ∈ R^d . The classical FFN’s dimensional expansion can be understood as a consequence of the real restriction of a quantum computation that naturally operates on complex states.

Local U(1)d Symmetry and the Consistency Argument for the Dimensional Expansion

The companion paper [27] establishes that the quantum formulation admits a local U(1) gauge symmetry in the eigenbasis. This symmetry, invisible classically, removes redundant phase degrees of freedom, ensuring that the quantum and classical representations have the same physical content. The cubic NLSE nonlinearity breaks this local U(1)d symmetry, inducing a component-dependent phase rotation that allows the relative phases between components to carry new physical information.

The dimensional expansion factor of 4 is consistent with two factors of 2 arising from the quantum framework: (i) the real restriction of the unitary, requiring 2d real numbers to represent a d-dimensional complex space, and (ii) the breaking of the U(1)d symmetry by the nonlinearity, requiring approximately 2d real dimensions to represent the new phase information. This gives the expansion and contraction cycle

The dimensional expansion is therefore not an arbitrary empirical choice but the natural intermediate dimension of the real-valued shadow of a quantum computation involving symmetry breaking and restoration.

The Quantum FFN as Continuous Evolution

In the fully quantum-native architecture, the quantum state flows continuously through all L layers, with the single POVM measurement occurring only at the output. The FFN is therefore not a discrete three-stage operation but a segment of continuous nonlinear quantum evolution, governed by

measures how well this explanation fits trained models. A small ϵ_W in minimal models would be consistent with the quantum explanation; a large ϵW would indicate that the explanation is incomplete or that the minimal-regime assumption does not apply, but would not by itself falsify the quantum framework, since the classical FFN structure is itself an empirical design choice rather than a derived necessity.

Symmetry Enhancement in Large Models and the Higgs Analogy

In large multi-head models the relevant symmetry structure is richer than in the minimal single-head regime. With H attention heads, each operating on a subspace of dimension d/H, the global U(1)d phase freedom decomposes into H independent local phase freedoms, promoting the global symmetry group to a local gauge symmetry

even when the global relation W2 ≈ WT fails.

Several empirical phenomena in large LLMs are qualitatively consistent with this picture: the tendency of large models to commit to a single semantic interpretation rather than maintaining superpositions, the sharpening of polysemy resolution with model scale, and the spontaneous specialization of attention heads to distinct functional roles. Each corresponds naturally to a feature of Mexican hat symmetry breaking, and none has a comparably natural explanation within the classical transformer framework.

The full derivation of Eqs. (47) and (48) from first principles, including the precise identification of the Goldstone modes and the mechanism of their absorption into W2, is identified as a separate research question.

Bias Terms as the Free Hamiltonian

Returning to the minimal-regime quantum FFN, the bias terms b1 and b2 of the classical FFN have a natural quantum counterpart in the free Hamiltonian Hˆ0 of the NLSE (Eq. (35)). A constant bias b shifts all components of the representation uniformly; the free Hamiltonian Hˆ0 plays the same architectural role, providing a state-independent baseline evolution that is present regardless of the input. In the pilot, Hˆ0 = 0 is used, corresponding to zero bias, so that the NLSE reduces to pure nonlinear phase rotation. In a trained model, Hˆ0 would be a learnable Hermitian operator, playing the role of both bias terms simultaneously.

External Parameterization of the Nonlinear Coupling

In classical transformers, activation functions (ReLU, GELU, SiLU) are static architectural choices, fixed before training. Learning occurs through weight matrices that exploit this fixed nonlinearity. The NLSE coupling γ plays an analogous role in the QSC: a fixed, pre-training architectural parameter whose presence and norm-compatibility matter more than its precise value.

In a quantum-native setting, however, γ has a unique physical consequence: it is a property of the computational medium (e.g., the Kerr coefficient of a nonlinear photonic waveguide), not a software parameter. Different values of γ require different physical media, just as different activation functions require different architectural choices that cannot be changed mid-training. QSC training therefore has a distinct two-level structure: (i) Architecture design: choosing the nonlinear medium and fixing γ, analogous to choosing the activation function; and (ii) Training: updating gate angles with γ held constant. Comparing different values of γ requires separate training runs, analogous to ablation studies over activation functions.

Output as Generalized Measurement

The output layer of the transformer maps the final hidden state to a probability distribution over the vocabulary. In the quantum framework, this is a positive operator-valued measure (POVM)

The Quantum Semantic Circuit

This section provides a formal definition of the Quantum Semantic Circuit (QSC) architecture, detailing its components, layer-by-layer operation, and trainable parameters. It establishes the QSC as a concrete quantum-native implementation of the transformer concept, highlighting key differences in parameterization and operation compared to classical transformers.

Formal Definition

The Quantum Semantic Circuit (QSC) is a parameterized quantum system designed to mirror the key operations of a classical transformer in a quantum-native way. It can be formally described as a tuple

Layer-by-Layer Operation

Position encoding is applied once before the layer loop, as shown in Figure 4

Trainable Parameters of the QSC

Having established the full architecture, we can now identify precisely which quantities are trainable in the QSC and how they compare to the classical transformer. While this section catalogues the trainable parameters and their scaling, the experimental estimation of these parameters, analogous to backpropagation in classical models, is a separate research question.

Parameter Taxonomy

The QSC has three qualitatively distinct categories of parameter, which differ not only in quantity but in the nature of how they are set. Category 1: Gate angles (learned during training). The primary trainable parameters are the rotation angles of the parameterized unitary circuits (U_Q, U_K, U_V , U_ff, and U_embed). For an L-layer hardware-efficient ansatz on n qubits, each unitary contributes O(nL) angles. Table 3 summarizes the gate angle count per QSC layer, compared to the classical transformer.

Total Parameter Count and Scaling

Table 4 summarizes the total trainable parameter counts for representative model scales, comparing the classical transformer to the QSC. The quantum column counts only Category 1 parameters (gate angles); γ is excluded as it is not updated during training.

WHAT IS NOT COUNTED

Two classical parameter categories are absent from the QSC count entirely, not merely compressed:

i) Layer normalization parameters: Classical transformers require learned affine parameters for recentering and rescaling. The QSC requires no such parameters because unitarity automatically enforces ⟨ψ |ψ⟩ = 1 at every step, preserving the full geometric structure of the state.

ii) FFN output projection W₂: The classical W2 contracts the expanded FFN representation back to dimension d. This contraction is unnecessary in the QSC because the state remains in C²ⁿ throughout, leveraging phase information and avoiding dimensional expansion. Consequently, the W₂ parameters have no quantum analog.

Simulator Pilot

This section describes a simulator pilot conducted to validate the QSC architecture and demonstrate the computational realizability of its key components.

Setup and Objectives

The pilot validates the QSC architecture on the Qiskit Aer statevector simulator. The objectives are:

i. Validate all eight correspondences: five (embedding, projection, attention, NLSE, and output) by direct circuit execution, two (position encoding and FFN correspondence) by mathematical argument, and one (FFN unitary) by simplifying assumption (Uff = k ).

ii. Demonstrate that the QSC correctly disambiguates a polysemous token in context.

iii. Confirm that the NLSE nonlinearity does real computational work beyond linear evolution.

iv. Verify norm preservation throughout, maintaining the Born rule interpretation.

Task: Lexical Disambiguation

The pilot uses a 4-token vocabulary = money, bank, river, water on n = 3 qubits, spanning = C⁸. The task is lexical disambiguation: given context [river, water] and query token bank, predict the next token. The token bank is polysemous, with both a finance sense (bank account, money) and a nature sense (river bank, water’s edge); the context should activate the nature sense, making water the expected output. The task is minimal yet non-trivial, requiring the full pipeline to function correctly and consistently, see Fig 5. The pilot parameters are:

• Qubits: n = 3, dimension d = 8

• NLSE: γ = 0.05, t = 0.05, N_steps = 50, total evolution time T = N_steps

• I = 2.5; effective nonlinear phase γT = 0.125

• Attention inverse temperature: β = 5.0

• Residual weight: α = 0.2, chosen as a pilot design parameter giving a non-trivial mix between the residual and processed outputs; the classical transformer corresponds to the symmetric case α = 0.5 in the normalized combination, and the optimal value for a trained model would be learned from data.

• FFN unitary: Uff = k (identity) for this pilot; the FFN unitary step is omitted as a simplifying assumption, and the FFN correspondence is validated by mathematical argument rather than direct circuit execution.

Token Embeddings

Token embeddings are hand-crafted 8-dimensional unit vectors encoding the required semantic geometry. The embedding axes are:

• Dimension 0: finance (+) / nature (−)

• Dimension 1: water feature (primary)

• Dimension 2: nature/outdoor feature

• Dimension 3: land/earth feature

• Dimension 4: flow/movement feature

• Dimensions 5–7: secondary features The key design requirement is cos(bank, water) > cos(bank, river), so that water receives higher attention weight than river when bank is the query. This is achieved by making bank primarily a water-edge token (high dimension 1,

Table 5: Cosine similarity matrix for the pilot vocabulary. The critical requirement cos(bank,water) = 0.810 > cos(bank,river) = 0.764 ensures that water receives higher attention weight than river when bank is the query. Negative similarities between money and the nature tokens confirm that the finance and nature senses are geometrically separated.

NLSE Parameter Sweep

Before running the full pilot, a parameter sweep over (γ , _t , N_steps) identifies valid configurations. A configuration is valid if:

(i) Norm preservation: maxsteps||ψ_step|| − 1 < 10⁻⁶

(ii) Nonlinearity signal: ||ψ_nl − ψlin||² > 10⁻³

The sweep covers γ ∈ {0.001, 0.005, 0.01, 0.05, 0.1}, Δt ∈ {0.001, 0.005, 0.01, 0.05}, and N_steps ∈ {10, 20, 50, 100}, giving 80 configurations in total, of which 45 satisfy both criteria. The pilot uses γ = 0.05, Δt = 0.05, N_steps = 50, giving total evolution time T = 2.5 and nonlinearity signal 5.51 × 10−2.

Component Validation

All eight validation tests pass. The first five are validated by direct circuit execution; two by mathematical argument; and one (FFN unitary) by simplifying assumption (Uff = 1 ).

(i) Normalization. All four token embeddings satisfy ||ψ_i⟩|| = 1 to numerical precision (< 10−10).

(ii) Structural isomorphism. For all (4) = 6 token pairs, the shifted cosine similarity S′ = (1 + a · b)/2 equals the Born rule probability 2 C

P(|0 · · · 0⟩) = |⟨m| a⟩|2, where |m⟩ = (|a⟩+|b⟩)/||a⟩+|b⟩||. Maximum deviation: < 10−10.

(iii) Interference projection. The two-layer parameterized circuit implements a unitary satisfying U†U = k to < 10−10, preserving the norm of all token embeddings exactly.

(iv) Attention beta rescaling. Boltzmann (β = 5.0) and Born rule (β = 10.0) assign identical rank ordering to all context tokens. Maximum numerical difference: 2.03 × 10−2, consistent with the claim of rank preservation rather than numerical equality.

(v) NLSE parameter sweep. 45 of 80 configurations satisfy both validity criteria. The pilot configuration is confirmed valid with norm drift at machine epsilon and nonlinearity signal 5.51 × 10−2.

(vi) Position encoding (derivational). The quantum phase rotation preserves all amplitudes |ψ |2 exactly (norm deviation < 10−10), while the classical additive encoding changes the norm. The linearization error 1.49×10−1 confirms that the classical encoding discards higher-order terms. The real shadow error is 0 to machine precision, confirming that the classical sinusoidal functions are exactly the real and imaginary parts of e^ij/100002k/d.

(vii) FFN correspondence (functional). The quantum FFN operates in C⁸ throughout with no dimensional change (d = d = 8), while out the classical FFN expands to 4d = 32. Norm is preserved to < 10−10 without layer normalization. The phase degree-of-freedom fraction 0.049 > 0 confirms that phase degrees of freedom are active and carry information not representable in the real-valued classical FFN.

(viii) FFN unitary (simplifying assumption). Uff = k is assumed for this pilot. Norm is trivially preserved.

Partition Function Bridge

Table 6 verifies the partition function bridge that both distributions assign higher weight to water than to river, confirming that the rank-preserving rescaling β → 2β does not alter the attention ordering. The semantic energies E_water = −0.2864 < E_river = −0.2701 are consistent with cos(bank, water) = 0.810 > cos(bank, river) = 0.764.

Table 6: Partition function bridge verification. Boltzmann (β = 5.0) and Born rule (β = 10.0) assign the same rank ordering: water > river in both cases. The numerical difference 2.03 × 10⁻² is expected: the claim is rank preservation under β → 2β, not numerical equality.

End-to-End Forward Pass

Geometric Consistency of the Pipeline

The QSC pipeline maintains geometric consistency throughout by operating in a single space: the value-projected Hilbert space UV H. The attention output, NLSE activation, residual term, and POVM basis vectors all live in UV H

This consistency is essential: mixing projected and unprojected states would make the POVM overlaps geometrically meaningless

Output Probabilities

Table 7 shows the output probability distributions for the linear (γ = 0) and nonlinear (γ = 0.05) forward passes. Both correctly predict water. The POVM overlaps confirm why: |⟨UV ψwater | ψout⟩|2 = 0.881 > |⟨UV ψ_river | ψ_out⟩|2 = 0.853 > |⟨UV ψ_bank | ψ_out⟩|2 = 0.708, giving P (water) = 0.3125 as the highest probability after renormalization.

Table 7: Output probability distributions for the linear (γ = 0) and nonlinear (γ = 0.05) QSC forward passes. Both correctly predict water. The NLSE redistributes probability mass, weakening bank (−0.0021) and strengthening money (+0.0022) while leaving the water prediction intact, demonstrating that the cubic self-interaction does real computational work beyond linear evolution. â??P = nonlinear minus linear probability.

The dominant NLSE effect is a state-dependent phase rotation

which modifies the interference pattern between the NLSE output and the residual term, shifting overlap from bank toward money. The correct prediction (water) is robust to this redistribution, remaining the highest probability token in both passes. The effect magnitude (∼ 10−3) is consistent with the nonlinearity signal 5.51 × 10−2 attenuated by the residual weight and POVM projection geometry.

Summary of Pilot Results

Table 8 summarizes the validation status for all eight correspondences.

Table 8: Validation summary for all eight correspondences. Type abbreviations: Repr. = Representational; Func. = Functional; Deriv. = Derivational; Ex.up. = Exact up to rescaling; Assumpt. = Simplifying assumption. The first five correspondences are validated by direct circuit execution on the Qiskit Aer statevector simulator; position encoding and FFN correspondence are validated by mathematical argument; FFN unitary is validated by simplifying assumption.

The pilot establishes that the QSC architecture is self-consistent, that it correctly resolves the disambiguation task, and that the NLSE nonlinearity does real computational work while preserving norm to machine epsilon throughout.

What the pilot does not establish is equally important to state. The 4-token, 3-qubit task is a proof of concept, not a performance benchmark. The embeddings are hand-crafted; a trained model would learn them. The projection unitaries are random; a trained model would optimize them. The NLSE coupling γ is fixed as an architectural constant; a trained model would be trained separately under different fixed values of γ, analogous to ablation studies over activation functions. The pilot validates the architecture’s self-consistency and computational realizability, not its performance at scale.

Discussion

The QSC framework establishes correspondences between classical transformer components and quantum mechanical operations, classified by their logical status. Three correspondences are exact mathematical identities: L2normalized embeddings are quantum states, cosine similarity is a Born rule probability, and softmax attention weights are a Boltzmann distribution. These hold regardless of whether a quantum computer is ever used. The classical sinusoidal position encoding is the first-order real projection of an exact quantum phase rotation; the quantum version is the natural object and the classical version is its shadow. A key aspect of the QSC is the use of quantum interference in the projection operations, providing a richer and more nuanced mechanism for computing relationships between token states compared to classical dot products. The cubic NLSE is the unique minimal norm-preserving nonlinearity consistent with the Hilbert-space structure of the encoding, derived from three requirements rather than chosen by analogy. The simulator pilot confirms that all eight correspondences are self-consistent and that the NLSE does measurable computational work on a non-trivial disambiguation task.

The framework also offers a natural explanation for empirically observed structure in the classical FFN that has no account within the classical framework itself. The dimensional expansion d → 4d → d and the asymmetry between W₁ and W₂ can be understood as shadows of quantum symmetry breaking and restoration in the semantic superspace. In the minimal single-head regime, the symmetry cycle suggests W2 ≈ W ₁T as a consistency indicator; in large multi-head models, the Higgs analogy suggests that the rank of W₂ − W T scales with the number of attention heads H and that the blockwise transpose relation holds within each head’s subspace. These explanations are informative regardless of whether they are confirmed empirically, since the classical FFN structure is itself an empirical design choice rather than a derived necessity.

Several directions present themselves for empirical follow-up. Whether trained transformer projection matrices have the structured cancellation and low-rank correlation patterns that would make the quantum circuit ansatz a good fit is an open question; the LoRA evidence for low-rank weight updates is suggestive but not conclusive. Assessing how well a hardware-efficient ansatz approximates trained projection matrices, and whether the approximation error correlates with task performance, would directly test the practical reach of the quantum parameterization. On the hardware side, the NLSE nonlinearity cannot be implemented on current gate-based devices, but photonic Kerr nonlinearities, measurement-and-feedback schemes, and double-well potential systems in superconducting circuit QED each offer concrete paths toward physical realization, and determining which is most practical for transformer-scale computation is an engineering question that can be addressed independently of the theoretical framework. Parameter compression and computational speedup relative to classical transformers are theoretically possible but contingent on conditions not yet verified: the learned transformations must lie within the submanifold accessible to the circuit ansatz, and the shot overhead of quantum measurement must be overcome either through larger embedding dimensions or demonstrated parameter efficiency at comparable task performance.

The QSC framework is a theoretical foundation, not a deployed system. The correspondences established in this paper map each classical transformer component to its most natural quantum mechanical counterpart, preserving the architectural logic of the classical system within the quantum framework. This mapping is a necessary first step, but it is not necessarily the correct or complete path to a native quantum language model. The classical transformer architecture was designed under the constraints of real-valued computation, and its structure reflects those constraints. A truly native quantum LLM may require additional structures with no classical analog, or may need to modify the quantum counterparts presented here in ways that depart substantially from the classical blueprint. The density matrix formulation of value aggregation, the soliton interpretation of stable representations, and the imaginary component of the attention score accessible via the SH circuit are examples of quantum structures that have no direct classical counterpart and whose role in a native quantum LLM is not yet understood. If a system built from the components described in this paper produces suboptimal results, this should not be interpreted as evidence that the theoretical framework is wrong. It may instead indicate that the framework is incomplete: that the correct quantum architecture requires structures beyond those obtained by direct translation from the classical case.

References

1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
2. Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. International Conference on Learning Representations, 2014.
3. Rush, A. M., Chopra, S., & Weston, J. (2015, September). A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 379-389).
4. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Computer Science, Linguistics, 2018.
5. Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2022). Efficient transformers: A survey. ACM Computing Surveys, 55(6), 1-28.
6. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., ... & Chen, W. (2022). Lora: Low-rank adaptation of large language models. International Conference on Learning Representations.1(2) 3.
7. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
8. Dunjko, V., & Briegel, H. J. (2018). Machine learning & artificial intelligence in the quantum domain: a review of recent progress.Reports on Progress in Physics, 81(7), 074001.
9. Rebentrost, P., Mohseni, M., & Lloyd, S. (2014). Quantum support vector machine for big data classification. Physical review letters, 113(13), 130503.
10. Farhi, E., & Neven, H. (2018). Classification with quantum neural networks on near term processors. arXiv preprint arXiv:1802.06002.
11. Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., & Lloyd, S. (2017). Quantum machine learning. Nature, 549(7671), 195-202.
12. Preskill, J. (2018). Quantum computing in the NISQ era and beyond. Quantum, 2, 79.
13. Ladd, T. D., Jelezko, F., Laflamme, R., Nakamura, Y., Monroe, C., & O’Brien, J. L. (2010). Quantum computers. nature, 464(7285), 45-53.
14. Nielsen, M. A., & Chuang, I. L. (2010). Quantum computation and quantum information. Cambridge university press.
15. Arute, F., Arya, K., Babbush, R., Bacon, D., Bardin, J. C., Barends, R., ... & Martinis, J. M. (2019). Quantum supremacy using a programmable superconducting processor. nature, 574(7779), 505-510.
16. Coecke, B., de Felice, G., Meichanetzidis, K., & Toumi, A. (2020). Foundations for near-term quantum natural language processing. arXiv preprint arXiv:2012.03755.
17. Widdows, D., & Widdows, D. (2004). Geometry and meaning (Vol. 773). Stanford: CSLI publications.
18. Coecke, B., Sadrzadeh, M., & Clark, S. (2010). Mathematical foundations for a compositional distributional model of meaning. Lambek Festschrift Linguistic Analysis, 36(1).
19. Gupta, A., Kaur, K., Gupta, V., & Shah, C. (2025). QLENS: Towards A Quantum Perspective of Language Transformers. arXiv preprint arXiv:2510.11963.
20. Laine, T. A. (2025). Semantic Wave Functions: Exploring Meaning in Large Language Models Through Quantum Formalism. OA J Applied Sci Technol, 3(1), 01-22.
21. Laine, T. A. (2025). The Quantum LLM: Modeling Semantic Spaces with Quantum Principles. OA J Applied Sci Technol, 3(2), 01-13.
22. Laine, T. A. (2026). Quantum LLMs Using Quantum Computing to Analyze and Process Semantic Information. OA J Applied Sci Technol, 4(1), 01-19.
23. Laine, T. A. (2026). Discrete semantic states and Hamiltonian dynamics in LLM embedding spaces. OA Journal of Applied Science and Technology, 4(1), 1–23.
24. Laine, T. A. (2026). Quantum computation of partition function similarity for large language models. OA Journal of Applied Science and Technology, 4(1), 1–11.
25. Laine, T. A. (2026). Quantum hierarchy for understanding LLM representations by modeling linear projections and nonlinear dynamics. OA Journal of Applied Science and Technology, 4(1), 1–43.
26. Laine, T. A. (2026). Quantum algorithms for large language models on noisy intermediate-scale quantum computers. OA Journal of Applied Science and Technology, 4(1), 1–13.
27. Laine, T. A. (2026). Structural Isomorphism Between LLM Embedding Spaces and Quantum Mechanical Systems. OA J Applied Sci Technol. 4(2), 1-18.