Advances in Machine Learning & Artificial Intelligence(AMLAI)

Department of Family Medicine, Dong-eui Medical Center, Yangjeong-ro, Busanjin-gu, Busan, Republic of Korea

Publications

• Research Article   
  Bose-Einstein Condensation Analogy in Transformer Token Collapse: A Cross-Modal and Cross-Domain Analysis of Contraction Dynamics
  Author(s): Chur Chin*
  
  Token representation collapse, wherein token embeddings converge to near-identical vectors as network depth increases, represents a fundamental challenge in modern Transformer architectures across text, vision, and audio modalities. This paper presents a theoretical and empirical framework demonstrating that the mathematical structure underlying this collapse phenomenon is isomorphic to Bose-Einstein Condensation (BEC) in quantum physics, flat-band superconductivity in Magic-Angle Twisted Bilayer Graphene (MATBG), and hyperbolic phonon polariton propagation in hexagonal boron nitride (hBN) heterostructures [1]. By analyzing the layer-wise cosine similarity trajectories of Vision Transformer (ViT) and Wav2Vec2 models, we show that audio modality undergoes dramatically faster and more severe oversmoothing than visual modality due to strong temporal autocorrelation in the initial embeddi.. Read More»
  

  Abstract HTML PDF