Sparse Context Augmentation (SCA): A Dual-Output Indexing Architecture for Mitigation of O(N2) Scaling and Auditable LLM Deployment

Abstract

Chaiya Tantisukarom

Current Large Language Model (LLM) architectures face a fundamental quadratic scaling bottleneck (O(N2 )) inherent to the self-attention mechanism, leading to prohibitive costs and latency for long-context tasks. This paper proposes a framework, Sparse Context Augmentation (SCA), which implements a DualOutput Indexing Architecture utilizing a Just-in-Time Retrieval-Augmented Generation (JIT-RAG) strategy. This approach, intended as a strategic, creator- side mitigation, decouples long-context processing from the expensive internal context window by migrating history to external, low-cost storage. Crucially, the model is architected to simultaneously generate the standard user response (var2 ) and a compact, fixed-maximum-size semantic index (var1 ). This index (var1) acts as a highly condensed, low- fluidity key message log, providing System State Transparency and a predictable cost floor. This strategy maintains conversational coherence with a complexity of O(k2 ) per turn, where k is a fixed, small maximum context size, effectively decoupling the cost from the total history length T. The framework provides a mutual value proposition, transforming the LLM’s auditable state logs into user-facing transparency features, resulting in massive cost savings for the creator and unparalleled reliability for the user.

PDF