…Semantic self-consistency improves this by clustering sampled answers by meaning using natural language inference , but it adds both sampling cost and external inference overhead. We show that first-token confidence , phi…
…existing strategies are either target-misaligned, incapable of few-step generation, or too costly to scale. We propose Causal Forcing++, a principled and scalable pipeline that uses causal consistency distillation ( causal CD…
…However, existing approaches typically require either training recurrent models from scratch or applying disruptive retrofits, which involve substantial computational costs and may compromise pretrained capabilities. To address these limitations, we introduce Looped…
…AI-generated summary The key-value (KV) cache is a major bottleneck in long-context inference , where memory and computation grow with sequence length. Existing KV eviction methods reduce this cost but…
…AI-generated summary Long-context inference in decoder-only language models is costly because long prompts are processed during Prefill , cached at every layer, and repeatedly attended to during autoregressive Decode . We…
…AI-generated summary While Mixture-of-Experts (MoE) scales model capacity without proportionally increasing computation, its massive total parameter footprint creates significant storage and memory-access bottlenecks, which hinder efficient end-side…
…While diffusion language models attempt to break this barrier via parallel generation, they suffer from significant performance degradation, high training costs, and a lack of rigorous convergence guarantees. Orthrus resolves this dichotomy…
…provides the greatest improvements while proposing entropy-based diagnostics to determine when such costly rewrites are beneficial. AI-generated summary Embedding-based code retrieval often suffers when encoders overfit to surface syntax…
…attention with a routed mixture-of-experts approach that reduces computational cost while maintaining performance and handling long contexts effectively. AI-generated summary DeepSeek Sparse Attention (DSA) sets the state of the…
…To address the first challenge, we propose Lite3R, a model-agnostic teacher-student framework that replaces dense attention with Sparse Linear Attention to preserve important geometric interactions while reducing attention cost. To…