…View arXiv page View PDF Add to collection Community We investigate rare token and contextual collapse problem in LLM design and propose to inject token identity information to each transformer layer. the…
…short-horizon heads can receive the largest routing share because their prediction targets are easier, even when they are less aligned with delayed task success. As a structural response, we study Target…
…View arXiv page View PDF Project page GitHub 5 Add to collection Community LWS is a simple “free lunch” for full-duplex speech models: without changing the model architecture, we add a…
…View arXiv page View PDF GitHub 20 Add to collection Community Sharing our ACL Findings paper, WindowsWorld. TL;DR: We introduce a process-centric benchmark for evaluating GUI agents on professional cross…
…View arXiv page View PDF GitHub 20 Add to collection Community SparDA adds a fourth per-layer projection, the Forecast, that predicts which KV blocks the next layer will select. Running selection…
…The methodological contributions -- the relative-rate estimator that cancels the shared-difficulty confound, the wrong-group control , and a pre-specified failure taxonomy -- transfer to any inductive bias whose strength can be…
…View arXiv page View PDF GitHub 5 Add to collection Community Prior work has shown that fine-tuning large language models on malicious or incorrect outputs in narrow domains can induce broad…
…View arXiv page View PDF GitHub 0 Add to collection Community Excited to share our latest work on visual RAG! 🚀 Existing iRAG systems linearise documents into plain text, which (1) breaks visual…
…Both stages share a skeleton-aware Global-Local Graph-guided Multi-Head Attention (GL-GMHA) module for joint-level local reasoning and global coordination . Experiments on Truebones Zoo and Objaverse show that…
…To remain expressive, the indexer uses many query heads (for example, 64 on DeepSeek-V3.2) that share the same selected token set; this multi-head design is precisely what makes the…