… Overall, we offer a data-centric account of why larger models learn tasks that smaller models fail to. This helps explain why larger models are better in practice, and it can inform practical questions concerning model sizing and training data mixtures . …
… This paper provides that account under a unified loss family JQ using the Tsallis q-logarithm. …
Papers arxiv:2605.01640 Prescriptive Scaling Laws for Data Constrained Training Published on May 2 Submitted by Justin Lovelace on May 8 Authors: , , , , Abstract A modified scaling law accounts for data repetition effects and provides compute-optimal training strategies for data-constrained scenar… …
… Probing and Steering Contextual Privacy Norms in Large Language Model Representations 2026 Beyond Social Pressure: Benchmarking Epistemic Attack in Large Language Models 2026 Please give a thumbs up to this comment if you found it helpful! …
… But adopting marginal token allocation as the shared accounting object explains why systems that locally minimize tokens globally misallocate them, predicts a small set of recurring failure modes over-routing, over-delegation, under-verification, serving congestion, stale rollouts, cache misuse , a… …
… Conventional stochastic differential equation SDE solvers fail to account for this dynamic, naively injecting uniform white noise throughout the entire process and misusing the finite energy budget. …
… The following papers were recommended by the Semantic Scholar API Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling 2026 DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference 2026 SinkRouter: Sink-Aware Routing for Efficient Lon… …
… Ablation experiments establish the SSM as the dominant compression engine: it alone accounts for a 46.6% size reduction over a frequency-count baseline and beats xz without any n-gram component, while n-gram tables provide a complementary 4.1% gain through exact context memorisation. …
… We present TIDE, the first framework for cross-architecture dLLM distillation , comprising three modular components: 1 TIDAL , which jointly modulates distillation strength across training progress and diffusion timestep to account for the teacher's noise-dependent reliability ; 2 CompDemo , which … …
… We show that this account is true but incomplete. …