…May 7 Submitted by Yuwei Yin on May 11 University of British Columbia Authors: Yuwei Yin , Chuyuan Li , Abstract IntentGrasp is a benchmark for evaluating large language models' intent understanding capability, demonstrating…
…Across the eight model backends, Lucky rates range from 0.5% to 23.2%, and some models move by as many as five rank positions when ranked by quality score instead of…
…We evaluate 10 frontier LLMs on Ψ-Bench and find that while most models can produce coherent and reasonable arguments, even state-of-the-art models still leave considerable room for improvement…
…to evaluate conversational agent reliability, revealing significant performance gaps among leading models. Generated by Qwen/Qwen2.5-Coder-32B-Instruct As recommender systems transition toward agentic, multi-turn conversational interfaces , evaluation paradigms…
…The following papers were recommended by the Semantic Scholar API GEditBench v2: A Human-Aligned Benchmark for General Image Editing (2026) Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation…
…The following papers were recommended by the Semantic Scholar API WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models (2026) WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native…
Papers arxiv:2606.00285 Model-Based Quality Assessment for Massively Multilingual Parallel Data Published on May 29 Submitted by Shaoxiong on Jun 2 MaLA-LM Authors: , Zihao Li , , Abstract Multilingual parallel-data…
…for Evaluating Agent Values Published on May 11 Submitted by Haoran Ye on May 13 Peking University Authors: Haonan Dong , , , , , Abstract Autonomous agents exhibit distinct value systems from underlying language models, requiring…
…We evaluate 13 frontier models under a unified protocol. Key findings: The top model (Claude Opus 4.6) passes only 66.7% of tasks, and no model reaches 70%. Local workspace repair…
…Do VLMs Know When Not to Answer Spatial Questions (and Why)? Published on May 28 Submitted by Yue Zhang on Jun 1 Authors: , , , , , Abstract Vision-language models exhibit overconfidence in spatial reasoning…