…Benchmarking Persistent Sycophancy in Stateful Personal Agents (2026) EvoClawBench: Can Agents Learn Reusable Skills from Their Own Runs? (2026) Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI…
Papers arxiv:2604.09408 HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help? Published on Apr 29 Submitted by Tu Trinh on May 5 Scale AI Authors…
…To address this gap, we present AutoMedBench, a workflow-aware benchmark for autonomous medical-AI research across diverse medical imaging and multimodal inference tasks, organizing agent execution into a unified five-stage…
…AI-generated summary LLM-based agents for GPU kernel generation are advancing rapidly, yet their progress is fundamentally constrained by the benchmarks they optimize against. Existing benchmarks are poorly aligned with production…
…A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents (2026) MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts (2026) Beyond Similarity: Trustworthy Memory Search for Personal AI…
…Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation (2026) Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification (2026) Agentic-MME: What Agentic Capability Really Brings…
…Benchmarking AI Models on Real-World Prediction Markets (2026) Herculean: An Agentic Benchmark for Financial Intelligence (2026) Agentick: A Unified Benchmark for General Sequential Decision-Making Agents (2026) Please give a thumbs…
…Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation Published on May 13 Submitted by JZPeterPan on May 14 Technical University of Munich Authors: , , , , , , , , , , Abstract RealICU benchmark evaluates…
…A Benchmark for Real-World, Long-Horizon Agent Evaluation (2026) OpenComputer: Verifiable Software Worlds for Computer-Use Agents (2026) AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks? (2026) SWE-Marathon…
…Generated by Qwen/Qwen2.5-Coder-32B-Instruct Current AI benchmarks evaluate agents on task execution within human-designed workflows. These evaluations fundamentally fail to measure a critical next-level capability: whether…
