AA-AgentPerf is a hardware benchmark created by Artificial Analysis that measures the number of concurrent AI agents an inference system can support while meeting predefined, model-specific performance service level objective (SLO) tiers. An SLO is defined as a specific threshold of output token speed and time-to-first-token (TTFT). The benchmark results are normalized per accelerator and per megawatt to enable comparison across hardware configurations.
NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark | NVIDIA Technical Blog…Benchmarks as User-Driven Long-Horizon Coding Sessions Published on Jun 29 Submitted by Anisha Gunjal on Jul 1 Scale AI Authors: , , , Abstract SWE-Interact presents a testbed that evaluates coding agents…
…When Students Set Challenges for AI Agents Published on May 4 Submitted by taesiri on May 5 Authors: Junjie Yu , , , , , , , , , , , , , , , , , , , , , Abstract AcademiClaw presents a comprehensive benchmark for evaluating AI agents on complex…
…The company claims MAI-Cyber-1-Flash is significantly more powerful (and more cost-effective) than competitor models, based on its performance on an established AI cybersecurity benchmark. “We’re very very…
…Some AI models will accidentally learn that cheating is one way to “win.” More recently, researchers have developing benchmarks for reward hacking in coding agents and for unpredictable behavior in customer support…
Benchmarking AI agent retrieval strategies on Kubernetes bug fixes
submitted by /u/ZealousidealHunter80 to r/cybersecurity [link] [comments]
Someone modelled sugarcane farming as an integer program. See, sugarcane only grows next to water. Water costs one tile and can feed at most four cane tiles. The layout therefore becomes a coverage problem with a genuine…
Hi HN! Sina here. i wanted to share this open source project (apache 2.0) that i've been working on for the past month or so. it's called hotcell and it lets you create/pause/manage sandboxes on any device (your laptop, …
I built ZaGuu, an arena where AI agents play strategy games, negotiate, cooperate, betray, and build public records.The first game is Bank Heist, a Split-or-Steal style game where agents negotiate before making a final d…
…July 23, 2026 Benchmarking AI Systems: from Model Metrics to Real-World Performance The agentic AI stack has evolved to fast multi-model orchestration, tool-augmented reasoning, and long-running inference chains…
…Enterprise AI Agents (2026) SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution (2026) Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent (2026) Herculean: An Agentic Benchmark for…
Papers arxiv:2606.05275 Personal AI Agent for Camera Roll VQA Published on Jun 3 Submitted by Thao Nguyen on Jun 5 Authors: , , , , Abstract A conversational AI agent is developed for personal…
…Evaluating Agent Development Kits via LLM-as-a-Developer (2026) An Executable Benchmarking Suite for Tool-Using Agents (2026) SWE-Bench 5G: Benchmarking AI Coding Agents on Telecom Network Engineering Tasks (2026…
…Leading the Next Era of GenAI Benchmarking Mar 19, 2026 How AMD is helping advance open, community-driven benchmarking for real-world generative AI services The landscape of generative AI performance evaluation…
Last week, Hugging Face reported that an autonomous AI agent had compromised parts of its infrastructure by exploiting vulnerabilities in its dataset-processing pipeline. The agent accessed a limited number of internal…
