… Across 19 frontier models, the best, Claude Opus 4.7, reaches only 62.2% overall under OpenClaw, while every other model stays below 60%, and switching harness alone shifts a single model by up to 18 points. …
… First, harness-updating is flat in base capability : models from different capability tiers produce harness updates that lead to surprisingly similar gains; even Qwen3.5-9B's updates yield gains comparable to those of Claude Opus~4.6. …
… We evaluate an empty-program floor, a classical computer-vision heuristic, Claude Opus 4.7 at high and max effort, and GPT-5.5 at medium and extra high reasoning effort. …
… On the Challenge subset, frontier models including Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 reach 30.4%, 26.4%, and 10.4% respectively, leaving substantial headroom, while leading open-weight models such as Qwen3-235B, GPT-OSS-120B, and Kimi-2.5 remain below 15%. …
… We also find that weaker agents in the loop can defend against much stronger hackers: Gemini 3 Flash's loop drives the stronger Gemini 3.1 Pro and Claude Opus 4.7's attack success rate from 76% and 61% to 0% on KernelBench , and Gemini 3.1 Pro's from 39% to 17% on Terminal Bench across 77 tasks. …
… Key findings: The top model Claude Opus 4.6 passes only 66.7% of tasks, and no model reaches 70%. …
… We evaluated three frontier VLMs GPT-5.5, Gemini 3.1 Pro, and Claude Opus 4.7 across 270 games in both homogeneous and cross-model adversarial settings. …