AI begins challenging real-world engineering design: GPT-6 Astra posts a 37% pass rate, while five other models failed to earn a single pass.

Beating AI News Brief: AI startup Xplore has launched RWE-Bench, a benchmark to test large language models (LLMs)’ ability to solve real-world engineering problems. 11 models were evaluated on 20 tasks spanning mechanical, electrical, and aerospace engineering. GPT-6 Astra ranked first with a 37% single-attempt average pass rate, followed by Claude Opus 5.5 at 29% and GPT-6.1 Sol at 25%. Five models, including DeepSeek V4.1 Flash, GLM-5.3, and Grok 4.7, failed all tasks. The tasks covered engine crankshaft design, circuit board routing, and turbine blade thermal simulation: crankshaft design requires balancing size, weight, strength, and fatigue life; routing an 8-layer circuit board with nearly 300 installed components demands connecting 71 signal lines while controlling timing errors and impedance; turbine blade thermal simulation needs accurate calculation of heat transfer between high-temperature airflow and the blade. Each model used its respective Agent toolchain to operate professional engineering software directly. Each test ran for a maximum of 24 hours, with 5 independent attempts per task. Submitted designs were automatically validated via engineering software and physical solvers; any unmet hard requirement resulted in a failure. Out of 1,100 total tests, 4 tasks remained unsolved. However, after allowing 5 attempts per task, Claude Opus 5.5’s task pass rate rose to 70%, overtaking GPT-6 Astra’s 60%. Xplore aims to turn such engineering tasks into reinforcement learning training environments, enabling AI to improve engineering capabilities through repeated design, simulation, and error correction.