The question of whether large language models can handle complex, spatial reasoning tasks has moved from theoretical debate to empirical leaderboard. Portal2Bench, a new benchmark hosted by Kingston Yip, measures the ability of AI agents to solve specific test chambers from the video game Portal 2. The results, posted on Hacker News on November 14, 2026, show that GPT-6.1 Sol, running on the Codex harness, successfully solved 20 out of 26 available chambers. This performance ties with GPT-6 Astra, also on Codex, while GPT-6.1 Terra solved 19 chambers. The benchmark is not affiliated with Valve, the game's developer, but serves as a rigorous test of embodied reasoning in a controlled digital environment.
The Cost of Intelligence
While the success rates are impressive, the financial and computational costs required to achieve them are substantial. Each chamber run is capped at a budget of $30 and 300 'plans' (agent actions). The median cost for a solved chamber varies significantly by model. GPT-6.1 Terra operates most efficiently with a median cost of $0.74 per chamber, while GPT-6.1 Sol averages $1.00. In contrast, Anthropic’s Opus 5.5, running on the Claude Code harness, incurred a median cost of $10.60 per chamber. Fable 5.1 was the most expensive, hitting a median of $13.12. These figures highlight a clear trade-off between raw reasoning capability and economic efficiency in current agent architectures.
Methodology and Constraints
Portal2Bench enforces strict constraints to prevent agents from cheating through external knowledge or shell access. The agent interacts with the game by sending keyboard inputs and view turns, receiving only screenshots and positional data in return. It has access to a local notes folder but lacks web browsing, console, or shell capabilities. A QA check ensures that no external tools are used. If an agent dies, it respawns at the last checkpoint, mimicking normal gameplay, and is informed only of its death. This setup isolates the agent’s ability to reason about spatial relationships and physics using only visual inputs and internal memory.
Leaderboard Performance
The current leaderboard, updated with results from November 9, 2026, shows a dominance by OpenAI’s GPT-6 series when paired with the Codex harness. GPT-6.1 Sol and GPT-6 Astra lead with 20/26 chambers solved. Anthropic’s models trail significantly: Opus 5.5 solved 11 chambers, Sonnet 5.5 solved 6, and Fable 5.1 managed only 3 out of 15 attempted. GPT-6 Luna, however, failed to solve any chambers (0/26), suggesting that not all variants in the GPT-6 family possess equal spatial reasoning capabilities. Gemini 4 Argon and Fable 5.2 are listed but have not yet completed their runs due to compute constraints.
Key Takeaways
- GPT-6.1 Sol and GPT-6 Astra (Codex) tie for first place with 20/26 chambers solved.
- Median cost per chamber ranges from $0.74 (GPT-6.1 Terra) to $13.12 (Fable 5.1).
- Anthropic’s Opus 5.5 solved 11 chambers, significantly fewer than the top GPT-6 models.
- Agents are restricted to visual inputs and keyboard commands; no shell or web access is permitted.
- The benchmark is independent of Valve and focuses on Chapters 2 through 5 of Portal 2.
The Bottom Line
Current frontier models can navigate complex spatial puzzles, but the economic viability of such reasoning remains a major hurdle for widespread agent adoption.