Today, we're publishing a new benchmark result for BDH-CQ, a 150-million-parameter reasoning model built on Pathway's BDH architecture.
BDH-CQ scored 29.4% pass@2 on the public ARC-AGI-1 evaluation set at a computed inference cost of $0.00070 per task. No previously reported system reached the same accuracy at an equal or lower cost. GPT Luna 5.6 (Low) scores 34.2% while costing 57 times more.
An independent black-box evaluation of the deployed system reproduced the results on public ARC-AGI-1. The report is public and linked in the full technical report. If you are also interested in independently validating the result through API, reach out to us.
What ARC-AGI measures
ARC-AGI presents a system with a handful of before-and-after examples of a visual rule, then asks it to apply that rule to a new input. The rule changes with every task and must be inferred entirely from the examples.
The benchmark is designed to test not only whether a system possesses a skill, but how efficiently it can acquire and apply a new one.
BDH-CQ is specialized for this kind of visual reasoning. While the commercial models on the leaderboard are general-purpose systems, the result matters because meaningful reasoning performance was achieved with a small fraction of the inference compute used by leading reasoning systems.
A different approach to reasoning
Transformers have unlocked extraordinary capabilities and are now powering the transition from chatbots to agents that can reason, use tools, write code, and act across increasingly complex workflows.
However, we are also becoming accustomed to a particular shape of progress: larger context windows, longer reasoning traces, more elaborate agent harnesses, enormous compute budgets, and vast amounts of carefully developed training data and environments.
These approaches will continue to produce remarkable systems. But foundational improvements remain possible in how models remember, adapt, and perform computation.
Many current reasoning systems allocate additional inference compute by generating longer sequences of intermediate tokens. BDH-CQ instead performs iterative computation in a recurrent latent state and decodes only its candidate answers. Examples presented at inference time update the model's memory as they are processed. It can then work through a new problem without generating a long verbalized reasoning trace, modifying its parameters, or retraining for that task.
This result demonstrates that architecture can be another important lever alongside model scale, data, and inference compute.
Beyond visual puzzles
The tasks in this paper are compact colored grids. The underlying challenge is much broader: take in new information, preserve the state that matters, reason across interacting constraints, and adapt coherently when something changes.
Consider planning a complex trip across several countries. Flights, visas, budgets, weather, opening hours, and personal preferences all constrain one another. When a flight is cancelled or a plan changes, a capable system should understand what has been affected, preserve what remains valid, and coherently rebuild the rest.
Handling that well is not simply a retrieval problem. Having every relevant document and historical change inside a context window is not enough. The system must maintain an accurate understanding of the situation and update it as the situation evolves.
The same underlying problem appears in investigating a cyberattack, coordinating a transportation network, managing a scientific research program, or operating an autonomous agent across weeks of work. New forms of memory and computation could eventually unlock categories of utility that remain too costly, slow, or unreliable today.
What comes next
We are early in the journey toward another phase of machine intelligence.
Next, we will scale BDH-CQ, extend the approach to ARC-AGI-2 and more complex constraint-satisfaction problems, and bring it to more general language and mathematical reasoning tasks.
We will continue publishing what we learn, including new capabilities, failures, and open questions. To stay up to date with our research, follow Pathway.
We will also move beyond benchmarks. We are beginning to work with design partners to identify applications where persistent state, latent reasoning, and low inference cost could create entirely new forms of utility.
If you are working on a problem that requires reasoning across large amounts of changing information, interacting constraints, or long-running workflows, we would like to hear from you. Reach out.