This concluding chapter explores how BDH could move AI beyond the Transformer era, enabling models that remember, adapt, and reason at a fraction of today’s inference cost.

What BDH establishes

BDH brings identifiable neurons and synapses, persistent network-level memory, local computation, and GPU-compatible sparse dynamics into a single construction. The design is grounded in formal equations with inspirations from the brain and intuitions from physical processes, and its empirical behavior is validated with several experiments on language tasks.

Rewriting attention as a linear operation in a high-dimensional, sparse, non-negative neuron space turns the memory of the model into an evolving matrix of neuron-to-neuron connections, updated by sparse outer products that take the form of Hebbian writes. This gives us a graph view of the model, in which neurons carry fast transient activity, synapses carry slower contextual state, and every interaction is local. Low-rank factorization with thresholding then connects that conceptual graph to the dense tensor operations GPUs execute efficiently, so the graph is never materialized in memory and still remains recoverable for analysis.

Further empirical analyses show that BDH masters language-based abilities while extending beyond language: it solves constraint-based tasks such as Sudoku through iterative latent reasoning and retains relevant information across long-context tasks such as BABILong. Its recurrent memory continuously integrates new context, while a high-dimensional latent workspace can preserve and refine multiple hypotheses without verbalizing every intermediate step, enabling flexible, efficient reasoning without routing all computation through tokens.

There is a further consequence that matters more than we initially expected. Unlike other LLMs available today, BDH scales like a population of cooperating neurons rather than through globally coordinated computation. Its most remarkable effect is locality: important data sits close to where it is processed. This minimizes communication and alleviates one of the main bottlenecks for reasoning models during inference: memory-to-core bandwidth. It therefore opens the door to very high inference speeds on new generations of AI hardware, from dedicated chips to next-generation GPUs.

At Pathway, we’re building the next generations of BDH models in order to unlock new features and use cases that are out of reach for current LLMs, at a fraction of the cost. Here are our concluding thoughts:

• Transformers are a bad starting point to tackle the continual learning and reasoning challenges. Long-term memory and adaptation won’t come from patchy markdown notes that increase token consumption and governance burden.
• BDH is a beautiful and efficient model for moving past the Transformer era. It is built for memory, continual learning, and reasoning to emerge from efficient brain-like local dynamics rather than from engineering patches.

Further reading

• The Dragon Hatchling: The Missing Link between the Transformer and Models of the Brain, the original BDH research paper.
• BDH on GitHub, the reference implementation accompanying the paper, including a minimal training example.
• The Sudoku Test: Why We Solved the Puzzle That Still Stumps LLMs.
• The Equations of Reasoning, a deep dive into how and why BDH works.