3 min read
Modes of Discovery
Learning is symbiotic

Neural networks or symbolic systems. Choose a side, run your experiments. Forty years, and no clear winner. It’s not because “both sides have merit”, a polite thing people say to avoid engaging, but because both sides are willfully blind to the other.

Gradient descent moves through configurations looking for invariants, the stable patterns that hold across variation in the data. Edges, frequencies, objects, structures. There’s a loss function, but no motivation. A neural network doesn’t know what it’s looking for, it doesn’t care, and that’s the point. It finds structure that was invisible before training began.

But finding structure and using structure, these are different operations. Once the invariants are uncovered, reasoning over them, chaining them, applying them to novel situations, this is a traversal problem.

Symbolic methods work best at navigating structure. They operate over discrete, relational structures. They chain. They compose. Explanation is the output of the steps they take. What these systems have never been good at is generating structure from experience. The original symbolic AI program failed not because symbolic architecture is ineffective, but because hand-engineering the structure that a neural network discovers automatically doesn’t scale. The symbols were fine, the discovery method wasn’t.

The connectionist says: learning is gradient descent. The symbolist says: learning is symbol manipulation. And this is where they’re both right, but only when you understand both are different descriptions of search. Both are trial and error, operating in different spaces, at different phases, with different stakes. The debate is between two groups, each solving half the problem, each claiming the solution to all of it.

It’s assuming a single architecture for discovery and traversal is that gets you get neural networks that can’t reason compositionally and symbolic systems that can’t learn from data. The compelling question was never which wins. It’s what happens when discovery and traversal are interdependent, designed to work as a single system from the start.

Two modes of search, one complete system.