Reasoning in AI: A Learnable Rule-Based Process

In the vast and often nebulous world of artificial intelligence, few terms are thrown around as frequently, yet understood as vaguely, as "reasoning." Everyone talks about it, but pinning down what it actually means in an operational sense has been a persistent challenge. A recent position paper, authored by Rachel Lawrence and Jacqueline Maasch, aims to cut through this confusion. Titled "Position: Reasoning is a Learnable Rule-Based Process," and accepted for ICML 2026, the paper offers a much-needed operational definition and a practical checklist for writing and evaluating reasoning research.

Bridging the Historical Divide in AI Reasoning

The paper immediately highlights a subtle but significant historical disconnect. Reasoning was once the exclusive domain of symbolic AI, where explicit rules and logical structures reigned supreme. Fast forward to today, and deep probabilistic generative models dominate the landscape. Yet, the two traditions have never truly aligned on a common definition of "reasoning." The generative AI community, in particular, often sidesteps the rich history of logic and verifiable automated reasoning, leading to a babel of interpretations. What one paper calls "reasoning capability," another might interpret entirely differently, and a third might use it as a marketing buzzword.

This ambiguity isn't just academic nitpicking. The authors point out that a lack of clear definition cripples the construct validity of reasoning evaluations. In simpler terms, if you claim to be measuring reasoning, but your tests are actually just assessing pattern matching or rote memorization, your results are fundamentally flawed. This makes it incredibly difficult to quantify progress, compare different models accurately, or build trust in AI systems that claim to reason. As the paper's abstract succinctly puts it:

"Ambiguity invalidates reasoning evaluation and undermines quantitative progress towards trustworthy autonomous reasoning."

Redefining Reasoning: A Learnable Rule-Based Process

The good news, according to Lawrence and Maasch, is that this definitional quagmire is solvable. Drawing from extensive literature, they propose that effective and sound reasoning can be understood as a learnable rule-based process. The core of this definition lies in its dual emphasis: it insists on the logical notions of "validity" and "soundness" as central criteria, while simultaneously acknowledging that these rules can be learned from data, rather than being exclusively hard-coded through manual symbolic systems.

This isn't a call for a full return to symbolic AI, nor is it a simple concession to neural networks. Instead, it's a pragmatic attempt at reconciliation: rules provide a verifiable, logical backbone, and learning offers the pathway to acquire these rules. For researchers grappling with the persistent problem of "pseudo-reasoning" in large language models (LLMs), this framework offers a compelling alternative to simply hoping for emergent intelligence.

Practical Takeaways for the Research Community

Beyond the definition, the paper provides a crucial best practices checklist designed to improve how AI reasoning research is communicated and evaluated. While the specific details aren't fully elaborated in the abstract, such a list typically covers points like: explicitly stating which concept of reasoning is being used, clearly linking evaluation tasks to the chosen definition, avoiding metaphorical language (e.g., "the model thinks"), and transparently reporting uncertainties and failure cases. For anyone regularly writing or reviewing papers on AI reasoning, this checklist could be an invaluable, actionable tool.

The practical impact extends beyond academia. Any team relying on AI evaluation reports to make decisions—say, comparing different models' reasoning capabilities for a specific application—will find this paper highly relevant. It serves as a vital reminder: a "reasoning score" without a rigorous, agreed-upon definition behind it should be viewed with a healthy dose of skepticism.

A Catalyst for Discussion, Not a Final Verdict

As a position paper, its primary goal isn't to deliver the ultimate answer, but rather to spark critical discussion and establish better research norms. Whether the proposed definition gains widespread acceptance remains to be seen, and will undoubtedly require further work and validation from the community. However, it undeniably provides a much-needed common ground for a conversation that has, for too long, been fragmented and ill-defined.

If you're involved in reasoning-related evaluations or research, consider this paper an opportunity for introspection: Are my experiments truly measuring reasoning? Is the gap between my chosen metrics and the term "reasoning" defensible under scrutiny? This is a chance to elevate the discourse and build more trustworthy AI systems.