Trajectory‑Level Early‑Warning Signals in Autoregressive Reasoning
Simple internal metrics reveal brittle reasoning regimes long before surface failure.
Autoregressive language models can remain fluent while their internal dynamics drift into brittle regimes, especially during multi‑step reasoning. Existing evaluations focus on correctness or reward, which often miss early instability. This paper presents a controlled experiment showing that trajectory‑level internal statistics differ systematically by task type under identical decoding conditions.
Using Mistral‑7B with full‑stack instrumentation across 32 layers, we measure last‑token hidden‑state norms and cross‑layer variance at each generation step. Reasoning tasks exhibit stronger internal convergence and declining norms, while explanatory tasks remain stable. Under depth pressure, the reasoning trajectory enters an incorrect attractor early and degrades over time, even though surface fluency persists.
These results show that simple, low‑overhead internal metrics can act as early‑warning signals, flagging brittle reasoning regimes before or alongside visible failure. The method is reproducible, inexpensive, and architecture‑agnostic, providing a practical diagnostic layer for autoregressive systems.
Index
1. Abstract
Internal trajectory statistics differ by task type; reasoning shows early collapse signatures.
2. Problem
Fluency hides internal drift; surface correctness misses early instability; need for trajectory‑level signals.
3. Method
Model: Mistral‑7B‑Instruct
Decoding: greedy + sampling
Depth: matched 96‑token runs
Instrumentation: full 32‑layer stack
Metrics: layer norms, cross‑layer variance, slope ΔVar
4. Tasks
Reasoning: bat‑and‑ball problem
Explainer: superconductivity explanation
5. Results
Reasoning task shows:
declining norms
strong variance collapse
negative slope
early wrong attractor
staged degradation
Explainer remains coherent.
6. Interpretation
Internal convergence = brittleness; stable dispersion = coherent reasoning substrate.
7. Robustness
Sampling preserves the trend; surface oscillations do not remove the collapse signature.
8. What This Shows
task sensitivity
depth sensitivity
correlation with failure
reproducibility
9. What This Does Not Claim
No universal thresholds; no failure‑time prediction; no cross‑model generalization yet.
10. Why This Matters
Trajectory‑level metrics provide early‑warning signals with minimal overhead.
11. Next Steps
Lead‑time quantification, model sweep, task suite expansion, variance‑vs‑step visualization.
12. Reproducibility Notes
Fixed seed, KV cache disabled, float32 metrics, small .npz artifacts.