Large language models (LLMs) have demonstrated impressive mathematical problem-solving capabilities. However, a significant gap exists between an LLM's ability to recite mathematical definitions and its capacity to effectively apply these concepts in novel problem-solving scenarios, a phenomenon we term the definition–application gap. To systematically investigate and address this gap, we leverage a classical mathematical textbook with clear associations between concepts and exercises, enabling both qualitative gap analysis and targeted training.
We introduce CORE (Concept-Oriented REinforcement), an RL-based framework with three training recipes: CORE-Base trains directly on concept-aligned quizzes; CORE-CR (Concept Replacement) dynamically replaces failed trajectories with concept-guided ones; and CORE-KL uses forward KL divergence to distill concept-primed reasoning into the base policy. Experiments across 11 math benchmarks show consistent improvements over strong baselines, with gains of up to +9.6% on TheoremQA and +9.3% on the in-domain Textbook test set.
Overview of the CORE framework. For a given query, the policy model generates multiple candidate solutions. If any is correct, CORE-Base proceeds with standard policy update. When all solutions fail, CORE activates concept-guided correction: the Concept Recall module retrieves relevant domain knowledge, and Concept Injection re-prompts the model. CORE-CR replaces failed trajectories with concept-primed ones, while CORE-KL distills the reasoning via forward KL loss.
CORE provides three complementary approaches to inject conceptual understanding into RL training:
Standard GRPO trained directly on 1,110 concept-aligned quizzes generated from the textbook. The data-level recipe: the model learns to implicitly connect concepts with problem-solving through rich concept-grounded training signals.
When all N responses fail, retrieves the relevant concept, re-prompts the model, and replaces failed trajectories with concept-primed ones. Adds a bonus reward r=0.4 to guide learning.
Instead of replacing trajectories, adds a forward KL divergence loss to align the base policy with concept-primed reasoning. Uses asymmetric weights: λ=0.03 for correct, λ=0.005 for incorrect.
CORE achieves consistent improvements across in-domain and out-of-domain math benchmarks.
| Method | Textbook | GSM8K | ASDiv | MAWPS | TabMWP | MATH | MMLU-STEM | Gaokao | CounterMath | TheoremQA | OlympiadBench |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Vanilla | 46.4 | 89.8 | 95.1 | 96.8 | 90.2 | 69.1 | 72.9 | 55.3 | 13.2 | 34.6 | 28.7 |
| SFT | 45.0 | 86.6 | 94.1 | 96.6 | 85.6 | 59.4 | 72.4 | 46.5 | 16.7 | 44.2 | 19.7 |
| CORE-Base | 50.7 | 90.8 | 95.4 | 97.2 | 92.6 | 71.1 | 72.9 | 59.5 | 13.5 | 40.4 | 33.9 |
| CORE-CR | 52.1 | 91.1 | 95.7 | 97.3 | 93.6 | 71.4 | 72.6 | 58.4 | 15.5 | 42.3 | 34.5 |
| CORE-KL | 55.7 | 90.7 | 95.5 | 97.5 | 90.6 | 70.5 | 73.1 | 59.5 | 15.8 | 44.2 | 32.9 |
All results under SC@21 (self-consistency with 21 samples, T=0.7). Bold = best in column.
We curate a corpus from Advanced Algebra (3rd Edition, Yao & Xie, 2015), a canonical Chinese textbook manually translated to English.
If you find this work useful, please cite our paper: