Ch. 16 — Open Weights, Code, and Data

Chapter 16 of DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Abstract

This chapter establishes the technical frameworks for evaluating reasoning capabilities and factual correctness in open-weight language models, specifically within the context of the DeepSeek-R1 lineage. It outlines rigorous protocols for measuring performance on Chinese factual queries via C-SimpleQA and complex mathematical reasoning through standardized benchmarks like AIME and MATH. The central contribution is the definition of verifiable grading logic, utilizing rule-based parsers such as SymPy to assess final answers against ground truth conditions. By integrating open-source reproducibility references, this section underscores the transition from proprietary benchmarks to collaborative evaluation standards in the field of reinforcement learning from human feedback.

Key Concepts

• C-SimpleQA Evaluation Protocol: This concept defines the metric for assessing a model’s ability to answer short, fact-seeking questions in Chinese with precise, verifiable correctness. It establishes three distinct output categories for grading: Correct, Incorrect, or Not Attempted. The protocol mandates that models must provide the exact factual answer without hallucination, distinguishing between a failure to answer and an incorrect attempt.
• $b$-eautiful Integer Definition: A positive integer $n$ is defined as $b$-eautiful if it possesses exactly two digits when expressed in base $b$, and the sum of these digits equals the square root of $n$. This mathematical construct serves as a specific constraint for testing the reasoning capabilities of models in base conversion arithmetic. It requires the model to solve for the least integer $b\ge 2$ satisfying specific digit sum properties.
• Rule-Based Ground Truth Parsing: Evaluation logic relies on parsing the model’s output to extract the final answer enclosed within specific delimiters, specifically the ​ tag. This method mitigates the ambiguity of natural language responses by enforcing a structured format for computational verification. The system utilizes SymPy to parse numerical expressions and determine equality with the ground truth.
• Open Reproducibility Standards: The chapter highlights the availability of fully open reproductions of reasoning models, such as the “Open R1” project by H Face. This initiative provides access to code and weights, enabling the community to verify results and contribute to the development of reasoning architectures. It references specific model cards like Llama 3.1 and technical reports from DeepSeek-AI as benchmark standards.
• Distinction of Failure Modes: The evaluation framework explicitly differentiates between an “Incorrect” response and a “Not Attempted” response. An incorrect response implies the model generated a plausible but factually wrong answer, whereas “Not Attempted” indicates the model requested more context or refused to answer. This distinction is critical for measuring recall versus accuracy in safety-aligned systems.
• Chinese Language Factuality: C-SimpleQA specifically targets Chinese language factuality, addressing a gap in multilingual evaluation benchmarks. It requires the system to handle queries such as “显脉香茶菜可以用来治疗急性的什么类型的黄疸型肝炎？” with precise retrieval capabilities. This ensures that open-weight models perform equitably across high-resource and low-resource language domains.
• Multi-Task Reasoning Benchmarks: The text references a suite of mathematical benchmarks including AIME, MATH, and CNMO (Chinese National Mathematics Olympiad). These tasks collectively evaluate the model’s performance on abstract reasoning and mathematical problem solving rather than simple text completion. They serve as the primary quantitative metrics for the reasoning capability incentivized by reinforcement learning.
• Literature and Reference Ecosystem: The chapter integrates a comprehensive list of references spanning from foundational reinforcement learning (PPO, Human Feedback) to recent scaling laws and test-time training methods. These citations provide the theoretical underpinning for the evaluation methods discussed, linking current practices to works by authors such as Brown, Hendrycks, and Bai.

Key Equations and Algorithms

• Base Conversion Condition: The definition of a $b$-eautiful integer $n$ requires $n$ to be expressible as $d_{1}b+d_0$ where $d_1,d_0$ are the digits in base $b$. The condition is formally $d_1+d_0=\sqrt{n}$, imposing a constraint on both the representation and the numerical value of the integer. This equation governs the search space for the mathematical evaluation tasks.
• Digit Sum Constraint: For a number to satisfy the $b$-eautiful property, the sum of its digits must equal the arithmetic square root of the number itself. This relationship is expressed as $\sum d_i=\sqrt{n}$ where the sum is over the digits of $n$ in base $b$. The algorithm iterates through integer values to identify which base $b$ yields more than ten such integers.
• SymPy Parsing Logic: The evaluation algorithm uses the SymPy library to parse the mathematical expression extracted from the model’s ​ tag. The logic is represented as parse_expr(answer) compared against ground_truth with rounding applied for numerical values. This ensures that equivalent mathematical forms (e.g., $1+1$ vs $2$) are recognized as correct.
• Evaluation Grading Function: The grading procedure maps the comparison result to a discrete set of labels $\{A,B,C\}$. If the parsed answer equals the ground truth, the function returns A (Correct); if it differs, it returns B (Incorrect); if the model refuses to answer, it returns C (Not Attempted). This logic is implemented as a deterministic rule set rather than a learned classifier to ensure consistency.

Key Claims and Findings

• C-SimpleQA provides a precise mechanism for verifying factual correctness in Chinese queries independent of generative fluency.
• Mathematical evaluation benchmarks require strict adherence to answer formatting, specifically the extraction of values within ​ tags.
• The $b$-eautiful integer problem serves as a scalable test for base conversion logic and arithmetic reasoning within large language models.
• Open-source reproduction efforts, such as “Open R1,” are critical for validating the reasoning capabilities of models like DeepSeek-R1.
• The grading logic must explicitly penalize hallucinations (Incorrect) differently from refusal (Not Attempted) to accurately measure model confidence.
• References to Llama 3.1 and DeepSeek-V2 indicate the competitive landscape and architectural reliance on mixture-of-experts designs.
• Test-time training and latent reasoning scaling are emerging methods referenced as potential improvements to the evaluated capabilities.
• Fact-checking requires a structured evaluation pipeline that distinguishes between retrieval failure and generation errors.

Terminology

• C-SimpleQA: A benchmark dataset designed to measure a model’s ability to answer short, fact-seeking questions in Chinese with precise correctness. It utilizes a standardized prompt format to elicit Yes/No or specific factual entities.
• $b$-eautiful Number: A mathematical term defined in the context of Table 32, referring to a positive integer $n$ that has exactly two digits in base $b$ with digits summing to $\sqrt{n}$.
• Ground Truth: The reference standard used for evaluation, against which the model’s prediction is compared to determine correctness. In math tasks, this is the specific numerical value or expression.
• SymPy: An open-source Python library for symbolic mathematics used in the evaluation pipeline to parse and compare mathematical expressions automatically.
• Not Attempted: A specific evaluation category (C) assigned when a model responds with uncertainty, requests context, or explicitly states an inability to answer.
• Test-Time Training: A reference concept in the provided bibliography (Akyürek et al.) describing methods where models adapt their parameters during inference for abstract reasoning.
• Mixture-of-Experts (MoE): A model architecture design referenced in the DeepSeek-V2 technical report, indicating the structural efficiency of certain evaluated language models.
• ​: A LaTeX formatting convention used to enclose the final answer in mathematical reasoning tasks, serving as the extraction anchor for the rule-based grader.