for(geeks)
AI

• 4 min read

Reasoning effort is now a model feature

Modern models now ship with multiple reasoning modes. This walkthrough explains how RLVR, inference scaling, and simple training tricks make that possible.

Image: Build A Reasoning Model (From Scratch)

Nearly two years after OpenAI introduced o1, reasoning models have gone from a curiosity to a standard feature. DeepSeek-R1 followed about four months later and helped popularize reinforcement learning with verifiable rewards (RLVR) as a way to train them. Last week, OpenAI released the GPT-5.6 family with three sizes and roughly five or six reasoning-effort settings per model.

Figure 1: The GPT 5.6 Sol model with different reasoning effort settings. (Benchmark numbers for Ultra are currently not available but should be relatively similar to Max, since it uses a similar effort level but accelerates the work with four subagents.)
Figure 1: The GPT 5.6 Sol model with different reasoning effort settings. (Benchmark numbers for Ultra are currently not available but should be relatively similar to Max, since it uses a similar effort level but accelerates the work with four subagents.)

In Sebastian Raschka’s account, the key shift is not just that models can “reason,” but that they can do so at different effort levels. In LLM research, a reasoning model does not mean a system that reasons like a human. It means a model that produces an intermediate reasoning trace—a step-by-step internal response before the final answer.

Figure 3: Illustration of a conventional LLM answer (left) and an answer by a reasoning model (right).
Figure 3: Illustration of a conventional LLM answer (left) and an answer by a reasoning model (right).

There are two broad ways to improve performance: training scaling and inference scaling.

Figure 4: Training and inference scaling are two ways to improve LLM and reasoning model problem-solving capabilities. Plot based on Learning to reason with LLMs.
Figure 4: Training and inference scaling are two ways to improve LLM and reasoning model problem-solving capabilities. Plot based on Learning to reason with LLMs.

How RLVR trains reasoning models

The training side is now familiar. DeepSeek-R1 used RLVR to reward correct answers in domains where correctness can be checked automatically, such as math and code. The reward signal is simple: 0 = incorrect and 1 = correct, with tools like SymPy, WolframAlpha, compilers, unit tests, or platforms like LeetCode used for verification.

Recommended reading

AI Won’t Replace Coders, but It Will Reshape the Job

Figure 5: Illustration of accuracy and format rewards during RLVR training.
Figure 5: Illustration of accuracy and format rewards during RLVR training.

A notable detail: the reasoning trace itself is not what trains the model. During RLVR, the model is rewarded based on the final answer and the response format, while the intermediate trace is ignored.

Even so, that setup is enough for models to learn behaviors associated with reasoning: writing intermediate explanations, backtracking, and correcting mistakes. Those self-corrections are often described as “Aha” moments.

Figure 7: An example of an aha moment, where a reasoning model notices an error in its intermediate reasoning and corrects it before producing the final answer.
Figure 7: An example of an aha moment, where a reasoning model notices an error in its intermediate reasoning and corrects it before producing the final answer.

Raschka also notes that Kimi K1.5 appeared on arXiv the same day as DeepSeek-R1—22 Jan 2025—and that the term RLVR had already been coined two months earlier in Tülu 3: Pushing Frontiers in Open Language Model Post-Training. DeepSeek’s paper still became the breakout reference partly because it showed reasoning behavior could emerge from pure RL.

In practice, though, full training pipelines are usually more complicated. Tülu 3 and Kimi K1.5 applied reinforcement learning on top of a supervised fine-tuned (SFT) model. DeepSeek-R1 itself was trained from an SFT checkpoint of the DeepSeek-V3 base model, alongside an R1-Zero variant trained with pure RLVR. R1-Zero was weaker than R1, but it served as a proof of concept that RLVR alone can teach a model to generate and use reasoning traces.

Figure 9: More detailed reasoning model training pipeline. This one depicts the various DeepSeek-R1 models. For more details, see my other article: Understanding Reasoning LLMs.
Figure 9: More detailed reasoning model training pipeline. This one depicts the various DeepSeek-R1 models. For more details, see my other article: Understanding Reasoning LLMs.

Inference scaling and reasoning-mode toggles

Training is only half the story. The other lever is inference compute scaling: spending more compute at runtime to improve answers. Reasoning models already do some of this implicitly because they generate more tokens than conventional LLMs. Effort levels push that further by controlling output length and compute budget.

A common additional technique is self-consistency, where the model is queried multiple times and the final answer is chosen by majority vote.

Figure 10: An example of self-consistency, a popular inference scaling technique.
Figure 10: An example of self-consistency, a popular inference scaling technique.

Raschka points to DeepSeekMath-V2 as an example of taking this to an extreme, layering aggressive inference scaling on top of a reasoning model specialized for math to reach state-of-the-art performance on difficult olympiad-style problems.

He also spends time on a common misconception: <think></think> tokens are not what make a model reason.

Figure 12: Common formatting tokens in reasoning models.
Figure 12: Common formatting tokens in reasoning models.

Those tags are mainly formatting markers, used to separate the reasoning trace from the final answer so the training pipeline or user interface can hide it. They can be reinforced with a formatting reward during RLVR, as in DeepSeek-R1, where:

R_total = R_accuracy + R_format

The first generation of reasoning systems were dedicated models like DeepSeek-R1, which tended to produce long, verbose responses for almost any prompt.

Figure 13: Reasoning models are very verbose, even for the simplest prompts.
Figure 13: Reasoning models are very verbose, even for the simplest prompts.

Newer systems such as Qwen3 moved toward hybrid behavior, letting the same model act either as a standard instruction-tuned model or a reasoning model on demand. In Qwen3, that switch is exposed through the tokenizer as enable_thinking=True or enable_thinking=False. With enable_thinking=False, the system effectively injects an empty <think></think> block at the start of the assistant response to disable reasoning mode.

Figure 14: Response of Qwen3 0.6B reasoning model with thinking=False and thinking=True. (The empty <think></think> tags are hidden in the interface on the left as they are part of the modified input prompt, not the generated answer.)
Figure 14: Response of Qwen3 0.6B reasoning model with thinking=False and thinking=True. (The empty <think></think> tags are hidden in the interface on the left as they are part of the modified input prompt, not the generated answer.)

According to the Qwen3 technical report, that toggle is introduced mainly through SFT and then reinforced during general RL in its largest flagship models. After initial long-chain-of-thought SFT and reasoning RL, Qwen adds a Thinking Mode Fusion stage where the model sees both formats:

  • /think: <think>{reasoning}</think>{answer}
  • /no_think: <think></think>{answer}

Thinking remains the default behavior, so /think can be omitted. The result is a model that can change its reasoning effort at inference time—a capability that is quickly becoming a default expectation in frontier releases.

Ava Chen

AI Editor

Ava covers the rapidly evolving world of artificial intelligence, from foundational models and research labs to the real-world economics of intelligence. With a background in computational linguistics, she cuts through the hype to find out what actually works. She firmly believes that benchmarks are just marketing until reproduced in the wild.

via Hacker News

// Keep reading