Sherry 35B 0.1: controllable reasoning for code, analysis, and hard tasks

Today we are publishing Sherry 35B-A3B 0.1 GRPO Preview: a reasoning model built from Qwen/Qwen3.6-35B-A3B with one practical addition. You can tell it how much effort to spend on each request.

Sherry exposes three reasoning levels: low, medium, and high. That gives an application a way to trade answer quality, latency, and token cost without switching models or imposing the same hard thinking budget on every prompt.

The preview checkpoint is available on Hugging Face. This article explains what improved, where Sherry fits, and how we trained the behaviour.

What Sherry improves

We re-ran the base model and Sherry through the same internal harness: identical prompts, BF16 vLLM serving, greedy decoding, one generation per problem, and the same completion budgets. The results below compare the base model with Sherry at reasoning_effort=high.

Three results are clearly positive in this comparison. Sherry is more reliable on symbolic math under math_verify, performs better on difficult science and knowledge questions, and passes more of our basic Python tasks.

The efficiency result matters just as much as the accuracy result. On GPQA-Diamond, Sherry used 7.5k completion tokens on average versus 20.0k for the base model. It hit the 32,768-token budget on 3.5% of problems versus 27.8% for the base. The preview is not only answering more questions in this test; it is spending its reasoning budget more predictably.

There is also a result we do not want to hide, with an important qualification. Sherry scored 50.0% on AIME 2025 versus 63.3% for the base model, but we are reviewing whether the current AIME test or its answer-normalisation path contains an issue. Until that review is complete, the result is an unresolved signal, not a definitive regression. We still should not present this preview as a competition-math specialist.

The AIME comparison is under review because the evaluation test or answer normalisation may be incorrect.

Where Sherry fits

Sherry is a good fit when the application needs a model that can reason through a problem without making every request pay the maximum cost.

• Code generation and debugging. The Python suite improved from 75.0% to 97.5% in our controlled run. Use medium for normal implementation and high when a task needs deeper debugging or planning.
• Technical and scientific analysis. GPQA-Diamond improved by 21.2 points. This is the kind of work where a few extra reasoning steps can turn an incomplete answer into a useful one.
• Structured multi-step reasoning. MATH-500 reached 97.1% with math_verify, making Sherry useful for general quantitative work where the output can be checked.
• Budget-aware agents. An agent can use low for routing and simple transformations, medium for normal tool calls, and high only when the task justifies a larger reasoning budget.

The levels are a control surface, not a promise that every problem becomes easier. low is for short, focused responses. medium is the default starting point for harder work. high gives the model room to explore and is the setting used for the benchmark comparison.

How to use the control

The published chat template accepts a reasoning_effort argument. With an OpenAI-compatible deployment, pass the effort level through chat_template_kwargs:

1{
2  "model": "valendra/sherry-35b-a3b-0.1-grpo-preview",
3  "messages": [
4    {
5      "role": "user",
6      "content": "Review this function and find the edge cases."
7    }
8  ],
9  "chat_template_kwargs": {
10    "reasoning_effort": "medium"
11  }
12}
13

The interface is backed by three model-specific tokens:

1<|reasoning_effort_low|>
2<|reasoning_effort_medium|>
3<|reasoning_effort_high|>
4

They are emitted immediately before the thinking block. Omitting reasoning_effort preserves the base template behaviour; the effort control is opt-in.

How we built the training data

The effort labels came from a small self-distillation dataset that we published with the model: Sherry Reasoning Effort Dataset 0.1.

We sampled numeric-answer problems from NuminaMath-CoT and used Qwen3.6-35B-A3B-NVFP4-Fast through an OpenAI-compatible API to generate multiple candidate solutions for each problem. The generation setup encouraged different reasoning depths, including longer solutions for difficult problems.

For every problem, we checked the final answer against the ground truth. Incorrect or truncated candidates were discarded. Among the verified traces, the shortest became low, the median became medium, and the longest became high, based on observed thinking-token length. The labels describe measured reasoning depth; they are not three different prompts that the dataset assumes the model already understands.

The published dataset contains 2,061 verified rows. Each row includes the problem, source, thinking trace, thinking-token count, final response, predicted answer, ground truth, correctness, finish reason, and assigned effort level. It is intended for supervised fine-tuning and for studying controllable reasoning effort.

How we trained Sherry

The training had two stages.

1. SFT teaches the interface

We used 1,854 verified traces for supervised fine-tuning and held out 207 rows. SFT taught the model to respect the effort label, produce a reasoning trace at the corresponding depth, close the thinking block, and return a parseable final answer.

The intermediate Sherry 35B-A3B 0.1 SFT Preview is public as a PEFT LoRA adapter. It loads on top of Qwen/Qwen3.6-35B-A3B; it is not a standalone set of model weights. The GRPO preview below is the merged BF16 checkpoint used for the published comparison.

2. GRPO optimises the behaviour

We then ran GRPO for 200 steps with four generations per problem and effort level. The reward was deliberately correctness-first:

This keeps the model from earning reward by producing long but wrong explanations. It also makes the levels behave differently: low and medium are shaped toward useful budgets, while high is not punished for using the reasoning space a hard problem needs.

The final adapter uses LoRA/QLoRA with rank 32 and is merged into the published BF16 preview checkpoint.

Training cost: approximately US$200 of compute for this training run. This is an approximate cost for the run, not an inference price or a universal estimate for training another model.

Limitations we are carrying forward

• Preview release. Sherry 35B-A3B 0.1 is evaluated in one controlled internal run, not a leaderboard claim.
• Training data. NuminaMath-CoT may overlap with MATH-500 and AIME 2025, so treat those scores as an upper bound.
• AIME 2025. The comparison is under review because the evaluation test or answer normalisation may contain an issue. Until that review is complete, treat it as an unresolved signal and do not position this preview as a competition-math specialist.
• GPQA-Diamond. The evaluation uses a public mirror because the official dataset is gated.
• Comparison protocol. The base model is budget-limited in this protocol, and its published model-card results use different sampling settings. The table above is the relevant comparison because both models were run under the same harness.
• Reward. The reward value is a training signal, not an accuracy score.

Try Sherry

The Sherry 35B-A3B 0.1 GRPO Preview and its self-distillation dataset are public. For a model that can run at different reasoning depths behind an OpenAI-compatible interface, start with medium, move to high for harder analysis, and validate the outputs against your own workload.

Deploy Sherry inside QDivZero

Sherry can also be deployed inside QDivZero. Run the published Hugging Face checkpoint on managed Compute, expose it through an OpenAI-compatible endpoint, and keep reasoning_effort as a per-request control while QDivZero handles the infrastructure underneath.

Deploy Sherry on QDivZero Compute