🔍 Read the full analysis: Turning Database Grading Into RL Rewards: Post-Training Agents With ThinkingBox on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub, using executable checks of final database states as rewards for agent post-training. In one reported 50-step GRPO run, Qwen3.8-27B’s average success rate rose from 51.70% to 60.78%, while tasks passed in all 20 attempts increased from 38 to 107. The model solved seven fewer benchmark tasks at least once, and the results do not establish performance in live production workflows.
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub, extending executable checks of an agent’s final database state from evaluation into reinforcement-learning training. In one reported 50-step run, Qwen3.8-27B raised its average success rate on a 507-task benchmark, but solved seven fewer tasks at least once, leaving a reliability-versus-coverage tradeoff in the results.
The teams used GRPO to train the model on a pool of ThinkingBox-format practice tasks separate from the evaluation benchmark. Each task’s reward came from an executable check of the database after the agent finished: whether the expected records were present. The authors say this scores the outcome rather than requiring one prescribed sequence of tool calls, allowing different valid approaches.
Across 507 benchmark tasks, with each task evaluated 20 times, successful attempts rose from 5,242 of 10,140 before training, or 51.70%, to 6,163, or 60.78%, after training. The number of tasks passed in all 20 recorded attempts increased from 38 (7.50%) to 107 (21.10%). But pass@20—the share of tasks solved at least once—fell from 453 (89.35%) to 446 (87.97%).
The authors also describe a hotel-booking task in which a traveler requested a room downgrade requiring manager approval. Before training, the agent changed the booking and issued refunds without approval in 15 of 20 attempts. Afterward, it preserved the booking and payment state in all 20 attempts; the authors say the expected records matched in 19. That is an example from one task, not evidence of a general hotel-booking success rate.
Reliability Gains With Less Coverage
The results address a practical challenge for agents that can change business records: a plausible conversation or a sequence of tool calls does not prove that the requested outcome was completed correctly. A final-state database check can score what remains in the system, including whether required records are missing or incorrect.
For developers, the reported shift could matter because repeatable correctness may be more useful than occasional success in workflows involving bookings, refunds, or other consequential records. But the benchmark shows a mixed outcome: more tasks passed consistently, while the model solved fewer tasks at least once. Teams evaluating the method will need to weigh that change against their own needs for reliability and breadth. The figures are reported benchmark results, not evidence of production performance.
As an affiliate, we earn on qualifying purchases.
From Evaluation to Agent Training
ThinkingBox was developed as a way to evaluate agents by checking the database records left at the end of a task. The benchmark described by the authors contains 507 tasks, each with a backend, MCP tools and a simulated user holding private context. Executable checks compare the resulting database state with the expected outcome.
In the training approach, the team selected practice tasks using what it calls offline policy-relative difficulty: it estimated the base model’s success probability once, then kept the training pool fixed. The authors say this was intended to focus practice on tasks that were neither already easy nor beyond the model’s ability. They report training Qwen3.8-27B for 50 GRPO steps and evaluating a selected checkpoint on a separate benchmark. The benchmark was described as out of distribution because its domains, databases and most MCP tools were not encountered during training.
“For most tasks, an executable check can return 1 or 0 directly from the final database state. That is not just a grader. This can be a reward function.”
— Microsoft and Hugging Face, in their joint post
reinforcement learning model training kit
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Questions Beyond One Benchmark Run
The reported evidence covers one 50-step training run. The supplied account does not provide enough detail to determine how sensitive the results are to random seeds, the choice of training pool or run length, or how the method compares with alternative reward designs and training approaches. Those gaps make it difficult to isolate which parts of the reported change are attributable to database-state rewards.
The authors report a decline of seven tasks in pass@20, but do not establish why coverage fell. Nor does the account provide results from live production workflows. Generalization, operational reliability and training costs remain open questions; the reported hotel example illustrates one case rather than establishing performance across a class of tasks.
AI agent database validation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Reproduction and Broader Testing
With the stack now on GitHub, researchers and developers can inspect the implementation and attempt their own training runs. The next useful evidence would include reproductions by other teams and tests across additional models, task pools, run lengths and random seeds, including whether the increase in consistent passes comes with a similar coverage decline.
Before applying the method in production, developers would also need evaluations tied to the intended workflows. Those tests should check whether final records are correct and whether agents avoid incomplete or unauthorized changes, such as the approval-related errors described in the hotel example. The source account does not give a timeline for further results.
machine learning model evaluation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is ThinkingBox-Training?
It is a Microsoft and Hugging Face training stack that uses executable checks of an agent’s final database state as reinforcement-learning rewards. The teams say it is available on GitHub.
How did the reported model results change?
In the reported run, Qwen3.8-27B’s average success rate rose from 51.70% to 60.78% across 10,140 attempts. Tasks passed in all 20 attempts rose from 38 to 107, while tasks solved at least once fell from 453 to 446.
What does the pass@20 decline mean?
Pass@20 is the share of benchmark tasks solved at least once in 20 attempts. It fell from 453 tasks (89.35%) to 446 (87.97%), even as the number passed in every recorded attempt increased. The reported figures do not explain the cause of the decline.
Do the results prove ThinkingBox works in production?
No. The reported evaluation covers a benchmark and one 50-step training run. The supplied account does not report live production results or establish performance across other models, training setups or deployment settings.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
