Nvidia used 20,000 training examples to fix its AI’s mistakes. Its coding score jumped.

The New Stack (AI)
Read full story· 4 min
Nvidia used Factory's Droid coding-agent to generate 20,000 post-training examples for its Nemotron 3 Ultra model, improving its coding accuracy significantly. They applied PivotRL reinforcement learning to enhance task execution and error recovery, achieving better performance with fewer rollouts.

What happened

Nvidia used Factory's Droid coding-agent to generate about 20,000 new training examples for its Nemotron 3 Ultra model, which was released in June. The company applied a reinforcement learning method called PivotRL to improve the model's ability to follow instructions, use tools, and recover from errors. Nvidia varied the training environments and combined data from several agent platforms to avoid overfitting to a single setup. As a result, the coding accuracy of Nemotron 3 Ultra improved significantly on benchmark tests.

Key details

  • Nemotron 3 Ultra is a 550-billion-parameter open model released in June.
  • Factory's Droid agent generated approximately 20,000 post-training data rows.
  • PivotRL reinforcement learning achieved similar accuracy to end-to-end RL with one-quarter as many rollout turns.
  • Nemotron 3 Ultra’s Terminal-Bench 2.0 score rose from 38.5% to 49%.
  • Nvidia published its reinforcement learning pipeline for reproduction on GB200 NVL72 hardware.

Why it matters

Developers and companies using Nemotron 3 Ultra now have access to a model with improved coding accuracy, which can reduce errors and increase productivity in coding tasks. Nvidia’s approach also lowers the computational cost of reinforcement learning, making it more practical for large-scale model training. This change gives organizations a more efficient and reliable tool for automated coding and related applications.


More in Machine Learning