Sakana AI has introduced new tools—ALE-Bench and ALE-Agent—designed for tasks requiring complex optimization and strategic thinking.
In collaboration with AtCoder Inc., the Sakana team created an open benchmark called ALE-Bench, which focuses on NP-hard problems. The main goal is not just to write code that works, but to develop solutions that are as efficient as possible and capable of improving over time.
What makes ALE-Bench special? Firstly, the problems require long-term planning and unconventional approaches, as simple solutions are often unattainable. However, it is always possible to improve the current results, allowing progress to be tracked. This benchmark is ideal for evaluating reasoning models and advanced AI coders capable of thinking on more complex levels.
Alongside this set of tools, ALE-Agent has been developed—a autonomous agent trained to independently solve such complex tasks from scratch. In May 2025, it demonstrated its capabilities by participating in the AtCoder Heuristic Contest, one of the most challenging algorithm competitions, finishing 21st out of 1,000 participants.
More information can be found in the Sakana AI blog, on arXiv, as well as in the dataset and source code links.