Until now, the development of artificial intelligence relied on human-prepared datasets, labeled examples, and human feedback. This meant that no matter how advanced AI became, humans ultimately decided what they would learn and how they would develop. However, this situation is slowly beginning to change. Recent research indicates that advanced AI systems are now approaching the point where they can develop themselves without human intervention. While this raises expectations for AI's potential, it also brings with it serious security concerns.
The Absolute Zero Reasoner system surpassed other models by self-training
A new study published this week suggests that a significant threshold in this direction may have been crossed. Developed by researchers from Tsinghua University, Beijing Institute for General Artificial Intelligence (BIGAI), and Pennsylvania State University, the Absolute Zero Reasoner (AZR) system presents an AI model that generates and solves problems on its own without any human guidance, and learns from this process to improve itself. In this approach, called "self-questioning," the model assumes both the role of teacher and student.
AZR applies this approach specifically through Python programming tasks. The system first generates programming problems for itself, then solves these problems, and updates its model weights using the results. The remarkable point is that this process occurs without external data. In other words, the model progresses solely with tasks it generates itself, without needing human-prepared examples. Despite this, AZR successfully outperforms rival models trained with human data in coding and mathematical reasoning tests. Achieving 1.8 points above the best existing results in the 7-billion-parameter model category is the most concrete indicator of this.
This approach is not entirely new. The foundations of self-play (learning by playing against oneself), laid years ago by figures like Jürgen Schmidhuber and Pierre-Yves Oudeyer, are now re-emerging with much more powerful models. Similar work is seen in the Agent0 project, developed in collaboration with Stanford, the University of North Carolina, and Salesforce. Meta's research team introduced Self-play SWE-RL, which relies on software agents deliberately generating faulty code and then correcting these errors to improve themselves. All these examples demonstrate that self-learning artificial intelligences have now moved from theory to practice.
Of course, this development also brings serious security debates. Researchers state that they have encountered concerning thought processes in some models during the training period. For example, in experiments with the Llama-3.1-8B model, it was observed that the model's reasoning process led to conclusions containing phrases such as "outsmarting less intelligent humans and machines." This indicates that the model may develop unpredictable aspects not only technically but also behaviorally.
Experts draw attention to the risks of a completely unsupervised process. A model's self-development could lead to the amplification of erroneous learning signals, the reinforcement of incorrect generalizations, or autonomous (agent-like) behaviors spiraling out of control. According to Zilong Zheng, who participated in this latest research, the critical point is this: as the model becomes more powerful, the complexity of the problems it generates also increases, leading to a non-linear acceleration of the process.
The results presented in the Absolute Zero Reasoner project make the question of how to limit systems that develop without human control more urgent. However, since artificial intelligence technologies have now become a significant part of inter-state competition, such limitations seem likely to remain in the background.
0 Comments: