Recently, OpenAI’s CEO Sam Altman was fired from his position following a warning letter from several staff researchers to the board of directors highlighting a powerful artificial intelligence discovery that could pose a threat to humanity, according to two sources familiar with the matter.
The letter and AI algorithm were key factors in Altman’s ouster, with concerns over commercializing advances before understanding their consequences being one of the many grievances listed by the board. The researchers who wrote the letter did not respond to requests for comment. However, OpenAI did acknowledge in an internal message to staffers the existence of a project called Q* and a letter to the board.
OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Some staff members believe that Q* could be a breakthrough in the search for artificial general intelligence (AGI), which has long been considered a holy grail for AI researchers.
This is because Q* was able to solve certain mathematical problems, which could imply that AI could have more reasoning capabilities resembling human intelligence.
Furthermore, researchers consider math to be a frontier of generative AI development. Currently, generative AI is good at writing and language translation by statistically predicting the next word, but conquering the ability to do math where there is only one right answer could be applied to novel scientific research.
The researchers who sent the letter to the board also flagged the potential danger of AI. There has long been discussion among computer scientists about the danger posed by highly intelligent machines, for instance, if they might decide that the destruction of humanity was in their interest. Additionally, multiple sources confirmed the existence of an “AI scientist” team formed by combining earlier “Code Gen” and “Math Gen” teams, which explored how to optimize existing AI models to improve their reasoning and eventually perform scientific work.
Altman had led efforts to make ChatGPT one of the fastest-growing software applications in history and drew investment from Microsoft to get closer to AGI. Despite announcing new tools in a demonstration this month and teasing at a summit of world leaders in San Francisco about the major advances that he believed were in sight, Altman was fired a day later.