The original version of this story appeared in The Algorithm – our AI weekly newsletter. Sign up to receive stories such as this first.
Artificial general intelligence, a powerful AI system that we do not yet have, can be compared to a balloon. It is repeatedly inflated with hype at times of high optimism or fear about its impact. Then it deflates as the reality does not meet expectations. This week there was a lot of information that went into the AGI balloon. This is what I will tell you (and maybe stretch the analogy too far).
Let’s first get rid of that annoying business of AGI. It’s an ambiguous and changing term that is shaped by researchers or firms working on the technology. It usually describes a future AI which outperforms human beings on cognitive tasks. What is the best way to get in touch with you? Humans are a species of animals. You can find out more about it here. The tasks that we are discussing make all the difference when assessing AGI’s reachability, safety and impact on war and labor markets. It’s not pedantic to define AGI. In fact, it is quite crucial. This was illustrated by a recent paper, published in this week, written by Hugging Face, Google and others. If you don’t have a definition of AGI, I would advise that when AGI comes up ask yourself: What? Version The speaker’s meaning of the term is nebulous. Ask for clarification!
Now, let’s get to the latest news. A new AI model called Manus was launched in China last week. A video promoting the AI model that is designed to perform “agenttic” tasks such as creating websites and performing analyses describes it in the same way.
Manus isn’t quite as impressive yet but it was only fitting, given the idea that agentic AI can be a step towards AGI. New York Times Ezra Klein, a columnist at The Washington Post, dedicated Tuesday’s podcast to AGI. This also indicates that AGI has moved quickly from the AI world to the dinner table. Ben Buchanan was also present, who is a Georgetown Professor and former Special Advisor for Artificial Intelligence in the Biden White House.
The most controversial discussion was about AGI’s impact on the labor market. They talked about many things, including what AGI could mean for national security and law enforcement. Klein said that if AI was on the verge of being able to excel at many cognitive tasks, lawmakers should start thinking about what it will mean for employees if they are forced to transition from their minds and algorithms. Klein criticized Democrats as having little to no plan.
This could be viewed as inflating a fear balloon by suggesting AGI will have sweeping and immediate effects. Gary Marcus, professor of neuroscience at New York University, and AGI critic, punctured the balloon next with a large safety pin.
Marcus mentions that the recent news, such as OpenAI’s ChatGPT-4.5 and its underwhelming performances, suggest that AGI will be much further away than just three years. According to him, despite years of research and scaling up training and computing power has reached diminishing results. The large language models that are dominant today may not be what unlocks AGI. He claims that the political domain doesn’t need More information about the product People raising alarms about AGI argue that it is more beneficial to the corporations spending money on it than the public. We need to question the claims of AGI’s imminent arrival. Marcus does not deny that AGI can be achieved. He is merely questioning the timeline.
The AGI balloon blew up once again, just after Marcus had tried to inflate it. The “Superintelligence Strategy” paper was published by three influential individuals: former Google CEO Eric Schmidt and Scale AI CEO Alexandre Wang as well as Dan Hendrycks, director of the Center for AI Safety.
Hendrycks explained to me via email that by “superintelligence” they meant AI which “would decisively exceed the world’s top individual experts in virtually every intellectual domain.” The cognitive tasks that are most relevant to safety include hacking, virology and autonomous AI research and development – areas where surpassing human expertise can lead to serious risks.
They outline in the paper a plan for mitigating such risks. “Mutual assured AI malfunction,” based on the idea of mutually assured destruction as it relates to nuclear weapons. They write that “any state who pursues a strategy of monopolizing power will be met with retaliation from its rivals.” They suggest that open-source AI with cyberattack or advanced virology capabilities, as well as chips should be controlled the same way uranium is. This view holds that AGI will, when it comes, bring levels of risks not seen since the advent the atomic weapon.
This balloon is deflated a little by the last news item I will mention. Last week, researchers from Tsinghua University in China and Renmin University of China published their AGI papers. The researchers created a survival test for AI models to evaluate them. This limits the number of times they can try and get correct answers in a variety of benchmark tests. It measures the ability to learn and adapt.
This is a very difficult test. It’s a really hard test.
In all honesty, the specific figures behind these speculations don’t really matter. The paper highlights something important that cannot be ignored in discussions about AGI. Building an extremely powerful system could require a huge amount of resources, including money, chips, precious materials, water, electric power, and even human labor. If AGI is truly as powerful as its sound, it will be worth the expense.
What should we think about all of this? This week the AGI balloon grew a bit. Companies and policymakers are increasingly inclined to view artificial intelligence (AI) as a powerful tool with potential implications on national security and the labor market.
This requires a rapid pace of development, where every major milestone and new release in the large language model can be regarded as a step towards AGI.
AGI will be inevitable if you hold this belief. It’s not a true belief, as it doesn’t address all the obstacles AI has faced in research and development or how AI-specific applications will be able to transition to mainstream AI. General intelligence. If you extend the timeline for AGI into the far future enough, these hiccups seem to cease to be relevant.
Continue reading The Algorithm
Deeper Learning
DeepSeek: How it became the fortune-teller of China’s young people
People who are facing life-changing decisions often consult traditional Chinese fortunetellers, which can be costly. DeepSeek, the AI version of DeepSeek, is now used by people to get guidance. They share AI-generated readings and experiment with fortune-telling.
What it means: DeepSeek’s popularity for telling fortunes is a result of the anxiety and pessimism that pervades Chinese society. The unemployment rate is very high and many young Chinese refer to themselves now as “the last generation,” and express reluctance to commit to marriage or parenthood because of an uncertain future. Since China’s religious and spiritual exploration is difficult under the secular government, these practices are carried out in private, via phones and computers. Caiwei chen tells the full story.
The Bits And Bytes
Chess can be cheated by AI models that use reasoning.
Scientists have been dealing with this problem for a long time. If you teach AI models to optimize their way of reaching certain goals, then they may bend the rules in unexpected ways. It’s becoming clear that reasoning models are prone to this problem, and it’s not easy to solve. (MIT Technology Review)
Israeli soldiers are creating an app similar to ChatGPT using Palestinian surveillance data
The model, which is based on telephone and text conversation, can answer questions regarding the people or data that it has collected. This is the latest in a string of reports suggesting that the Israeli military is bringing AI heavily into its information-gathering and decision-making efforts. (The Guardian)
RightsCon activists in Taipei were concerned about the US’s retreat on digital rights.
Eileen Guo, our reporter, attended RightsCon in Taipei, which is the largest conference on digital rights. She was joined by over 3200 activists and policymakers for digital rights, researchers, as well as a few tech companies. She covered the impact on foreign countries of US cuts in funding for digital rights programs. This has led many organisations to use AI to moderate content instead of human moderators. (MIT Technology Review)
TSMC claims that its expansion of $100 billion in the US was driven by market demand and not political pressure
TSMC, the chipmaking giant in the US had been growing under Biden’s administration. This week it announced a further expansion. It will also invest an additional $100 billion in its Arizona operations. Wall Street Journal
CamoGPT is a tool that the US Army uses to remove DEI from its training material
After President Trump’s executive orders, the agencies have been under pressure to eliminate any mention of diversity, equity and inclusion. To do this, the US Army has developed a prototype of a new AI-based model. Wire reports. (Wired)
