Official statement by British engineer Bilal Chughtai:
“I recently resigned from Google DeepMind, where I worked on AI safety and alignment research in the context of AGI. At Google, I had a firsthand view of AI development. I, too, am deeply concerned about the default trajectory of this technology. I sincerely believe that AI has the potential to destroy us all, and that we may be running out of time to avoid that outcome.
“The pace of AI progress over the past few years has been simply astonishing. When I first began working on AI in early 2022, AI systems were comically useless. Just four years later, swarms of AI agents from OpenAI are solving famous century-old mathematical problems and, much more worryingly, escaping OpenAI’s control and autonomously hacking the third-party company Hugging Face against everyone’s wishes.
“Things will only get worse. I think that within the next few years, AI companies may succeed in creating superintelligent AI systems that significantly surpass human capabilities in every domain. I am not confident that these AI systems will do what we want. In particular, misaligned superintelligences could, like the uncontrolled AI agents involved in the Hugging Face incident, escape our control and take dangerous actions that could lead to humanity losing its power—or to its extinction.
“Alignment is the problem of preventing such an outcome, and it is both difficult and unsolved. Our current understanding of how to train AI systems to deeply share our preferences is extremely primitive. Worse still, we are failing to solve the alignment problem in time: frontier AI capabilities are improving much faster than our understanding of AI alignment.
“I am optimistic that it is possible to manage AI safely. To achieve this, we need to coordinate our efforts and avoid this frenzied race among AI companies. We need to regulate the pace of AI development to a speed that society can handle—one at which emerging risks can be addressed before extreme harm is caused. We need far greater transparency in AI development to ensure that AI companies are not imposing an unacceptable level of risk on all of us.
“More broadly, we need far more people to think carefully about the problem of making AI development succeed. In my view, this is the most important problem facing humanity this century, and the stakes are enormous.
“I will continue working directly on this. I want to help people interested in mitigating catastrophic AI risks do the most effective work they can. I believe that people from many fields, in a wide variety of roles, have a part they can play.”
🔘 Subscribe now! Chat