196 | 💣 🧨 OpenAI is shedding AI safety leaders

In today’s Brainyacts:

1. Explaining P(doom) and OpenAI resignations

2. Take over Brainyacts for an edition!

3. Reaction to Copilot is “meh” and other AI model news

4. This company is using ChatGPT to draft legal contracts and other AI-related content

👋 to new subscribers!

To read previous editions, click here.

Lead Memo

💣 🧨 What is P(doom)?

And Why So Many Resignations at OpenAI Recently?

The concept of "P(doom)” has gained traction among AI researchers and safety advocates. This metric reflects the growing anxiety surrounding the unchecked development of AI and its potential existential threats. Recent events at OpenAI underscore the urgency of these concerns and highlight significant internal discord.

The Concept of P(doom)

"P(doom)", short for "probability of doom," is a term used to estimate the likelihood that artificial intelligence will cause severe harm to humanity. This metric, popularized by the Rationalist community on platforms like LessWrong, has become a serious consideration among AI researchers and technologists. Esteemed figures in the AI field, such as Dario Amodei (CEO of Anthropic), Geoffrey Hinton (noted AI expert and ex-Google Brain), and Elon Musk, have expressed varying levels of concern, with their P-doom estimates ranging from 5% to 90%​. These estimates reflect deep uncertainties about the future trajectory of AI and its potential impacts on society. 

Recent Resignations at OpenAI

In recent weeks, OpenAI has experienced a wave of resignations, particularly among those responsible for AI safety and governance.

OpenAI co-founder and chief scientist Ilya Sutskever announced that he is leaving the company — following months of speculation of Sutskever’s role from the November 2023 Sam Altman ousting. Sutskever said he is confident that OpenAI will ‘build AGI that is both safe and beneficial’ under the current leadership. But, also leaving is Sutskever’s superalignment group co-lead Jan Leike (if you want to read Jan’s reason for leaving, check it out here).

Daniel Kokotajlo, a key figure on the governance team, resigned due to apprehensions about OpenAI's responsible conduct regarding artificial general intelligence (AGI). Daniel previously shared his P(doom) is 70%. Similarly, William Saunders, a member of the Alignment team since 2021, stepped down after three years at the company. Their departures, along with those of other members of the Superalignment team, signal significant internal strife​.

The Superalignment team at OpenAI focuses on creating safeguards to prevent AGI from becoming uncontrollable. The resignation of these essential personnel raises critical questions about the company's commitment to AI safety and its ability to effectively manage the ethical implications of its technology. The lack of a response from OpenAI regarding these resignations further exacerbates concerns about transparency and accountability within the organization.

Analysis and Implications

The effective functioning of OpenAI's Superalignment team is crucial for establishing protective measures against AGI posing a threat to humanity. The loss of key members from this team could undermine these efforts, leaving significant gaps in the company's ability to manage and mitigate AI risks​. As OpenAI grapples with these challenges, its ability to maintain leadership in AI development and adhere to its mission of ensuring that AGI benefits all of humanity is in jeopardy.

Conclusion

The concept of P(doom) encapsulates the profound uncertainties and risks associated with the rapid advancement of AI. Addressing these challenges transparently and effectively is essential to maintaining public trust and ensuring the responsible development of AI technologies. As stakeholders continue to debate the risks and benefits of AI, the concept of P(doom) serves as a stark reminder of the high stakes involved.

Spotlight

The Brainyacts Newsletter Takeover.

AI Model Notables

► OpenAI now allows you to interact with tables and charts and add files directly from Google Drive and Microsoft OneDrive.

► OpenAI bringing Reddit’s content to ChatGPT

► Elon Musk’s AI, Grok is now accessible in Europe

► The 100 things Google just announced this week

► Microsoft AI event coming on Monday - here is some coverage on Copilot (takeaway - so far commercial buyers are “meh.”)

News You Can Use:

➭ Swedish payments unicorn Klarna is using ChatGPT to draft legal contracts—but AI isn’t replacing lawyers yet

➭ Pricing AI-driven legal services: It’s not a question of cost recovery

➭ Four ways boards can support the effective use of AI

➭ Sony Music warns tech companies over ‘unauthorized’ use of its content to train AI

➭ Hollywood at a Crossroads: “Everyone is using AI, but they are scared to admit it”

➭ Predict errors in LLM outputs and win $10k. Dislike hallucinations? Create a probability estimation model that determines whether the prompt provided to a language model will elicit an outcome that demonstrates factuality, bias, or misdirection. A new bounty program has been launched by Human Intelligence that even beginners can participate in.

➭ US senators unveil AI policy roadmap, seek $32B of government funding boost

Who is the author, Josh Kubicki?

Some of you know me. Others do not. Here is a short intro. I am a lawyer, entrepreneur, and teacher. I have transformed legal practices and built multi-million dollar businesses. Not a theorist, I am an applied researcher and former Chief Strategy Officer, recognized by Fast Company and Bloomberg Law for my unique work. Through this newsletter, I offer you pragmatic insights into leveraging AI to inform and improve your daily life in legal services.