In this episode of the Existential Hope Podcast, Beatrice Erkers is joined by David Duvenaud, Associate Professor at the University of Toronto and former researcher at Anthropic.

We discuss David’s work on post-AGI civilizational equilibria and the widely discussed paper Gradual Disempowerment. David reflects on why liberalism may not hold up in a world where humans are no longer needed, how UBI could be Goodharted into absurdity, and what it would take to design institutions that protect humans even when incentives don’t.

We also cover:

- Forecasting the long-term future using LLMs trained on historical data

- Robin Hanson’s idea of futarchy (governance by prediction markets)

- Asymmetrical but beneficial relationships between humans and AI

- Uploading, cultural legacies, and the possibility of “worthy successors”

On the Existential Hope Podcast hosts Allison Duettmann and Beatrice Erkers from the Foresight Institute invite scientists, founders, and philosophers for in-depth conversations on positive, high-tech futures.

Full transcript, listed resources, and more: https://www.existentialhope.com/podcasts

Follow on X.

Hosted on Acast. See acast.com/privacy for more information.

What does a genuinely positive future with AI look like? While dystopian visions are common, the most valuable—and scarcest—resource we have is a concrete, hopeful vision for where we're headed.In this episode, we're joined by Nathan Labenz, host of the popular Cognitive Revolution podcast, to explore the tangible possibilities of a beneficial AI-driven world.

Nathan shares his insights on everything from the near-term transformations in education and healthcare—like AI-driven antibiotic discovery and personalized learning—to the grand, long-term visions of curing all diseases and becoming a multi-planetary species.

We dive deep into crucial concepts like Eric Drexler's "comprehensive AI services" as a model for safety through narrowness, the transformative power of self-driving cars, and how we can collectively raise our ambitions to build the future we actually want.

On the Existential Hope Podcast hosts Allison Duettmann and Beatrice Erkers from the Foresight Institute invite scientists, founders, and philosophers for in-depth conversations on positive, high-tech futures.

Full transcript, listed resources, and more: https://www.existentialhope.com/podcasts

Follow on X.

Hosted on Acast. See acast.com/privacy for more information.

For years, the conversation about the long-term future has been dominated by a crucial question: how do we avoid extinction? But what if ensuring our survival is only half the battle? In this episode, Beatrice is joined by Fin Moorhouse, a researcher at Forethought and co-author with Will MacAskill of the Better Futures series, to make the case for focusing on the other half: flourishing. Or as we'd like to say in this podcast: Existential Hope!

Fin challenges the idea that a great future will emerge automatically if we just avoid the worst-case scenarios. Using the analogy of a grand sailing expedition, he explores the complexities of navigating towards a truly optimal world, questioning whether our current moral compass is enough to guide us.

The conversation dives into the concept of "moral catastrophes"—profound ethical failings, like industrial animal farming, that could persist even in technologically advanced futures. Fin also tackles the complex challenges posed by digital minds, from the risk of accidental suffering to the creation of "willing servants." He argues for the power of "moral trade" as a tool to build a more pluralistic and prosperous world, and explains why we should aim for a "Viatopia"—a stable and self-sustaining state that makes a great future highly likely.

On the Existential Hope Podcast hosts Allison Duettmann and Beatrice Erkers from the Foresight Institute invite scientists, founders, and philosophers for in-depth conversations on positive, high-tech futures.

Full transcript, listed resources, and more: https://www.existentialhope.com/podcasts

Follow on X.

Hosted on Acast. See acast.com/privacy for more information.