#122 | Frankly
A Country of Geniuses: Anthropic CEO’s Warnings, Plus Wide-Boundary Considerations on AI
Description
Last week there was so much news Nate recorded two Franklies – this is the second of those, which shares his reflections on a recent seminal essay posted by Anthropic CEO Dario Amodei, likening Artificial Intelligence as a “rite of passage” for the human species rather than just a narrow technological breakthrough. Amodei posits the possibility that we are now in what Carl Sagan once called a phase of “technological adolescence,” wherein humans’ technologies and tools become powerful enough to reshape or destabilize civilization faster than our collective wisdom can keep up. As a civilizational force, AI doesn’t automatically act as humanity’s salvation or catastrophe – it acts as a mirror that reflects the maturity (or immaturity) of the humans – and systems – deploying it.
In this episode, Nate then widens the boundaries of the AI conversation to incorporate the biophysical reality and institutional systems that support these technologies, emphasizing energy, materials, infrastructure, governance, and incentives as the real limiting factors and alignment challenges. By incorporating the deeper structures that shape societal outcomes in this dialogue, he raises questions about how the assumption of shared goals like growth and optimization might steer AI towards outcomes that undermine ecological and social stability.
What will it mean in biophysical terms if we introduce near-limitless cognitive power into a world already constrained by energy and materials? Is it possible for societies to build the wisdom, restraint, and governance needed to survive the “technological adolescence” of AI? And if “intelligence” becomes cheap and abundant with AI expansion, how might that impact humans’ shared semblances of values, goals, and definitions of success?
In French, we have a motto that says that a simple drawing is often better than a long explanation. Jean-Marc Jancovici Carbone 4 President
That’s very understandable because with left atmosphere thinking, one of the problems is that you see everything as a series of problems that must have solutions. Iain McGilchrist Neuroscientist and Philosopher
We can’t have hundreds and hundreds of real relationships that are healthy because that requires time and effort and full attention and awareness of being in real relationship and conversation with the other human. Nate Hagens Director of ISEOF
This is the crux of the whole problem. Individual parts of nature are more valuable than the biocomplexity of nature. Thomas Crowther Founder Restor
Show Notes & Links to Learn More
Download transcriptThe TGS team puts together these brief references and show notes for the learning and convenience of our listeners. However, most of the points made in episodes hold more nuance than one link can address, and we encourage you to dig deeper into any of these topics and come to your own informed conclusions.
00:04 – Carl Sagan
00:29 – Dario Amodei, Anthropic
00:40 – Dario Amodei’s paper: The Adolescence of Technology
02:39 – Fossil energy army
02:55 – Population of Spain
03:42 – Chatbot
04:17 – AI ability to be copied, run in parallel, problem solve
04:54 – AI building next generation of AI
05:19 – Recursive self-improvement, Feedback loops
05:32 – Frankly #97, The Superorganism in 7 Minutes
06:00 – Davos, Shift towards AI development
07:28 – Deception, blackmail, scheming in AI models
08:00 – AI and bioweapons
08:35 – Authoritarianism
10:49 – Philip Morris, Dopamine, Exxon Mobile
11:45 – Components of a datacenter
12:10 – Silver $115 an ounce, Silver makes up 40% of the cost of a solar panel
12:20 – Our expected copper requirements for future products are way bigger than projected supply
13:25 – Permitting for datacenter construction, increased grid capacity, water demand
13:45 – Rube Goldberg machine
16:06 – Tristan Harris, TGS Ep #16 Tristan Harris
16:11 – AI industry (guardrails) agreements
18:10 – Dennis Meadows, TGS Ep #12 Dennis Meadows
18:57 – William Butler Yeats
19:14 – Daniel Schmachtenberger, TGS playlist of Daniel Schmachtenberger episodes
20:28 – King Midas, The Terminator
22:16 – Eliezer Yudkowsky, Nate Soares, TGS Ep #203 Nate Soares
22:44 – Mary Shelley, Frankenstein
23:52 – Macroeconomics
24:07 – Ken Griffin, commentary at Davos and related X post
27:02 – Trojan horse
27:46 – Peak oil, Frankly 56 Peak Oil, AI, and the Straw
29:17 – Frodo Baggins, The Shire
29:28 – The Great Simplification
31:05 – E.O. Wilson
