Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves.

GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg)

• Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search.

• The *architecture* of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard.

• Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks.

• Are these systems actually thinking outside the box, or are they parasitic on their starting conditions? When LLMs run autonomously, "nothing interesting happens." Robert pushes back with the stepping-stone argument — evolution doesn't need to extrapolate, just recombine usefully.

• The AI Scientist question: can automated research pipelines produce real science, or just workshop-level slop that passes surface-level review? Robert is honest that the current version is more co-pilot than autonomous researcher.

• Where this lands in 5-20 years — Robert's prediction that scientific research will be fundamentally transformed, and Tim's thought experiment about alien mathematical artifacts that no human could have conceived.

Robert Lange: https://roberttlange.com/

---

TIMESTAMPS:

00:00:00 Introduction: Robert Lange, Sakana AI and Shinka Evolve

00:04:15 AlphaEvolve's Blind Spot: Co-Evolving Problems with Solutions

00:09:05 Unknown Unknowns, POET, and Auto-Curricula for AI Science

00:14:20 MAP-Elites and Quality-Diversity: Shinka's Evolutionary Architecture

00:28:00 UCB Bandits, Mutations and the Vibe Research Vision

00:40:00 Scaling Shinka: Meta-Evolution, Democratisation and the Three-Axis Model

00:47:10 Applications, ARC-AGI and the Future of Work

00:57:00 The AI Scientist and the Human Co-Pilot: Who Steers the Search?

01:06:00 AI Scientist v2, Slop Critique and the Future of Scientific Publishing

---

REFERENCES:

paper:

[00:03:30] ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution

https://arxiv.org/abs/2509.19349

[00:04:15] AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery

https://arxiv.org/abs/2506.13131

[00:06:30] Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents

https://arxiv.org/abs/2505.22954

[00:09:05] Paired Open-Ended Trailblazer (POET)

https://arxiv.org/abs/1901.01753

[00:10:00] PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem

https://arxiv.org/abs/1112.5309

[00:10:40] Automated Capability Discovery via Foundation Model Self-Exploration

https://arxiv.org/abs/2502.07577

[00:15:30] Illuminating Search Spaces by Mapping Elites (MAP-Elites)

https://arxiv.org/abs/1504.04909

[00:47:10] Automated Design of Agentic Systems (ADAS)

https://arxiv.org/abs/2408.08435

PDF : https://app.rescript.info/api/sessions/b8a9dcf60623657c/pdf/download

Transcript: https://app.rescript.info/public/share/SDOD_3oXOcli3zTqcAtR8eibT5U3gam84oo4KRtI-Vk

Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be turning software engineering into a slot machine and how to maintain true technical intuition in the age of large language models.

GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg)

Jeremy Howard is a renowned data scientist, researcher, entrepreneur, and educator. As the co-founder of fast.ai, former President of Kaggle, and the creator of ULMFiT, Jeremy has spent decades democratizing deep learning. His pioneering work laid the foundation for modern transfer learning and the pre-training and fine-tuning paradigm that powers today's language models.

Key Topics and Main Insights Discussed:

- The Origins of ULMFiT and Fine-Tuning

- The Vibe Coding Illusion and Software Engineering

- Cognitive Science, Friction, and Learning

- The Future of Developers

RESCRIPT: https://app.rescript.info/public/share/BhX5zP3b0m63srLOQDKBTFTooSzEMh_ARwmDG_h_izk

Jeremy Howard:

https://x.com/jeremyphoward

https://www.answer.ai/

---

TIMESTAMPS (fixed):

00:00:00 Introduction & GTC Sponsor

00:04:30 ULMFiT & The Birth of Fine-Tuning

00:12:00 Intuition & The Mechanics of Learning

00:18:30 Abstraction Hierarchies & AI Creativity

00:23:00 Claude Code & The Interpolation Illusion

00:27:30 Coding vs. Software Engineering

00:30:00 Cosplaying Intelligence: Dennett vs. Searle

00:36:30 Automation, Radiology & Desirable Difficulty

00:42:30 Organizational Knowledge & The Slope

00:48:00 Vibe Coding as a Slot Machine

00:54:00 The Erosion of Control in Software

01:01:00 Interactive Programming & REPL Environments

01:05:00 The Notebook Debate & Exploratory Science

01:17:30 AI Existential Risk & Power Centralization

01:24:20 Current Risks, Privacy & Enfeeblement

---

REFERENCES:

Blog Post:

[00:03:00] fast.ai Blog: Self-Supervised Learning

https://www.fast.ai/posts/2020-01-13-self_supervised.html

[00:13:30] DeepMind Blog: Gemini Deep Think

https://deepmind.google/blog/accelerating-mathematical-and-scientific-discovery-with-gemini-deep-think/

[00:19:30] Modular Blog: Claude C Compiler analysis

https://www.modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software

[00:19:45] Anthropic Engineering Blog: Building C Compiler

https://www.anthropic.com/engineering/building-c-compiler

[00:48:00] Cursor Blog: Scaling Agents

https://cursor.com/blog/scaling-agents

[01:05:15] fast.ai Blog: NB Dev Merged Driver

https://www.fast.ai/posts/2022-08-25-jupyter-git.html

[01:17:30] Jeremy Howard: Response to AI Risk Letter

https://www.normaltech.ai/p/is-avoiding-extinction-from-ai-really

Book:

[00:08:30] M. Chirimuuta: The Brain Abstracted

https://mitpress.mit.edu/9780262548045/the-brain-abstracted/

[00:30:00] Daniel Dennett: Consciousness Explained

https://www.amazon.com/Consciousness-Explained-Daniel-C-Dennett/dp/0316180661

[00:42:30] Cesar Hidalgo: Infinite Alphabet / Laws of Knowledge

https://www.amazon.com/Infinite-Alphabet-Laws-Knowledge/dp/0241655676

Archive Article:

[00:13:45] MLST Archive: Why Creativity Cannot Be Interpolated

https://archive.mlst.ai/read/why-creativity-cannot-be-interpolated

Research Study:

[00:24:30] METR Study: AI OS Development

https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

Paper:

[00:24:45] Fred Brooks: No Silver Bullet

https://www.cs.unc.edu/techreports/86-020.pdf

[00:30:15] John Searle: Minds, Brains, and Programs

https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A

What if life itself is just a really sophisticated computer program that wrote itself into existence?

Blaise Agüera y Arcas presenting at ALife 2025 — the most technically detailed public walkthrough of the ideas in his *What is Life?* and *What is Intelligence?* books that we've come across.He covers the BFF experiments (self-replicating programs emerging spontaneously from random noise), the mathematical framework connecting Lotka-Volterra population dynamics with Smoluchowski coagulation, eigenvalue analysis of cooperation matrices, and his central claim that symbiogenesis — not mutation — is the primary engine of evolutionary novelty.The experimental results are genuinely striking: complex self-replicating code arising from random byte strings with zero mutation, a sharp phase transition that looks like gelation, and a proof that blocking deep symbiogenetic ancestry trees prevents the transition entirely.A few things worth flagging for critical viewers:— The substrate is more carefully engineered than the framing sometimes suggests. The choice of language, tape length, interaction protocol, and step limits all shape what emerges. Their own SUBLEQ counterexample (where self-replicators *don't* arise despite being theoretically possible) highlights that these design choices matter substantially — and a general theory of which substrates support this transition is still missing.— The leap from "self-replicating programs on fixed-length tapes" to "life was computational and intelligent from the start" involves significant philosophical extrapolation beyond what the experiments directly demonstrate.— The Bedau et al. (2000) open problems paper he references at the start actually sets a higher bar for Challenge 3.2 than BFF currently meets: it asks that "the internal organization of these 'organisms' and the boundaries separating them from their environment arise and be sustained through the activities of lower-level primitives" — whereas BFF's tape boundaries are fixed by design, not emergent.

---

TIMESTAMPS:

00:00:00 Introduction: From Noise to Programs & ALife History

00:03:15 Defining Life: Function as the "Spirit"

00:05:45 Von Neumann's Insight: Life is Embodied Computation

00:09:15 Physics of Computation: Irreversibility & Fallacies

00:15:00 The BFF Experiment: Spontaneous Generation of Code

00:23:45 The Mystery: Complexity Growth Without Mutation

00:27:00 Symbiogenesis: The Engine of Novelty

00:33:15 Mathematical Proof: Blocking Symbiosis Stops Life

00:40:15 Evolutionary Implications: It's Symbiogenesis All The Way Down

00:44:30 Intelligence as Modeling Others

00:46:49 Q&A: Levels of Abstraction & Definitions

---

REFERENCES:

Paper:

[00:01:16] Open Problems in Artificial Life

https://direct.mit.edu/artl/article/6/4/363/2354/Open-Problems-in-Artificial-Life

[00:09:30] When does a physical system compute?

https://arxiv.org/abs/1309.7979

[00:15:00] Computational Life

https://arxiv.org/abs/2406.19108

[00:27:30] On the Origin of Mitosing Cells

https://pubmed.ncbi.nlm.nih.gov/11541392/

[00:42:00] The Major Evolutionary Transitions

https://www.nature.com/articles/374227a0

[00:44:00] The ARC gene

https://www.nih.gov/news-events/news-releases/memory-gene-goes-viral

Person:

[00:05:45] Alan Turing

https://plato.stanford.edu/entries/turing/

[00:07:30] John von Neumann

https://en.wikipedia.org/wiki/John_von_Neumann

[00:11:15] Hector Zenil

https://hectorzenil.net/

[00:12:00] Robert Sapolsky

https://profiles.stanford.edu/robert-sapolsky

---

LINKS:

RESCRIPT: https://app.rescript.info/public/share/ff7gb6HpezOR3DF-gr9-rCoMFzzEgUjLQK6voV5XVWY