Send us Fan Mail

Diffusion models have become the foundation of modern generative AI, powering state-of-the-art systems for image generation, video synthesis, protein design, and more. In the first episode of this series, we introduced generative modeling as a sampling problem and developed the probabilistic framework needed to think about how models learn complex data distributions.

In this second episode, we move into the core mathematics behind flow matching and diffusion modeling, continuing through the MIT lecture notes An Introduction to Flow Matching and Diffusion Models by Peter Holderrieth and Ezra Erives.

We explore how a simple, easy-to-sample distribution such as Gaussian noise can be continuously transformed into a complicated data distribution, and how this idea leads naturally to both deterministic and stochastic approaches to generative modeling.

Topics covered include:

• The basic idea behind flow matching
• Transforming a noise distribution into a data distribution
• Probability paths and time-dependent distributions
• Vector fields and continuous-time generative processes
• Ordinary differential equations (ODEs) in generative modeling
• Stochastic differential equations (SDEs) and the role of injected noise
• The relationship between diffusion models and continuous flows
• Forward and reverse diffusion processes
• How trained diffusion and flow models are used during inference
• Numerical inference algorithms for solving the resulting ODEs and SDEs
• Why the number of inference steps affects speed and generation quality
• How these mathematical formulations connect to practical modern diffusion systems

The goal of this episode is to build intuition for what a diffusion model is actually doing mathematically: rather than simply “removing noise,” we can view generation as learning a dynamical process that transports probability from a simple starting distribution toward the complex distribution of real data.

This framework provides the foundation for later discussions of score matching, denoising objectives, probability flow ODEs, modern sampling algorithms, and the architectures used in state-of-the-art generative models.

Lecture Notes:
https://diffusion.csail.mit.edu/docs/lecture-notes.pdf

Website:
https://arkitekt-ai.com

Contact:
support@arkitekt-ai.com

Send us Fan Mail

Diffusion models have become the foundation of modern generative AI, powering state-of-the-art systems for image generation, video synthesis, protein design, and more. But behind the impressive demos lies a surprisingly elegant mathematical framework.

In this first episode of a multi-part series, we begin working through the excellent MIT lecture notes An Introduction to Flow Matching and Diffusion Models by Peter Holderrieth and Ezra Erives. Rather than jumping straight into denoising or neural networks, we focus on the fundamental question: What does it actually mean to generate data?

Topics covered include:

• Why generative modeling is fundamentally a sampling problem
• Data distributions and probability densities
• Representing images, videos, and other data as vectors
• Conditional generation and prompts
• Why diffusion models are framed as learning distributions rather than memorizing examples
• An overview of where the mathematics is heading in the remainder of the course

This episode is intended for anyone who wants to understand diffusion models from first principles, building the mathematical intuition needed for later discussions on ODEs, SDEs, flow matching, score matching, and modern diffusion architectures.

Lecture Notes:
https://diffusion.csail.mit.edu/docs/lecture-notes.pdf

Website:
https://arkitekt-ai.com

Contact:
support@arkitekt-ai.com

Send us Fan Mail

In this episode, we explore the paper "Generative Recursive Reasoning (GRAM)," a fascinating new approach to AI reasoning co-authored by Yoshua Bengio and researchers from Mila and Samsung AI.

Most modern AI systems reason by generating more tokens. GRAM takes a different approach: instead of extending a chain of thought, it repeatedly refines an internal latent state. The key innovation is introducing probabilistic reasoning trajectories, allowing the model to explore multiple possible solutions simultaneously rather than committing to a single deterministic path.

We discuss:

• Recursive Reasoning Models (RRMs) and why they differ from traditional transformers
• The limitations of deterministic latent reasoning
• How GRAM introduces stochastic latent trajectories
• Variational inference and the roles of pθ and qϕ
• Multi-hypothesis reasoning and inference-time scaling
• Results on Sudoku, ARC-AGI, N-Queens, and other structured reasoning benchmarks
• Why latent-space reasoning may become an alternative to longer chain-of-thought prompting

The paper also demonstrates unconditional generation capabilities, suggesting a path toward reasoning systems that can both solve problems and generate structured outputs through recursive latent computation.

PDF:
Generative Recursive Reasoning

Arkitekt AI:
https://arkitekt-ai.com

Contact:
support@arkitekt-ai.com