Benchmarking AI with Stanislav Khromov

In this episode, Stanislav Khromov joins the Svelte Radio team to discuss his work on Svelte Bench, a

benchmarking tool that scientifically measures how well different LLMs understand and write Svelte 5 code.

The conversation explores the challenges of AI-assisted coding with Svelte 5, the development of the official

Svelte MCP (Model Context Protocol) that provides LLMs with documentation and auto-fixing capabilities, and

the surprising performance differences between major AI providers. Stanislav also shares insights on working

with AI tools, the future of local models, and the economics of AI coding assistants.

Notes

• Stanislav Khromov
• Svelte Bench
  • GitHub
• Svelte MCP
• Anthropic Spending VC Money

Unpopular Opinions

• Kevin: Serverless is overrated
• Stanislav: OpenAI has the worst models of the big three
• Antony: Heat Pumps are bad
  • Technology Connections
  • Tech Ingredients

Picks

• Kevin: Kasai Hibachi Grill (now tested, and can confirm, it is PENG!)
  Stanislav: Ball x Pit
  Antony: Saunas By the Sea