• Dynamic Token Merging for Efficient Byte-level Language Models with Julie Kallini - #724

  • Mar 24 2025
  • Duración: 51 m
  • Podcast

Dynamic Token Merging for Efficient Byte-level Language Models with Julie Kallini - #724

Escúchala gratis

Ver detalles del espectáculo

  • Resumen

  • Today, we're joined by Julie Kallini, PhD student at Stanford University to discuss her recent papers, “MrT5: Dynamic Token Merging for Efficient Byte-level Language Models” and “Mission: Impossible Language Models.” For the MrT5 paper, we explore the importance and failings of tokenization in large language models—including inefficient compression rates for under-resourced languages—and dig into byte-level modeling as an alternative. We discuss the architecture of MrT5, its ability to learn language-specific compression rates, its performance on multilingual benchmarks and character-level manipulation tasks, and its performance and efficiency. For the “Mission: Impossible Language Models” paper, we review the core idea behind the research, the definition and creation of impossible languages, the creation of impossible language training datasets, and explore the bias of language model architectures towards natural language. The complete show notes for this episode can be found at https://twimlai.com/go/724.
    Más Menos
Más Menos
adbl_web_global_use_to_activate_webcro768_stickypopup

Lo que los oyentes dicen sobre Dynamic Token Merging for Efficient Byte-level Language Models with Julie Kallini - #724

Calificaciones medias de los clientes

Reseñas - Selecciona las pestañas a continuación para cambiar el origen de las reseñas.

Reseñas de Audible.com

Opiniones de Amazon
No hay comentarios disponibles