MLG 034 Large Language Models 1

MLG 034 Large Language Models 1

From Machine Learning Guide by OCDevel

May 7, 2025 · 51 min · Season 1 · Episode 58

About this episode

This episode explains advancements in language models and the scaling laws that influence their performance.

Explains language models (LLMs) advancements. Scaling laws - the relationships among model size, data size, and compute - and how emergent abilities such as in-context learning, multi-step reasoning, and instruction following arise once certain scaling thresholds are crossed. The evolution of the transformer architecture with Mixture of Experts (MoE), describes the three-phase training process culminating in Reinforcement Learning from Human Feedback (RLHF) for model alignment, and explores advanced reasoning techniques such as chain-of-thought prompting which significantly improve complex task performance. Links Notes and resources at ocdevel.com/mlg/mlg34 Build the future of multi-agent software with AGNTCY Try a walking desk stay healthy & sharp while you learn & code Transformer Foundations and Scaling Laws Transformers : Introduced by the 2017 "Attention is All You Need" paper, transformers allow for parallel training and inference of sequences using self-attention, in contrast to the sequential nature of RNNs. Scaling Laws : Empirical research revealed that LLM performance improves predictably as model size (parameters), data size (training tokens), and compute are…

People in this episode

Host: OCDevel

Topics covered

  • language models
  • scaling laws
  • transformer architecture
  • Reinforcement Learning
  • multi-step reasoning
  • in-context learning

Keywords

  • large language models
  • scaling laws
  • transformers
  • Reinforcement Learning from Human Feedback
  • chain-of-thought prompting

Mentioned in this episode

Organizations: DeepMind

Products: GPT-3

Books & works: Attention is All You Need

More episodes of Machine Learning Guide

Explore listener stats, chart rankings, contacts and more on the Machine Learning Guide podcast page.