Tech & AI
How large language models actually work
11 min
Beyond the cliché that they just predict the next word: explore what transformers and attention really do, how training turns text into capability, and where the surprising behavior comes from.
Listen on the app, request early access:
Show notes
Transformers process entire sentences simultaneously rather than word-by-word to maintain long-range context.
Self-attention resolves pronoun ambiguity by mathematically linking words like it to their specific subjects.
Positional encoding uses sine and cosine waves to help models distinguish word order in a sentence.
Reinforcement learning from human feedback transforms raw text predictors into helpful conversational assistants.
The softmax layer converts internal numerical scores into a probability distribution across thousands of unique tokens.
Temperature settings allow users to toggle between strictly factual responses and more creative, random outputs.
In this episode
- 1Intro1 min
- 2The Transformer Revolution2 min
- 3The Mechanics of Self-Attention3 min
- 4From Training to Capability3 min
- 5The Inference Engine3 min
- 6Outro1 min
Sources
- LLMs: What's a large language model? | Machine Learning
- How LLMs Work: Transformers Explained Step-by-Step - machinelearningplus
- How Large Language Models Actually Work - Skillcef
- How Large Language Models Work in 2026 — A Complete Guide
- How Large Language Models Actually Work Under the Hood | Let's Data Science
- How Does an LLM Actually Work? (ChatGPT and Claude, Explained Without Math) | Build This Now
- How Large Language Models Work: Transformers Explained
- How Large Language Models Work: What's Actually Inside ChatGPT - I2notes
- How LLMs Actually Work: The Hidden State Is the Story
- How LLMs Actually Work | Tech With Nikola
- Transformer: A Novel Neural Network Architecture for Language Understanding
- Attention is All you Need
- Attention Is All You Need
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
- No, AI isn't a 'fancy autocomplete' - Andrew Kantor
- Attention is All You Need
- The Annotated Transformer
- Section 2.3: QKV, Scaled Dot-Product & Causal Masking
- [draft] Note 10: Self-Attention & Transformers selectfont10plus2minus5plus36plus3minus34plus2minus8plus2minus44plus2minus4plus2minus8plus2minus44plus2minusCS 224n: Natural Language Processing with Deep Learning
- Transformer (deep learning)
Fylom generates episodes like this on any topic you're curious about.
Fylom episodes are researched and written by AI. Automated checks help catch inaccuracies, but episodes aren't reviewed by a human and AI can still get things wrong. Treat them as a starting point, not a source of record — more in our accuracy disclaimer.