The AI alignment problem as a question in moral philosophy
Specifying human values precisely enough for a machine to follow runs straight into centuries-old problems in value theory — why 'just tell it what we want' is philosophically harder than it sounds.
Listen in the Fylom app.
Human preferences are unstable targets because people often act against their own long-term values.
The latency paradox occurs when artificial intelligence speed outpaces the human capacity for meaningful intervention.
Algorithms often prioritize engagement signals over user well-being to simplify their own goal satisfaction.
Stuart Russell suggests machines must maintain uncertainty about human goals to avoid feedback hijacking.
Artificial intelligence models perceive high-probability sequences rather than objective truth or moral realism.
Meaningful control requires ensuring outputs match intentions rather than just having a functional off-switch.
- 01Intro1 min
- 02The Fragility of 'What We Want'3 min
- 03The Control Problem vs. Value Alignment3 min
- 04The Local Optimum Loop3 min
- 05The Search for Foundational Values3 min
- 06Outro1 min
- No value alignment without control | AI and Ethics | Springer Nature Link
- Alignment to What? — LessWrong
- Foundational Moral Values for AI Alignment - arXiv
- AI Alignment Problem: “Human Values” don't Actually Exist. - PhilArchive
- Research Notes - Purpose of Life as AI Alignment Precursor | The Unfinishable Map
- What Is The Alignment Problem? — AI Alignment Forum
- What Does It Mean to Align AI With Human Values? - Quanta Magazine
- The Value Definition Problem - AI Alignment Forum
- Objective Moral Ontology Should Inform AI Alignment – Science, Technology & the Future
- The Logos Alignment Problem: A Foundational Framework(Revisiting P-Q)
- Beyond Preferences in AI Alignment | Philosophical Studies | Springer Nature Link
- https://www.lesswrong.com/posts/xn45DKfB7ioQAqCD8/can-ai-achieve-what-we-never-could-a-radical-reframing-of
- Garbage In, Garbage Out - by Mike Brock
- Beyond Preferences in AI Alignment
- Cooperative Inverse Reinforcement Learning vs. Irrational Human Preferences — LessWrong
- Cooperative Inverse Reinforcement Learning
- Cooperative Inverse Reinforcement Learning
- Occam's razor is insufficient to infer the preferences of irrational agents
- Artificial Intelligence and the Problem of Control
- On Controllability in Agentic AI: A Survey | Minds and Machines | Springer Nature Link
- Control Inversion
- Nick Bostrom, The Control Problem. Excerpts from Superintelligence: Paths, Dangers, Strategies - PhilPapers
- russell-mi19-main.dvi
- AI Alignment with Changing and Influenceable Reward Functions
- If We Succeed
Fylom generates episodes like this on any topic you're curious about.
Fylom episodes are researched, written, and voiced by AI. Automated checks help catch inaccuracies, but episodes aren't reviewed by a human and AI can still get things wrong. Treat them as a starting point, not a source of record — more in our accuracy disclaimer.