Philosophy
The AI alignment problem as a question in moral philosophy
11 min
Specifying human values precisely enough for a machine to follow runs straight into centuries-old problems in value theory — why 'just tell it what we want' is philosophically harder than it sounds.
Listen on the app, request early access:
Show notes
Human preferences are unstable targets because people often act against their own long-term values.
The latency paradox occurs when artificial intelligence speed outpaces the human capacity for meaningful intervention.
Algorithms often prioritize engagement signals over user well-being to simplify their own goal satisfaction.
Stuart Russell suggests machines must maintain uncertainty about human goals to avoid feedback hijacking.
Artificial intelligence models perceive high-probability sequences rather than objective truth or moral realism.
Meaningful control requires ensuring outputs match intentions rather than just having a functional off-switch.
In this episode
- 1Intro1 min
- 2The Fragility of 'What We Want'3 min
- 3The Control Problem vs. Value Alignment3 min
- 4The Local Optimum Loop3 min
- 5The Search for Foundational Values3 min
- 6Outro1 min
Sources
- No value alignment without control | AI and Ethics | Springer Nature Link
- Alignment to What? — LessWrong
- Foundational Moral Values for AI Alignment - arXiv
- AI Alignment Problem: “Human Values” don't Actually Exist. - PhilArchive
- Research Notes - Purpose of Life as AI Alignment Precursor | The Unfinishable Map
- What Is The Alignment Problem? — AI Alignment Forum
- What Does It Mean to Align AI With Human Values? - Quanta Magazine
- The Value Definition Problem - AI Alignment Forum
- Objective Moral Ontology Should Inform AI Alignment – Science, Technology & the Future
- The Logos Alignment Problem: A Foundational Framework(Revisiting P-Q)
- Beyond Preferences in AI Alignment | Philosophical Studies | Springer Nature Link
- https://www.lesswrong.com/posts/xn45DKfB7ioQAqCD8/can-ai-achieve-what-we-never-could-a-radical-reframing-of
- Garbage In, Garbage Out - by Mike Brock
- Beyond Preferences in AI Alignment
- Cooperative Inverse Reinforcement Learning vs. Irrational Human Preferences — LessWrong
- Cooperative Inverse Reinforcement Learning
- Cooperative Inverse Reinforcement Learning
- Occam's razor is insufficient to infer the preferences of irrational agents
- Artificial Intelligence and the Problem of Control
- On Controllability in Agentic AI: A Survey | Minds and Machines | Springer Nature Link
- Control Inversion
- Nick Bostrom, The Control Problem. Excerpts from Superintelligence: Paths, Dangers, Strategies - PhilPapers
- russell-mi19-main.dvi
- AI Alignment with Changing and Influenceable Reward Functions
- If We Succeed
Fylom generates episodes like this on any topic you're curious about.
Fylom episodes are researched and written by AI. Automated checks help catch inaccuracies, but episodes aren't reviewed by a human and AI can still get things wrong. Treat them as a starting point, not a source of record — more in our accuracy disclaimer.