Joe Carlsmith Audio
Audio versions of essays by Joe Carlsmith. Philosophy, futurism, and other topics. Text versions at joecarlsmith.com.
Joe Carlsmith Audio
Does scheming lead to adequate future empowerment? (Section 2.3.1.2 of "Scheming AIs")
•
Joe Carlsmith
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
This is section 2.3.1.2 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”
Text of the report here: https://arxiv.org/abs/2311.08379
Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
2.3.1.2 Adequate future empowerment
2.3.1.2.1 When is the “pay off” supposed to happen?
2.3.1.2.2 Even if the model’s values survive this generation of training, will they survive long
2.3.1.2.3 Will escape/take-over be suitably likely to succeed?
2.3.1.2.4 Will the time horizon of the model’s goals extend to cover escape/take-over?
2.3.1.2.5 Will the model’s values get enough power after escape/takeover?
2.3.1.2.6 How much does the model stand to gain from not training-gaming?
2.3.1.2.7 How “ambitious” is the model?
2.3.1.3 Overall assessment of the classic goal-guarding story