Smarter Than Us_ The Rise of Machine Intelligence_Stuart Armstrong.pdf

7+(5,6(2)0$&+,1(,1 7 ( / / , * ( 1 & ( 60$57(57+$1 8 6 -/1
,/
,-/,"  SmarterThanUs SmarterThanUs The Rise of Machine Intelligence Stuart ArmstrongMIRI Stuart Armstrong is a James Martin Research Fellow at the Future of Humanity Institute at Oxford University. His research focuses on formal decision theory, the risks and possibilities of Artificial Intelligence, the long term potential for intelligent life (and the difficulties of predicting this), and anthropic (self-locating) probability. Written by Stuart Armstrong Published in 2014 Machine Intelligence Research Institute Berkeley 94704 United States of America intelligence.org Released under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license. CC BY-NC-SA 3.0 isbn-10: 1939311063 isbn-13: 978-1-939311-06-1 (pdf) The Machine Intelligence Research Institute gratefully acknowledges the generous support of all those involved in the publication of this book. Cover photo credit: Google/Connie Zhou. Acknowledgments I wish to acknowledge the help and support of the Future of Humanity Institute, the Oxford Martin School, and the Machine Intelligence Re- search Institute, as well as the individual advice of Nick Bostrom, Seán Ó hÉigeartaigh, Eliezer Yudkowsky, Kaj Sotala, Luke Muehlhauser, Vincent C. Müller, Anders Sandberg, Lisa Makros, Daniel Dewey, Eric Drexler, Nick Beckstead, Cathy Douglass, and Miriam, Maia, and Kipper Armstrong. Contents Acknowledgments v Contents vii 1 Terminator versus the AI 1 2 Strength versus Intelligence 5 3 What Is Intelligence? Can We Achieve It Artificially?8 4 How Powerful Could AIs Become? 12 5 Talking to an Alien Mind 17 6 Our Values Are Complex and Fragile 22 7 What, Precisely, Do We Really (Really) Want? 28 8 We Need to Get It AllExactlyRight 32 9 Listen to the Sound of Absent Experts 38 10 A Summary 43 11 That’s WhereYouCome In . . . 47 About the Author 50 Bibliography 51 1 Terminator