Bill Gates on dangers of artificial intelligence: ‘I don’t understand why some people are not concerned’

Bill Gates on dangers of artificial intelligence: ‘I don’t understand why some people are not concerned’

By Peter Holley January 29 at 10:55 AM  

Bill Gates joined Reddit for an AMA on Wednesday. (Tobias Schwarz/AFP/Getty Images)

Bill Gates is a passionate technology advocate (big surprise), but his predictions about the future of computing aren't uniformly positive.

During a wide-ranging Reddit "Ask me Anything" session -- one that touched upon everything from his biggest regrets to his favorite spread to lather on bread -- the Microsoft co-founder and billionaire philanthropist outlined a future that is equal parts promising and ominous.

Midway through the discussion on Wednesday, Gates was asked what personal computing will look like in 2045. Gates responded by asserting that the next 30 years will be a time of rapid progress.

"Even in the next 10 problems like vision and speech understanding and translation will be very good," he wrote. "Mechanical robot tasks like picking fruit or moving a hospital patient will be solved. Once computers/robots get to a level of capability where seeing and moving is easy for them then they will be used very extensively."

He went on to highlight a Microsoft project known as the "Personal Agent," which is being designed to help people manage their memory, attention and focus. "The idea that you have to find applications and pick them and they each are trying to tell you what is new is just not the efficient model - the agent will help solve this," he said. "It will work across all your devices."

The response from Reddit users was mixed, with some making light of Gates's revelation ("Clippy 2.0?," wrote one user) -- and others sounding the alarm.

"This technology you are developing sounds at its essence like the centralization of knowledge intake," a Redditor wrote. "Ergo, whomever controls this will control what information people make their own. Even today, we see the daily consequences of people who live in an environment that essentially tunnel-visions their knowledge."

Shortly after, Gates was asked how much of an existential threat superintelligent machines pose to humans.

The question has been at the forefront of several recent discussions among prominent futurists. Last month, theoretical physicist Stephen Hawking said artificial intelligence "could spell the end of the human race."

[Why the world’s most intelligent people shouldn’t be so afraid of artificial intelligence]

Speaking at the MIT Aeronautics and Astronautics department’s Centennial Symposium in October, Tesla boss Elon Musk referred to artificial intelligence as "summoning the demon."

I think we should be very careful about artificial intelligence. If I were to guess like what our biggest existential threat is, it’s probably that. So we need to be very careful with the artificial intelligence. Increasingly scientists think there should be some regulatory oversight maybe at the national and international level, just to make sure that we don’t do something very foolish. With artificial intelligence we are summoning the demon. In all those stories where there’s the guy with the pentagram and the holy water, it’s like yeah he’s sure he can control the demon. Didn't work out.

British inventor Clive Sinclair has said he thinks artificial intelligence will doom mankind.

"Once you start to make machines that are rivaling and surpassing humans with intelligence, it's going to be very difficult for us to survive," he told the BBC. "It's just an inevitability."

After gushing about the immediate future of technology in his Reddit AMA, Gates aligned himself with the AI alarm-sounders.

"I am in the camp that is concerned about super intelligence," Gates wrote. "First the machines will do a lot of jobs for us and not be super intelligent. That should be positive if we manage it well. A few decades after that though the intelligence is strong enough to be a concern. I agree with Elon Musk and some others on this and don't understand why some people are not concerned."

Once he finished addressing the potential demise of humankind, Gates got back to answering more immediate, less serious questions, like revealing his favorite spread to put on bread.

"Butter? Peanut butter? Cheese spread?" he wrote. "Any of these."

The Microsoft co-founder's comments on AI came shortly after the managing director of Microsoft Research's Redmond Lab said the doomsday declarations about the threat to human life are overblown.

"There have been concerns about the long-term prospect that we lose control of certain kinds of intelligences," Eric Horvitz said, according to the BBC. "I fundamentally don't think that's going to happen. I think that we will be very proactive in terms of how we field AI systems, and that in the end we'll be able to get incredible benefits from machine intelligence in all realms of life, from science to education to economics to daily life."

Horvitz noted that "over a quarter of all attention and resources" at Microsoft Research are focused on artificial intelligence.

[This story, originally posted on Wednesday, has been updated.]

http://www.washingtonpost.com/blogs/the-switch/wp/2015/01/28/bill-gates-on-dangers-of-artificial-intelligence

“behavioural economics” 。 Make policy for real, not ideal, humans Martin Wolf

Herbert Simon 是行為經濟學大師，從不同意"理性 vs 感情"的二分法，而是探討它們在人之所以為人的關係。

December 16, 2014 7:19 pm

Make policy for real, not ideal, humans

Martin Wolf

Many believe dysfunctional behaviour in finance is due solely to distorted incentives

O

ut of the crooked timber of humanity, no straight thing was ever made. This famous remark of the German philosopher, Immanuel Kant, is particularly relevant to economists. “Homo economicus” is far-sighted, rational and self-interested. Real human beings are none of these things. We are bundles of emotions, not calculating machines. This matters.

The World Bank’s latest World Development Report examines this territory. It notes that “behavioural economics” alters our view of human behaviour in three ways: first, most of our thinking is not deliberative, but automatic; second, it is socially conditioned; and, third, it is shaped by inaccurate mental models.

High quality global journalism requires investment. Please share this article with others using the link below, do not cut & paste the article. See our Ts&Cs and Copyright Policy for more detail. Email ftsales.support@ft.com to buy additional rights. http://www.ft.com/cms/s/0/8d3b623e-8099-11e4-872b-00144feabdc0.html#ixzz3PWj61ZNp

The Nobel laureate, Daniel Kahneman, explored the idea that we think in two different ways in his 2011 book Thinking, Fast and Slow . The need for an automatic system is evident. Our ancestors did not have the time to work out answers to life’s challenges from first principles. They acquired automatic responses and a cultural predisposition towards rules of thumb. We inherited both these traits. Thus, we are influenced by how a problem is framed.

Another characteristic is “confirmation bias” — the tendency to interpret new information as support for pre-existing beliefs. We also suffer from loss aversion, fierce resistance to losing what one already has. For our ancestors, on the margin of survival, that made sense.

The fact that humans are intensely social is clear. Even the idea that we are autonomous is itself socially conditioned. We are also far from solely self-interested. A bad consequence of the power of norms is that societies may be stuck in destructive patterns of behaviour. Nepotism and corruption are examples. If they are entrenched, it may be difficult (or dangerous) for individuals not to participate. But social norms can also be valuable. Trust is a valuable norm. It rests on one of humanity’s strongest behaviours: conditional co-operation. People will punish free-riders even when it costs them to do so. This trait strengthens groups and so must raise members’ ability to survive.

Mental models are essential. Some seem to be inbuilt; and some can be damaging — as well as productive. Ideas about “us” and “them”, reinforced by social norms, may well lead to results that range from the merely unfair to the catastrophic. Equally important may be mental models that create self-fulfilling expectations of who will succeed and who will fail. There is evidence, notes the WDR, that mental models rooted in history may shape people’s view of the world for centuries: caste is an example. Such mental models survive because they are reproduced socially and become part of the automatic rather than the deliberative system. They influence not just our perceptions of others, but perceptions of ourselves.

To illustrate the relevance of these realities, the report analyses the policy challenges of poverty, early childhood development, household finance, productivity, health and climate change.

On household finance, for example, the report notes that it makes a difference whether would-be borrowers are told explicitly how much more expensive is a payday loan than an equivalent loan on a credit card. Revealing the status of low-caste boys in a mixed-caste classroom depresses the performance of students from lower castes compared with what happens if caste is not revealed. The boys respond to how they are presented. Again, poverty is not just a lack of material resources:it undermines the ability to think deliberately.

We are all made of Kant’s crooked timber: nobody has godlike wisdom and self-control

Tweet this quote

The way people think may also affect their productivity. An example is the benefits of contracts that penalise a worker for failing to meet the output targets she has chosen for herself. This is a way of closing the gap between good intentions and actual performance, such as when we agree to put money in the swearbox when we curse. We often disappoint ourselves. We may wish to bind ourselves to better behaviour, like Odysseus to his mast.

Health creates vital examples. One is the importance of mental models. An obvious one is the anti-vaccination hysteria. Another, illustrated by the WDR, is the tendency of poor women to believe that the right treatment for diarrhoea is to cut fluid intake, to stop their child “leaking”. Another is the tendency of people to be put off by even a very small charge for health products. The explanation for the reluctance to pay anything may, it suggests, be because free provision underpins the norm that everybody ought to take the medicine.

These then are intriguing examples of a more nuanced approach to policy. Another area where a narrow focus only on incentives is likely to be misleading is financial regulation. Many economists believe that dysfunctional behaviour in financial markets is due solely to distorted incentives: deposit insurance, the perception that institutions are “too big to fail” and a host of other explicit and implicit subsidies. Equally important, however, are behavioural norms, such as the view that the primary duty of bankers is to themselves not their customers; or inappropriate mental models, such as the widespread pre-crisis belief that house prices could not fall across the US. Regulation needs to be built on an understanding of such human frailties. It must focus on norms and groupthink, as well as on distorted incentives.

How far should policy be based on these perceptions, particularly since those who make policy are, as the WDR admits, prone to all sorts of biases in their own decision-making? We are all made of Kant’s crooked timber: nobody has godlike wisdom and self-control.

Yet policy must be made. It is surely better to make well-informed and realistic policy than base it on a grossly simplistic view of our true capacities. Moreover, nudging people in the direction they already want to go — by encouraging them to save, learn, behave healthily or bring up their children better — is hardly a gross violation of liberty. Yet encouragement should not slide too easily into coercion. Adults are not to be treated as children. That, too, is a social norm and quite a fundamental one.

martin.wolf@ft.com

-------------------------------------------

Letter in response to this column:

Look elsewhere for model behaviour / From Arnold Packer

Copyright The Financial Times Limited 2015. You may share using our article tools.
Please don't cut articles from FT.com and redistribute by email or post to the web.

2015年01月20日 07:45 AM

經濟政策的人性基礎

英國《金融時報》專欄作家 馬丁·沃爾夫

“人性這根曲木，絕然造不出任何筆直的東西。”經濟學家尤其應該聽聽德國哲學家伊曼努爾•康德(Immanuel Kant)的這句名言。“經濟人”有遠見、理性而且自私。真正的人跟經濟人完全不一樣。我們感情充沛，不是精於計算的機器。這非常關鍵。

世界銀行(World Bank)在其發布的最新一期《世界發展報告》(World Development Report)中研究了該領域。它指出，“行為經濟學”從3個方面改變了我們對人類行為的看法：首先，我們的大部分想法並非經過深思熟慮，而是自動產生的；其次，它受到社會的制約；第三，它脫胎於不精確的思維模式。

諾貝爾獎得主丹尼爾•卡內曼(Daniel Kahneman)在其2011年出版的《思維，快慢有別》(Thinking, Fast and Slow)一書中探討了人類有兩種不同思維方式的觀點。擁有一種自動反應的思維體系顯然是必要的。我們的祖先沒時間從基本原則中尋找解決生活挑戰的辦法。他們獲得了自動反應的本領，也形成了聽從經驗法則的文化傾向。這兩種特質我們都繼承了。因此，我們會受問題提出方式的影響。

另一個特徵是“確認偏誤”(confirmation bias)——即將新信息解讀為能夠支持已有觀點的傾向。我們還有“厭惡損失”(loss aversion)的傾向，即強烈抗拒失去我們已經擁有的東西。對我們只能勉強維持生存的祖先來說，這種傾向非常明智。

人類具有強烈的社會性，這是顯而易見的。就連我們是獨立的個體這個想法本身，也是受到社會制約的。我們也絕非完全自私。社會規範的強大威力帶來的一個糟糕後果是，社會可能陷入消極的行為模式。裙帶關系和腐敗就是例證。如果裙帶關系或腐敗根深蒂固，個人不參與其中或許就很困難（或危險）。但社會規範也可能是有用的。信任是一種有用的規範。它依賴於人類最擅長的行為之一：有條件的合作。人們將會懲罰搭便車者，即便懲罰他們要付出代價。這種特質增強了團隊的凝聚力，從而肯定會提高團隊成員的生存能力。

思維模式非常關鍵。有些似乎是人內在固有的，此外，有些模式可能同時具有創造性和破壞性。關於“我們”和“他們”的觀念如果經過社會規範的強化，很可能導致各種各樣的結果，從僅僅是不公平的，到釀成災難的。有些思維模式能夠產生有關誰將成功、誰將失敗的預期（並且這種預期具有自我實現的能力），這些思維模式同樣重要。《世界發展報告》指出，有證據表明，有深厚歷史根源的思維模式可能決定人們數百年的世界觀：種姓制度就是一個例子。此類思維模式之所以經久不衰，是因為它們在社會上不斷繁殖，成為了一種自動（而非經過深思熟慮）的反應。它們不僅影響我們對其他人的看法，還影響我們對自己的看法。

為了表明這些研究與現實密切相關，該報告還分析了貧窮、幼兒期發展、家庭財務、生產率、健康和氣候變化的政策挑戰。

例如，在家庭財務方面，該報告指出，潛在借款者是否被明確告知發薪日貸款(payday loan,一種小額、短期的高利貸,用於貸款人下一次發薪之前臨時急用——譯者註)與等額信用卡貸款相比有多麽昂貴，結果將大為不同。如果在一個種姓混雜的班級里讓大家知道誰是低種姓學生，那麽低種姓學生的表現就會不如其身份沒有暴露時的表現。別人如何介紹自己，會影響這些男孩的表現。同樣，貧窮不僅僅是物質資源的匱乏：它還削弱一個人審慎思考的能力。

人們的思考方式也可能影響他們的生產效率。一個例子是，簽訂這樣的合同有不錯的效果：約定工人如果未能完成自己選擇的產量目標就會受到懲罰。要消除良好意願與實際表現的差距，這是一種方法，比如我們同意只要我們罵人就要往罰款箱里投錢。我們常常讓自己失望。我們可能希望約束自己、迫使自己表現得更好，就像奧德修斯(Odysseus)讓人把自己捆在桅桿上（以抵擋海妖歌聲的誘惑）那樣。

健康領域產生了一些重要的例子，其中之一是思維模式的重要性。一種明顯的思維模式是非理性地反對接種疫苗。《世界發展報告》舉出的另一種思維模式是，貧窮的婦女往往認為，腹瀉的正確治療方法是減少液體的攝入，這樣他們的孩子就不再“拉稀”。還有一種思維模式是，人們往往不願購買收費的健康產品，哪怕金額極低。報告稱，人們之所以一分錢都不願花，可能是因為，免費提供才符合有難同當的社會規範。

因此這些有趣的例子表明瞭政策手段有必要更加細致。另一個關鍵領域是金融監管，在這一領域中，如果僅僅關註於激勵就可能導致誤解。許多經濟學家相信，金融市場中的失靈行為完全是扭曲的動機造成的，比如：存款保險、機構“太大而不能倒閉”的觀念，以及其他眾多顯性和隱性補貼。然而，同樣重要的是社會行為規範，比如認為銀行家主要應該對他們自己、而不是客戶負責的觀點；或者不適當的思維模式，比如危機前人們普遍認為，美國的房價不會普跌。監管需要建立在瞭解此類人性弱點的基礎之上。它必須不僅關註於扭曲的動機，還關註於社會規範和群體迷思(groupthink)。

政策應該在多大程度上以這些觀念為基礎？尤其是正如《世界發展報告》承認的那樣，那些制定政策的人士往往對他們自己的決策存在各種各樣的偏見。我們全都由康德所說的“曲木”製成：沒有人有上帝那樣的智慧和自製力。

然而政策是必須制定的。充分瞭解相關信息、從現實出發制定政策，肯定要比基於對我們真實能力的過分簡單化理解制定政策要好。此外，通過鼓勵人們儲蓄、學習、過健康生活或更好地撫育子女，推動人們朝他們本來就希望的方向前進，這不能說是嚴重侵犯自由。然而，鼓勵不應輕易地演變為強制。不應把成年人當做小孩對待。那也是一種社會規範，而且還是相當根深蒂固的一種。譯者/鄒策