Climbing the intelligence stairway

In recent years he has taken an interest in the ethical and safety aspects of artificial intelligence, travelling the world to talk to different experts, from philosophers to researchers to actual artificial intelligence programmers.

He argues that since evolution produced humans approximately 100,000 years ago, a significant change has occurred and humans have reached their optimum stage of evolution. He suggests that human-driven technological progress has largely replaced evolution as the main shaper of the future.

Tallinn believes we are witnessing a cascade-like pattern in which we are producing machines whose ability to control the future exceeds that of the process that produced them. He calls this pattern the "Intelligence Stairway".

He says this raises a major question: does the production of computers which are smarter than their creators constitute a similar transition to when evolution produced humans?

If the answer is yes, then we should treat the possible emergence of such artificial general intelligence (AGI) as the end of human-driven technological progress, and beginning of a new phase: an AGI-driven "intelligence explosion".

In turn, this leads to an uneasy conclusion that this "intelligence explosion" might effectively manifest itself as a sudden global ecological catastrophe suggest Tallinn.

There are currently two different approaches to avoiding the catastrophe. One is to prove that the path of the "intelligence explosion" will be favourable to humanity. The other approach is to limit artificial intelligence devices to narrow domains, so they would lack the ability to replace humans as the drivers of technological progress.

Tallinn argues that the latter approach is more pragmatic, so we need a co-ordinated effort to establish and enforce a "practical safety protocol" for artificial intelligence developers in order to make sure that they would not produce AGIs by accident.

He is also one of the programmers behind Kazaa file sharing platform. He is a co-founder of the Cambridge Centre for the Study of Existential Risk. He describes himself as a physicist/singularitarian/hacker/investor