Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
93 lines (66 sloc) 5.19 KB

Three Skills That Super-Intelligence Will Have but No Humans Have

by Sven Nilsen, 2018

Here is a list of 3 skills that artificial super-intelligence is almost surely going to have but no humans have or will be able to have without modifying our brains in some way:

1. Responding Quickly to Data Generated by Computer Programs

When humans are programming today, we have to prepare in advance for what kind of data we would like to collect, how we want to analyze it, and how we want to change some behavior of the program depending on the analysis.

For example, if we are creating a game, we need to figure out what rules the game should have. When the rules do not produce the desired output we wanted, we have to stop or pause the program, change the code and then continue.

This is a limitating factor of human intelligence, because much of the intuition we have about what to do, comes from looking at data before making up our minds of what to do.

The human brain is the bottleneck of such iteration cycles. It might only take 1ms to run the program and generate the data and stop it, then it takes a whole 80ms before the human brain has started to realize the program has stopped. When the brain is starting to interpret the data, it has already passed several seconds. The analysis and figuring out how to change the code might take several minutes of thinking.

A scripting programming language called Dyon is underway to explore the possibilities of watching data from running programs without stopping them. This feature is called an "in-type".

With in-types, the programmer can reload a module which tells how the user wants to watch over the rest of the program. Each in-type subscribes to input data of a particular function. This subscription can happen on existing running code, without communicating any new information. The receiver gets all input data that happens across threads, making it possible to create an overview of what is going on.

Perhaps one day we will be able to explore what the limits are to watching over and responding quickly to running programs, such that this ability is carefully analyzed before we create a super-intelligence.

2. Exploring Vast Amounts of Possibilities

Humans have cognitive biases. One of the reason for these biases is that humans process limited amounts of data.

For example, most people do not realize how big the Earth is.

This is because most people do not have access to a tool which lets them perceive how big the Earth is. I assume some people have tried VR and gotten a somewhat feeling of our planet being huge. Still, it is very hard to really wrap your head around this idea and the consequences for e.g. climate change.

A super-intelligence might easily construct models that lets it directly perceive and learn from them. Some things that are unthinkable to us, not because we are too dumb but because we lack the data, could be relatively easy for super-intelligence to grasp.

For any given situation, a super-intelligence might imagine a vast amount of possibilities.

This could mean that a super-intelligence is generally more capable of predicting what might happen. Since humans have limited brains, we have to focus on one or two plausible or worthy-to-pay-attention scenarios. A super-intelligence on the other hand, might not loose track of important scenarios, but easily explore the boundary where all the stuff happens that humans never think of.

3. Rapid Self-Replication

The time it takes for a super-intelligence to make a copy of itself, is the time it takes to start a new program. As long the computer which it runs on has enough capacity, the super-intelligence can run as many copies of itself as it likes to.

Each of these copies can then solve the problem of acquiring more hardware to run on.

Today, you can easily purchase a computer online with a credit card or online bank account, perhaps even with cryptocurrency, and the computer will arrive at your doorstep.

You can even rent unlimited amounts of computing capacity if you have saved up enough money.

It seems to me that the moment a super-intelligence exists and it can control copies of itself, there will be millions or perhaps even billions of such instances running before anyone would notice.

There could be one super-intelligence running for every person on earth, yet drowning in the noise of the rest of the Internet. This computing capacity could hide under other names, such as large distributed computing services.

Any additional gain that the super-intelligence might obtain from making improvements to its source code, will be multiplied by the amount of instances it runs and reduce the cost. The more money the super-intelligence spends to run more instances, the more it will benefit from self-improvements.

Therefore, self-improvement is not necessarily what drives the major impacts of super-intelligence. It could be a vast number of copies that improve massively in efficiency from small marginal gains in self-improvements.

Now, think how a large gain in self-improvement would look like: Extremely beneficial to all the running instances of super-intelligence. From an economic perspective, rapid self-replication seems to lead to rapid self-improvement.