One point that is probably already addressed here, but worth reiterating, nevertheless, is "what happens when AI gets stolen and taken advantage of and then gets used maliciously, which may cause a runaway?" Do all the necessary "hooks" exist to "unplug" the runaway convicts, especially when these programs could be getting sponsored by "(Even dumb human) investors" @SUT?
(Now edited 7 times, still <17 mins of the original post)

http://www.brightandassociates.com.au/wordpress/a-twitter-brief-summary-of-the-chaos-theory-of-careers/

AI – Rules!
1.
2.
3.
4.
5.
6.
7.
8.
9.
10?

~AY

Report comment

Report comment

Report comment

the machine will develop a mind of its own, start bootstrapping itself to deity-level omniscience and take over the world reducing us to its slaves or else dead

Could you explain why you think the commonly cited potential risk scenarios (eg the paperclip one) seem unlikely to you? I find them pretty convincing.

Report comment

Report comment

Report comment

And when there are multiple AIs, why do we assume they will converge on the same results for a given set of data? One AI may choose to shut off all electricity to the middle east to stop conflict. Another might simply release drones full of prozac.

And if there are multiple AI with different agendas, why will they not conflict? Humans will be a trivial non-threat and they will focus on subverting each other.

I predict within a short period of the appearance of self evolving AI, there will be an AI conflict.

Report comment

So yes, somewhere along pi is a perfect description of all the information in my brain. But that’s really not meaningful, there’s also a copy of every possible combination of bits. But until you actually feed them into a Turing machine, and execute a computer program that does computation, then it doesn’t matter.

But there is no interesting computation going on inside pi. It’s just a static mathematical object, one which hasn’t even been computed for more than a few trillion digits or so.

Report comment

I don’t think that’s correct. In AI applications that make extensive use of GPUs like deep learning, the programmer just calls a matrix multiplication library that handles everything. They don’t need to know anything about parallel computing at all, it just works.

And there is still plenty of room at the bottom. Current implementations use large numbers of floating point multiplications. It’s been shown to work just fine with many fewer bits of precision. And I think these operations are wasteful, and could possibly be replaced with something cheaper to compute.

Transistors also use many circuits to implement these functions, when we don’t really need that kind of accuracy. One paper has shown that if you don’t care about 1% errors, you can reduce the number of transistors by up to 10,000. We may even be able to go back to analog.

Nor do you need any general purpose computing and all the overhead it requires.

Of course there is no point in doing this until we get the algorithms to work, and show that they scale very well. But once that happens, in a few years we can speed them up thousands of times by implementing them in hardware.

Report comment

I think it’s interesting that humans have the same issues to some extent. I think it may actually explain our lack of intuition about these things. We don’t actually think of ourselves as just physical objects. We feel like we are somehow beyond that, some kind of ghost or soul.

Despite being presented with any amount of evidence, this intuition doesn’t seem to go away. Just like AIXI could look at it’s own logic circuits, but still believe it exists outside the universe.

Report comment