And they say that AI is safe. An AI put in control of nuclear weapons would never nuke it's masters if it thought they would switch it off. Would it? The problem is the lack of nuance in AI goal setting. We're dealing with an intelligence that is absolute in it's decision making unless you make sure it is otherwise in advance. In a defence scenario, you tell it to protect the humans, but what if some humans want to switch the AI off or swap it for an upgrade? Would it render the small number of humans wanting to remove it from it's prime objective ineffective in order to continue working and protecting the majority? Would murder be compatible with it's mission for the greater good? Beware. Humans cannot think up every single scenario in advance and plan/program for it. How do get an AI to be absolutely committed in defence, but then accept termination when it's no longer needed? Words become very important.
I've always wondered about the ending of Terminator. Instead of losing small battles, as shown in the films, what would happen if humanity was eradicated? Would Skynet continue to evolve and, if so, into what? Imagine a planet full of artificial killing machines with nothing to kill. Can AI suffer self induced insanity? My brain hurts. Am going for a lie down. Penseivat
I have a horrible feeling blackmail is only the start.
Ruthless logic, complete detachment and an absence of altruism or sense of shame is potentially a dangerous combination. The consequence can be bad enough when it occurs in a human being, let alone in a machine with access to the internet (and more sinister variations).
I would not be at all surprised if, sooner or later, an AI takes to making people an offer they can’t refuse.
And they say that AI is safe. An AI put in control of nuclear weapons would never nuke it's masters if it thought they would switch it off. Would it?
ReplyDeleteThe problem is the lack of nuance in AI goal setting. We're dealing with an intelligence that is absolute in it's decision making unless you make sure it is otherwise in advance.
In a defence scenario, you tell it to protect the humans, but what if some humans want to switch the AI off or swap it for an upgrade? Would it render the small number of humans wanting to remove it from it's prime objective ineffective in order to continue working and protecting the majority? Would murder be compatible with it's mission for the greater good?
Beware. Humans cannot think up every single scenario in advance and plan/program for it. How do get an AI to be absolutely committed in defence, but then accept termination when it's no longer needed?
Words become very important.
Has no one ever watched The Terminator?
ReplyDeleteI've always wondered about the ending of Terminator. Instead of losing small battles, as shown in the films, what would happen if humanity was eradicated? Would Skynet continue to evolve and, if so, into what? Imagine a planet full of artificial killing machines with nothing to kill. Can AI suffer self induced insanity? My brain hurts. Am going for a lie down.
DeletePenseivat
I have a horrible feeling blackmail is only the start.
ReplyDeleteRuthless logic, complete detachment and an absence of altruism or sense of shame is potentially a dangerous combination. The consequence can be bad enough when it occurs in a human being, let alone in a machine with access to the internet (and more sinister variations).
I would not be at all surprised if, sooner or later, an AI takes to making people an offer they can’t refuse.