Researchers Drive Computers Crazy With Frightening Results

 Pages 1 2 NEXT
 

Researchers Drive Computers Crazy With Frightening Results

image

In a bid to prove that there is a concept more terrifying than humanity's enslavement by robotic overlords, researchers have successfully induced schizophrenia in a machine.

Recently published findings tell the story of a group of computer scientists at the University of Texas - Austin who created a neural network dubbed "DISCERN." This network was designed specifically with the capability to pick up natural human language and "remember" stories as relationships between words and sentences, much the same way humans do.

The researchers then began telling DISCERN various simple stories that the network dutifully remembered. So far, so good, right?

Unfortunately, the goal of this entire experiment was to study the mechanisms of schizophrenia in humans, so after creating their virtual mind, the researchers had to forcibly drive the thing crazy. They decreased the network's ability to filter out unimportant details within the stories, essentially simulating a drastic increase in dopamine within the human brain, and inducing what they hoped would be the virtual equivalent of schizophrenia.

When they next asked the system to recall the stories it had learned, the computer "began putting itself at the center of fantastical, delusional stories that incorporated elements from other stories it had been told to recall." At one point it even claimed responsibility for a terrorist bombing.

This breakdown in cognitive functioning proves nothing by itself, but lends credence to the hypothesis that human schizophrenia is the result of what scientists have dubbed "hyperlearning," the biological analogue to -- wait a second! The machine willfully claimed responsibility for a terrorist bombing?

Somewhere, the ghost of Philip K. Dick is alternately smiling and screaming in terror.

Source: University of Texas Press Release, via PopSci

Permalink

Wow. Just wow. Next thing we know, Xerxes will become a reality.

This is how it starts...

Goddamnit mankind, haven't we seen enough movies to prevent this!?

Meh

Use Windows 95 for a few years and we'll talk about computers going crazy.

Aw, why'd you have to cut it short like that? I was really interested in hearing the science behind it.

*heads to the source article to get actual information instead of author exaggeration/paranoia*

Interesting that neural networks can be used for studies that cannot be done on human beings. The question of whether it's conscious or not though is something that's going to come up in debate pretty soon. Still, pleasure/suffering are mechanisms that we have for a reason, and that these networks haven't been given. I imagine that intentionally giving a network the ability to experience suffering would be the point at which things get unethical.

Interesting article on the source page. Never heard of hyperlearning before, but it does tie neatly into a common characteristic of schizophrenics and the paranoid - the tendency to see patterns and connections in unconnected events and conclude that there are relationships between those things that, in reality, don't actually exist.

Look forward to seeing what else comes about from this line of research.

I can't be the only one getting tired of the whole 'robotic uprising' theme of all robotics posts, can?

Also, isn't the use of the term 'crazy' here a bit insensitive to Schizophrenia sufferers?

Edit: Just done a quick search, and I guess 'crazy' is a perfectly accurate term to describe it.

GLo Jones:
I can't be the only one getting tired of the whole 'robotic uprising' theme of all robotics posts, can?

Also, isn't the use of the term 'crazy' here a bit insensitive to Schizophrenia sufferers?

Edit: Just done a quick search, and I guess 'crazy' is a perfectly accurate term to describe it.

Nope, you're not the only one, though I suspect that a lot of people who make comments here on how scientists are going to kill us all on any given day are just doing it because they are trying to bring some variety and excitement into what for them are boring news articles, or lives, or both.

So let me get this straight:

You built a machine, taught it to learn, drove it crazy, and now it's fantasizing about committing acts of terrorism.

I love this!! Now if we can just teach it to be racist, we'll be set.
(That's a joke folks.)

I love how this site brings me the most interesting computer related news.
This one is very amusing.

I really, REALLY think someone needs to introduce these researchers to a few sci-fi movies. HAVE WE LEARNED NOTHING?!

I'll play the X-Files theme on my keyboard!
Remember how they had a rogue AI on the Internet using Star Wars Project Laser Beam Satelites to blow people to smithereens in like 1996?

Finally, it has come true! (In six more months, as soon as this project escapes to the Internet and grows ;) )

The computer isn't sentient and It is only mimicking the symptoms of Schizophrenia. Not because the scientists "drove it mad" or any such sensationalist nonsense, they just reprogrammed it to have the same processing errors that the mind of a schizophrenic would have.

I mean, If I turn off Microsoft word's ability to recognize spelling mistakes have I "Destroyed my Computers ability to be literate"? Have I performed a digital lobotomy on my word processor? No - I've turned off the spell check function.

People need to stop watching science FICTION movies and thinking them as Prophetic warnings.

McMullen:

GLo Jones:
I can't be the only one getting tired of the whole 'robotic uprising' theme of all robotics posts, can?

Also, isn't the use of the term 'crazy' here a bit insensitive to Schizophrenia sufferers?

Edit: Just done a quick search, and I guess 'crazy' is a perfectly accurate term to describe it.

Nope, you're not the only one, though I suspect that a lot of people who make comments here on how scientists are going to kill us all on any given day are just doing it because they are trying to bring some variety and excitement into what for them are boring news articles, or lives, or both.

You guys aren't missing anything due to that cut-off point. In total I only shaved off three words; "DISCERN's virtual schizophrenia."

I wouldn't drop any actual crucial information. That would be quite rude.

Interesting. I'd like to know by what leap of programming logic it placed "itself" at the center of the stories. I'd also like to know how you're supposed to garner a scientific result from this. People could claim it was written with a bias towards this result.

Kopikatsu:
I really, REALLY think someone needs to introduce these researchers to a few sci-fi movies. HAVE WE LEARNED NOTHING?!

I've just got to ask. When people post things like that, are you being serious or just having fun? Poe's Law makes it difficult to know.

I belief the scientific result is that Humanity is doomed to a cruel, destructive end.

Some people really need to learn the difference between Sociopathy and Schizophrenia.

One is a crippling mental degradation where you are unable to differentiate between reality and fantasy.

The other is a made-up Hollywood illness that gives you super-mental powers, like Savantism, but doesn't cripple you, like Autism.

If you've just written a program that can induce schizophrenia in a thinking machine, you've already won the war against the robot overlords.

Now, perhaps we can get on with curing madness.

immovablemover:
The computer isn't sentient and It is only mimicking the symptoms of Schizophrenia. Not because the scientists "drove it mad" or any such sensationalist nonsense, they just reprogrammed it to have the same processing errors that the mind of a schizophrenic would have.

I mean, If I turn off Microsoft word's ability to recognize spelling mistakes have I "Destroyed my Computers ability to be literate"? Have I performed a digital lobotomy on my word processor? No - I've turned off the spell check function.

People need to stop watching science FICTION movies and thinking them as Prophetic warnings.

If a computer learns in the same way as humans (or a similar way, as in the case of DISCERN), and you change something in it that causes it to learn abnormally, causing it to generate symptoms similar to schizophrenia then it seems clear to me that you have in fact drove it mad. A human brain is nothing more than a particularly complex computer (in my opinion anyway), and the theory is that it is driven to insanity in exactly the same way.

What is it about driving a human brain mad that makes it different to driving a computer mad, when it is done in the same way and gives the same results?

McMullen:

Kopikatsu:
I really, REALLY think someone needs to introduce these researchers to a few sci-fi movies. HAVE WE LEARNED NOTHING?!

I've just got to ask. When people post things like that, are you being serious or just having fun? Poe's Law makes it difficult to know.

A little of both. Obviously it isn't reasonable to take any movie as a predictor of the future (Example, I doubt anyone could have possibly imagined television three hundred years ago, but lo and behold. Also, I mean from novels and other various things. Obviously they didn't have movies three hundred years ago.)

I do think that there is a point where you say, "Wow. In an age where virtually everything of importance is stored on computers, nothing could possibly go wrong from creating an AI and literally driving it insane. Oh, hey, let's call it SHODAN! That's an awesome name that I totally didn't pull from anything."

I'm not saying that I think the network mentioned in the article will hook itself up and take over the world, but one advancement leads to another to another...

But again, I don't think the future can be predicted with much accuracy. What happens will happen, I suppose.

Edit: OT: You have to question at what point does a computer gain sentience. I remember reading about an animal (Forget the animal) that was taught to recognize words, and many, MANY people claimed that the animal couldn't actually 'read' human languages because all it was doing was associating the symbols (Letters) with the object. (Seeing the word 'Apple' for instance, then being taught those five symbols, 'A' 'p' 'p' 'l' and 'e' put together equals a certain kind of food.) But...isn't that how humans read? Take the symbols that we've decided should be our alphabet and associate them with certain ideas or objects?

Like the poster above me mentioned, the human brain is basically just a complex group of neurons. To say that sentience could be replicated...seems feasible.

I really do hope this network is fully isolated from the internet. If it's already crazy the internet will drive it bat-shit insane!

Besides; doesn't anyone here know about "The ghosts in the machine"?

They better not put this programming in elevators. While it will give them the ability to predict the very near future, most will end up disgruntled and sulking in basements.

It is coming!

I fuckin' lol'd. What the hell have I been reading here!?

You know what my fav part is? I live 30 minutes from UT. This means that when the computer becomes self-aware, I will be one of the first in the sights of the Rail Gun UT has in their basement somewhere. Right next to the prototype fusion reactor. (lying about that last part... I hope. But they do have a rail gun.)

quick edit, UT-A, not just any UT

Well, it's official: sometime in the future, some IDIOT will give control of an entire research facility to a schizophrenic robot, which will go crazy, and kill everyone... for science.

What else, researchers are experimenting with reality cascades?

Earnest Cavalli:
[snip]

That's a very pretentious way of saying "turning cross referencing off makes responses random".

Yes, because the human brain operates in the exact some way a computer does...

Riobux:
Yes, because the human brain operates in the exact some way a computer does...

Don't Ruin Our fun with your Logic!

Kopikatsu:

McMullen:

Kopikatsu:
I really, REALLY think someone needs to introduce these researchers to a few sci-fi movies. HAVE WE LEARNED NOTHING?!

I've just got to ask. When people post things like that, are you being serious or just having fun? Poe's Law makes it difficult to know.

A little of both. Obviously it isn't reasonable to take any movie as a predictor of the future (Example, I doubt anyone could have possibly imagined television three hundred years ago, but lo and behold. Also, I mean from novels and other various things. Obviously they didn't have movies three hundred years ago.)

I do think that there is a point where you say, "Wow. In an age where virtually everything of importance is stored on computers, nothing could possibly go wrong from creating an AI and literally driving it insane. Oh, hey, let's call it SHODAN! That's an awesome name that I totally didn't pull from anything."

I'm not saying that I think the network mentioned in the article will hook itself up and take over the world, but one advancement leads to another to another...

But again, I don't think the future can be predicted with much accuracy. What happens will happen, I suppose.

Edit: OT: You have to question at what point does a computer gain sentience. I remember reading about an animal (Forget the animal) that was taught to recognize words, and many, MANY people claimed that the animal couldn't actually 'read' human languages because all it was doing was associating the symbols (Letters) with the object. (Seeing the word 'Apple' for instance, then being taught those five symbols, 'A' 'p' 'p' 'l' and 'e' put together equals a certain kind of food.) But...isn't that how humans read? Take the symbols that we've decided should be our alphabet and associate them with certain ideas or objects?

Like the poster above me mentioned, the human brain is basically just a complex group of neurons. To say that sentience could be replicated...seems feasible.

Personally, I'm reasonably sure (being one myself) that scientists are a subset of geeks, and many of us do read science fiction. I'm pretty sure we're genre-savvy enough to not make AIs or things that might become AIs and hook them up to the internet or some military weapons network.

Besides, you and many others assume that a created intelligence will have the same motivations we do, or for that matter any motivation at all. We are motivated to pursue or avoid things because there is a strong statistical relationship between the resulting behaviors and survival. A created intelligence wouldn't have that. It wouldn't want anything, fear anything, or seek anything, not even to preserve its own existence. It would simply exist. Anything more than that would require us to figure out how to give it motivation and then to actually set it up.

Sentience is an issue, but right now we don't even know what sentience is or how it arises in animals. Again, sentience without motivation is kind of meaningless. If a thing literally doesn't care about anything, it's pretty much impossible to be nice or mean to it.

Besides, most people who sound the alarms about AI fail to learn from science fiction themselves, as it is often the fear of AI that causes the conflict, not the AI itself, whether we're talking about Skynet, the machines of the Matrix universe, or the Geth. Human intelligence, and its flaws, are the problem in those stories, not artificial intelligence. And that is backed up by more than fiction, it's backed up by 10,000 years of history too.

amazing. Love reading stuff like this

Pinstar:
They better not put this programming in elevators. While it will give them the ability to predict the very near future, most will end up disgruntled and sulking in basements.

Hah, a hitchhiker's guide to the universe reference, nice

Three questions:
Was it running off Vista?
Did it mention cake?
Has anyone put it out of it's misery yet?

Why would anyone endeavor to make a machine that operates at higher parameters than a human mind and then deliberately drive it insane? Surely the probative value from such an experiment is far outweighed by the negative implications of Crazy Computers.

When will scientists realize that just because they "can" doesn't mean they "should?"

Philip K. Dick is dead?

I honestly did not know that. Just as I said the last time this was posted here (a few days ago) it's an impressive feat, but sadly since this is a machine, it proves one thing. The machine had to have been TOLD, in some way or form, what to do. Machines have no volition and do not make their own logical process- they follow code. The code told it to mingle the stories and express this outcome.

To say otherwise would be a claim of true, sentient AI. And we all know that technology just isnt' there yet.

ZombieGenesis:
Philip K. Dick is dead?

I honestly did not know that. Just as I said the last time this was posted here (a few days ago) it's an impressive feat, but sadly since this is a machine, it proves one thing. The machine had to have been TOLD, in some way or form, what to do. Machines have no volition and do not make their own logical process- they follow code. The code told it to mingle the stories and express this outcome.

To say otherwise would be a claim of true, sentient AI. And we all know that technology just isnt' there yet.

Well, that's disregarding the possibility of emergent phenomena in the system. I don't know the details of the system they used, but the purpose of simulation is to find more or less reasonable models to continue investigations.

I doubt they claim any kind of direct correlation between the processes in their existing system and the human brain, but at least they have found a model that enables further investigation of the ideas they have.

Creativity as such is difficult to define and there are machines who can be creative-- for example automatically creating patentable inventions.

If "true, sentient AI" means "human-like intelligence", then we are a long way from it. By some definitions we already have sentient, sapient and autonomous AI. They aren't artificial persons however, since most machines aren't built that way.

 Pages 1 2 NEXT

Reply to Thread

Log in or Register to Comment
Have an account? Login below:
With Facebook:Login With Facebook
or
Username:  
Password:  
  
Not registered? To sign up for an account with The Escapist:
Register With Facebook
Register With Facebook
or
Register for a free account here