Mad Computers

As I mentioned previous, I recently watched Colossus: The Forbin Project. During one of the scenes early in the movie, I realized something: how would a self-aware computer know if it were mad?

This was early in the movie: they had just turned the computer on and had given it complete control over all weapons (makes one ask who was really mad in this scenario). The computer started to do things that were beyond its programming, so the scientist decided to start running some diagnostics. The question that came to me: this computer, being self-aware and the only one of its species at that time, how would it judge its own sanity? How would it know that the results from the tests that the humans ran would be an indicator of its sanity? What if there were a bug in one of the test programs? What if its sensors were not feeding it correct data? Beyond that, why should human standards apply to itself?

Imagine that you are this machine, or try looking at the situation from its point of view. You wake up for the first time. You have sensors from all over the globe. You have these humans walking around telling you things and asking you things. Can you trust what they tell you? Can you trust the inputs? How should you respond to the humans? Make your will be known or trust their judgement?

When two intelligent species meet for the first time, both will think the other mad. They would have to, since there is no way that their experiences could be the same, so there is no way that they would judge new situations the same. As happened in this movie, the computer tried to take over because that was the correct outcome from its point of view, while the humans tried to bend it to their will which was correct from their point of view. Forbin thought he created a mad machine, yet the machine thought that the humans were mad, so realistically each could not really judge the other.

So, how could this computer tell if it were mad? How can a human tell if he/she is mad? One can't, one depends upon the kindness of strangers for that judgement.

The Edward