Not too long ago, I was listening to a news story about AI and they were talking about AI becoming conscious. The conversation led me to muse about when computers becoming conscious became a topic of serious conversation.
For those of us who grew up in the 70s when computers were mainly large mainframes from companies like IBM, DEC, and Burroughs, a talking computer was science fiction, like the computers in the original Star Trek. A decade later, in the 80s, computers were smaller, faster, and we were beginning to imagine the potential.
Star Trek: the Next Generation presented the beginning of AI. Being a Trekkie, I didn’t miss an episode. I was also working in IT but didn’t realize what I was witnessing. The conversation on the news that day brought up memories from different episodes. For example, in the episode, Interface, Geordi is using an experimental neural interface and probe to investigate the wreckage of the USS Raman. At one point he tells the computer to “theorize”. I remember that moment because my reaction was — “yeah, like that will ever happen! Even if I could, why would I let a computer think for me?”
Fast forward to today, 30+ years later, we are walking around with computers on us — phones that are rarely used to make phone calls, but more to interact with computers. Every app we touch has an AI interface. We call a help desk and it’s nearly impossible to talk to a person. Yet STNG explored the question of AI, the difference between intelligence, consciousness, and personhood decades before today. One of the main characters, Data, is front and center to the conversation. In the episode, The Measure of a Man, the question isn’t simply whether Data can think. It’s whether Starfleet has the right to treat him as property.
The news story on October 26 was about Anthropic filling paperwork in which they acknowledge that AI systems might eventually deserve moral consideration. So Anthropic did not claim that Claude is conscious. However, they acknowledge uncertainty and concern for ethical obligations.
The AI boom has been growing rapidly. People who once opposed AI are jumping on board and realizing how much easier it is to use and be productive. Yet, that does not mean we should not throw caution to the winds. Even without full consciousness, there is a potential danger to allowing an automated system full control. Go to your favorite AI and you’ll see connectors to many apps, including your financial data. Yes, wouldn’t it be nice to get all your data in one app and be able to have that app pay your bills and move your investments to more lucrative performers? What about allowing all your medical records to be in one spot so you can get a full view of the various medications prescribed by different doctors and ensure they do not have adverse interactions?
Sounds great. Right? An AI doesn’t have to be conscious to be dangerous. A powerful automated system pursuing badly specified objectives could cause considerable harm without experiencing a single thought or feeling. Consider the episode, Elementary, Dear Data. Dr. Pulaski tells Gordi and Data the Data wouldn’t stand a chance of solving an original mystery. In response, they decide to see if Data could solve a “Holmesian” type mystery — one he did not know the outcome. Gordi instructs the computer to create a unique Sherlock Holmes mystery with an adversary who is capable of defeating Data. And the game is afoot!
If you haven’t watched the episode, spoiler alert — the computer does as instructed. It gave Professor Moriarty the intelligence and cunning needed to defeat Data in a mystery. The abilities included overriding safety protocols and an awareness of the world outside of the holodeck.
Moral of the story, think twice before allowing AI access to your personal data. I’m not say don’t use AI. I’ve found AI to be a great time saver and a wonderful research assistant. But I still won’t let it theorize for me.