The insanity of utopian thinking about Artificial Intelligence

Interesting:

Since I’m being critical of AI alarmism, it’s only fair that I put my own cards on the table.

I think our understanding of the mind is in the same position that alchemy was in in the seventeenth century.

Alchemists get a bad rap. We think of them as mystics who did not do a lot of experimental work. Modern research has revealed that they were far more diligent bench chemists than we gave them credit for.

In many cases they used modern experimental techniques, kept lab notebooks, and asked good questions.

The alchemists got a lot right! For example, they were convinced of the corpuscular theory of matter: that everything is made of little tiny bits, and that you can re-combine the bits with one another to create different substances, which is correct!

Their problem was they didn’t have precise enough equipment to make the discoveries they needed to.

The big discovery you need to make as an alchemist is mass balance: that everything you start with weights as much as your final products. But some of those might be gases or evanescent liquids, and alchemists just didn’t have the precision.

Modern chemistry was not possible until the 18th century.

The alchemists also had clues that led them astray. For one thing, the were obsessed with mercury. Mercury is not very interesting chemically, but it is the only metal that is a liquid at room temperature.

This seemed very significant to the alchemists, and caused them to place mercury at the heart of their alchemical system, and their search for the Philosopher’s Stone, a way to turn base metals into gold.

It didn’t help that mercury was neurotoxic, so if you spent too much time playing with it, you started to think weird thoughts. In that way, it resembles our current thought experiments with superintelligence.

Imagine if we could send a modern chemistry textbook back in time to a great alchemist like George Starkey or Isaac Newton.

The first thing they would do would be flip through to see if we found the Philosopher’s Stone. And they’d discover that we had! We realized their dream.!

Except we aren’t all that excited about it, because when we turn base metals into gold, it comes out radioactive. Stand next to an ingot of transubstantiated gold and it will kill you with invisible, magic rays.

You can imagine what a struggle it would be to not make the modern concepts of radioactivity and atomic energy sound mystical to them.

We would have to go on to explain what we do use the “philosopher’s stone” for: to make a metal that never existed on earth, two handfuls of which are sufficient to blow up a city if brought together with sufficient speed.

What’s more, we would have to explain to the alchemists that every star they see in the sky is a “philosopher’s stone”, converting elements from one to another, and that every particle in our bodies comes from stars in the firmament that existed and exploded before the creation of the Earth.

Finally, they would learn that the forces that hold our bodies together are the forces that make lightning in the sky, and that the reason you or I can see anything is the same reason that a lodestone attracts metal, and the same reason that I can stand on this stage without falling through it.

They would learn that everything we see, touch and smell is governed by this single force, which obeys mathematical laws so simple we can write them on an index card.

Why it is so simple is a deep mystery even to us. But to them it would sound like pure mysticism.

I think we are in the same boat with the theory of mind.

We have some important clues. The most important of these is the experience of consciousness. This box of meat on my neck is self-aware, and hopefully (unless we’re in a simulation) you guys also experience the same thing I do.

But while this is the most basic and obvious fact in the world, we understand it so poorly we can’t even frame scientific questions about it.

We also have other clues that may be important, or may be false leads. We know that all intelligent creatures sleep, and dream. We know how brains develop in children, we know that emotions and language seem to have a profound effect on cognition.

We know that minds have to play and learn to interact with the world, before they reach their full mental capacity.

And we have clues from computer science as well. We’ve discovered computer techniques that detect images and sounds in ways that seem to mimic the visual and auditory preprocessing done in the brain.