It's so cute when theists basically pat their audience on the head and say, "There, there, thinking machines aren't a threat. We won't have to treat them any better than we do a dog."
David Gelernter throws out words like "consciousness" and "awareness" and even the hoary "free will" without ever acknowledging that none of these terms have any serious definition. The assumption is always the same: thinking is irreducible to brain activity, which is as incorrect as saying a video game were somehow irreducible to electrons passing through silicon.
And Gelernter refuses to acknowledge what Wittgenstein pointed out over a century ago: when things start acting as if they acted indistinguishable from those we accorded the label "ensouled," then we are morally obligated to treat them as if they, too, had souls. Anything else is repugnant.
Gelernter's fantasy of "human-like machines" doesn't really deal with the future at all. We don't worry about human-like machines: those are actually quite boring from a researcher's point of view. (As a writer, of course, I have other reasons for delving into the topic.) Our big worries are in targetted intelligences for acheiving specific intellectual or scientific aims, the results of which involve so much intellectual capacity that one ordinary human mind cannot encompass the result: we're left to using the recipes left by our vast, cool, and unsympathetic engines of thought, and explaining them to each other by a process indistinguishable from hermeneutics. At that point, we had better have tight reins on our intellectual progeny, because these vastly smarter but still unconscious, technological but unself-aware machines, will have incidental agendas built into them about which we may be completely mistaken.
And then there will be real trouble.
David Gelernter throws out words like "consciousness" and "awareness" and even the hoary "free will" without ever acknowledging that none of these terms have any serious definition. The assumption is always the same: thinking is irreducible to brain activity, which is as incorrect as saying a video game were somehow irreducible to electrons passing through silicon.
And Gelernter refuses to acknowledge what Wittgenstein pointed out over a century ago: when things start acting as if they acted indistinguishable from those we accorded the label "ensouled," then we are morally obligated to treat them as if they, too, had souls. Anything else is repugnant.
Gelernter's fantasy of "human-like machines" doesn't really deal with the future at all. We don't worry about human-like machines: those are actually quite boring from a researcher's point of view. (As a writer, of course, I have other reasons for delving into the topic.) Our big worries are in targetted intelligences for acheiving specific intellectual or scientific aims, the results of which involve so much intellectual capacity that one ordinary human mind cannot encompass the result: we're left to using the recipes left by our vast, cool, and unsympathetic engines of thought, and explaining them to each other by a process indistinguishable from hermeneutics. At that point, we had better have tight reins on our intellectual progeny, because these vastly smarter but still unconscious, technological but unself-aware machines, will have incidental agendas built into them about which we may be completely mistaken.
And then there will be real trouble.
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}