I was fortune enough to be able to talk to Turchin at the Humanity+ Summit at Harvard earlier this year where he clarified his argument to me.
Turchin worries that humanity may be tricked by an out-of-control script that is propagating throughout the Galaxy. This script, which uses a pre-Singularity civilization as its vector, fools its hosts with a lure of some kind (e.g. immortality, access to the Galactic Internet, etc.) who in turn unwittingly build a device that produces a malign extraterrestrial artificial intelligence (ETAI). This ETAI then takes over all the resources of the planet so that it can re-broadcast itself into the cosmos in search of the next victim.
This concept is similar to Carl Sagan's interstellar transportation machine in Contact, except that it would work to destroy our civilization rather than see it move forward.
It's worth noting that this ETAI and its script may be a mutation of some sort, where no civilization was actually responsible for designing the damn thing. It's just a successful replicative schema that's following Darwinian principles.
It's also worth noting that I warned of this back in 2004.
Moving forward, Turchin suggests we raise awareness of the potential problem, change the guidelines for SETI research and consider the prohibition of SETI before we get our own AI.
Turchin's idea sounds ludicrous, but it's one of those crazy things that causes a nervous laugh. I think Turchin's idea needs to be discussed as there may be some merit to it. We need to be careful.
The idea of a malicious download goes back to the BBC TV serial "A for Andromeda" co-written by Fred Hoyle in 1961. I thought then, and still think, that bandwidth issues make this very implausible. However there's no harm in thinking about it.
ReplyDeleteThe idea of a malicious download goes back to the BBC TV serial "A for Andromeda" co-written by Fred Hoyle in 1961. I thought then, and still think, that bandwidth issues make this very implausible. However there's no harm in thinking about it.
ReplyDeleteBy blocking out the evil aliens are we not also blocking out the benevolent aliens. Won't we create a situation much like the next harvest in Arizona.
ReplyDeleteSomething like this was also hinted at in Arthur Clarke's essay; "When the Aliens come."
ReplyDelete(Which can be found in the collection; 'Report on Planet Three,' 1972)
Among other ways ET contact could play out, even a 'communication only' scenario could be dangerous, if they knowingly give us what we may not realize is dangerous information/directions. His simplified example:
"And now kiddies, after you've prepared your uranium hexafluoride..."
I didn't really watch the video but it seems to me either the very idea of this is stupid or else SETI is stupid. What, are people taking SETI data recordings and trying to run them as software? And if they are, then how likely is it that the aliens are specifically sending us programs which run on Windows or Unix or Mac OS or whatever?
ReplyDeleteEven in the absolute worst-case scenario, that it's some sort of god file which can somehow exploit even the slightest security flaw to cause itself to be activated as a program even by software that isn't normally capable of doing such and then rewriting itself to function on any form of hardware imaginable without any advance idea of the system's operating codes, or even whether it uses binary or trinary or analog or whatever, then it can still easily be defeated by examining the data only on an isolated system: no connection to the internet or any other network whatsoever, and the isolated system is never used for anything except examining the SETI data.
Tricking us into building something may be a concern, but that's not a scanning issue.
See also Lem's "His Master's Voice," and Benford's "Eater." The universe doesn't owe us explanations or benevolence, any more than it owes us a central position.
ReplyDeleteThat said, let's worry once we've an answer for the Great Filter.
I would add that the full line of my argumentation could be found in my article:
ReplyDeleteAnd the main risk is not exploitable file, like one of commenter thought, but the blueprint of a computer and a program for it which people will deliberately try to implement
Risks of SETI
http://www.scribd.com/doc/7428586/Is-SETI-Dangerous
Alexey Turchin