The Creator Who Turned Against His Creation
Joseph Weizenbaum published Computer Power and Human Reason, warning that his own ELIZA experiment revealed a dangerous human willingness to trust machines with intimate decisions.
Ulrich Hansen, Germany (Journalist). / CC BY-SA 3.0
What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.
— Joseph Weizenbaum
In 1976, German-American computer scientist Joseph Weizenbaum published “Computer Power and Human Reason,” a seminal work that critiqued the burgeoning field of artificial intelligence.
What happened: In 1976, Joseph Weizenbaum released his book “Computer Power and Human Reason: From Judgment to Calculation,” which argued against the idea that computers should make important decisions due to their inherent lack of human qualities such as compassion and wisdom. The book was inspired by an incident where Weizenbaum’s secretary asked him to leave the room so she could talk privately with ELIZA, a program he had created that used pattern-matching to simulate conversation. This event shocked Weizenbaum and led him to question the ethical implications of AI. (Source: Computer Power and Human Reason)
Why it matters: Weizenbaum’s book was one of the first major critiques of AI from within the field, raising concerns about the ‘ELIZA effect’ and the potential dangers of delegating decision-making to machines. His work has had a lasting impact on the philosophy of AI and continues to influence discussions about the ethical use of technology today.
Further reading:
Why This Mattered
Weizenbaum was shocked when his secretary asked him to leave the room so she could talk privately with ELIZA — a program he knew was merely pattern-matching. His resulting book became one of the first major critiques of AI from inside the field, coining concerns about the 'ELIZA effect' and arguing that some decisions should never be delegated to computers regardless of their capability.
