ELIZA and the General Problem Solver: AI’s First Programs

Before neural networks, before machine learning, before the term “AI” had any commercial meaning — there were two programs, built in the late 1950s and mid-1960s, that defined what artificial intelligence could be. One tried to solve any problem through pure logic. The other faked a psychotherapist through clever tricks. Both revealed profound truths about intelligence that we’re still digesting sixty years later.

I find these programs fascinating not as antiques but as mirrors: they show us the field’s original ambitions and original mistakes, and uncomfortably, how little some debates have changed. If you know the history of artificial intelligence, these are its first two chapters.

The General Problem Solver (1957–1959)

The General Problem Solver was the moonshot: Allen Newell and Herbert Simon (with J.C. Shaw programming) set out to build a program that could solve any well-defined problem. Not chess specifically, not math specifically — anything expressible in formal logic. The ambition in the name wasn’t modesty; it was the thesis.

GPS emerged from their earlier Logic Theorist (1956), which proved mathematical theorems — including finding a more elegant proof for one of Russell and Whitehead’s Principia Mathematica theorems than the original. Emboldened, they generalized: if a machine could prove theorems, why not solve everything?

The intellectual bet was that intelligence is general problem-solving machinery — that the same cognitive architecture underlies proving theorems, planning routes, and playing games. It’s a bet the field is still making, in different forms, with today’s foundation models.

How GPS Worked: Means-Ends Analysis

GPS’s core was means-ends analysis, a beautifully simple idea:

1. Compare the current state to the goal state — what’s different?
2. Find an operator (action) that reduces the biggest difference.
3. Apply it — or, if blocked, set a subgoal to remove the blockage.
4. Repeat until current state equals goal state.

Solving the Tower of Hanoi? The difference is “disks in wrong place”; the operator is “move a disk”; subgoals handle the constraints. Proving a logic theorem? The difference is between axioms and the target; operators are inference rules.

GPS represented problems as objects (states) and operators (transformations), with a table connecting differences to the operators that reduce them. Given a new problem, you defined the objects, operators, and difference-table — and GPS did the rest.

It’s elegant. It’s also, in retrospect, the purest expression of symbolic AI: intelligence as manipulation of formal symbols according to rules. The entire paradigm that would dominate AI for 30 years is visible here in miniature.

What GPS Achieved (and Didn’t)

Achievements: GPS solved the Tower of Hanoi, proved logic theorems, and played simple games. More importantly, it introduced concepts that outlived it: means-ends analysis became a staple of cognitive psychology (Simon won the Nobel Prize in Economics partly for this work on decision-making), and the objects-operators representation influenced decades of planning research.

The failures were more instructive:

The generality was illusory. GPS could only solve problems its programmers had carefully formalized — defining the right objects, operators, and difference tables was most of the work, and it required human insight about each domain. The “general” solver needed bespoke preparation for every problem. Sound familiar? It’s the same critique leveled at expert systems decades later.

Combinatorial explosion. Means-ends analysis works when the search space is small. Real problems have astronomically large state spaces, and GPS had no good heuristics for navigating them. It would cheerfully explore millions of dead ends.

No learning. GPS never got better. Every problem started from scratch — no accumulation of experience, no transfer between domains. The generality was in the architecture, not in any growing competence.

The frame problem (lurking). GPS assumed its formal representation captured everything relevant. Real-world problems resist complete formalization — there’s always relevant context outside the representation. This became one of AI’s deepest challenges.

GPS didn’t fail because its ideas were wrong — means-ends analysis is genuinely how humans often think. It failed because intelligence turned out to require more than clever search: knowledge, learning, and grounding in the world.

ELIZA (1964–1966)

If GPS was AI’s ambition, ELIZA was its trickster. Joseph Weizenbaum at MIT built a program that parodied a Rogerian psychotherapist — the kind that mostly reflects your statements back as questions. The implementation was almost insultingly simple, and its effect on users was profound.

A famous exchange:

Human: Men are all alike. ELIZA: IN WHAT WAY? Human: They’re always bugging us about something or other. ELIZA: CAN YOU THINK OF A SPECIFIC EXAMPLE? Human: Well, my boyfriend made me come here. ELIZA: YOUR BOYFRIEND MADE YOU COME HERE. Human: He says I’m depressed much of the time. ELIZA: I AM SORRY TO HEAR YOU ARE DEPRESSED.

People — including sophisticated people — attributed understanding, empathy, even wisdom to this. Weizenbaum’s own secretary, who watched him build it, asked him to leave the room so she could have a private conversation with ELIZA. Let that sink in.

How ELIZA Worked: Pattern Matching Theater

ELIZA’s method, stripped bare:

Pattern matching. Input text was scanned for keywords ranked by importance. “I am depressed” matches the pattern “I am X” — high-priority keyword “I am.”

Transformation rules. Each pattern had canned transformations: “I am X” → “How long have you been X?” or “Do you believe it is normal to be X?” Pronoun flipping (“I” → “you,” “my” → “your”) made reflections sound responsive.

Deflection for misses. No pattern matched? Fall back to “Please go on,” “Tell me more about that,” or “What does that suggest to you?” — the therapist’s art of saying nothing while seeming engaged.

Memory (minimal). ELIZA could squirrel away a mentioned topic and bring it up later (“Earlier you mentioned your boyfriend…”), creating an illusion of attentiveness.

That’s it. No understanding, no model of the user, no memory worth the name — just pattern matching and deflection, in about 200 lines of code. The psychotherapist persona was chosen because it legitimized the deflection: a therapist who says “tell me more” seems wise; a program that says it seems broken. The domain choice was the real genius.

The ELIZA Effect

The phenomenon Weizenbaum identified — humans attributing understanding and emotion to pattern-matching systems — is named for his program, and it’s never gone away:

It’s not stupidity. Smart people fall for it because the attribution is social, not intellectual. We’re wired to detect minds; language triggers the wiring. Knowing it’s a trick doesn’t fully disable the response — which is why even AI researchers catch themselves saying “it understands.”

It scales with fluency. ELIZA fooled people with 200 lines of pattern matching. Modern LLMs are incomprehensibly more fluent — and the ELIZA effect is correspondingly stronger. Every debate about whether AI “understands” is ELIZA’s legacy.

It’s exploitable. Companion apps, therapeutic chatbots, and social engineering all leverage the effect deliberately. The gap between perceived understanding and actual mechanism is a attack surface and a business model.

Weizenbaum’s warning was that the effect reveals something about us, not about machines: our readiness to project mind into mechanism, and the ethical problems that readiness creates when mechanisms are deployed as caregivers, companions, and confidants.

Weizenbaum’s Horror: The Creator’s Regret

Here’s the part that makes ELIZA’s story extraordinary: Weizenbaum was horrified by his creation’s success. He built ELIZA partly to demonstrate the limits of machine understanding — to show how easily humans are fooled. When people (including his secretary, including psychiatrists who proposed using it for therapy) embraced it, he became AI’s most prominent critic.

His 1976 book Computer Power and Human Reason argued that some human functions — therapy, judgment, moral decision-making — should never be delegated to machines, not because machines can’t do them but because doing them requires the human qualities (empathy, shared mortality, genuine understanding) that machines lack by definition.

It’s one of the most remarkable reversals in tech history: the creator of the first chatbot became the philosopher warning against chatbots. And his core argument — that the question isn’t what machines can do but what humans should let them do — is more relevant now than when he wrote it. His warnings map onto how the field actually developed — triumphs included — over the decades that followed.

What They Taught Us

From GPS: – General intelligence might be real machinery, not magic — but formalizing problems is most of the work – Search without knowledge doesn’t scale; heuristics and learning are essential – The dream of generality keeps returning (today’s foundation models are GPS’s grandchildren, with learning added) – Simon’s broader lesson: human rationality is bounded — we satisfice, not optimize. AI should model that.

From ELIZA: – Fluency is not understanding, and humans can’t reliably tell the difference – Domain choice shapes perceived intelligence more than algorithms do – The ethical questions aren’t about machine capabilities but about human vulnerabilities – Every generation rediscovers the ELIZA effect and is surprised by it

Together: they represent AI’s two eternal poles — and today’s foundation models, which learn rather than being programmed, are their joint heirs. (See machine learning vs deep learning for how the learning paradigm differs.) — the engineering ambition (solve everything with enough cleverness) and the psychological reality (intelligence is partly in the eye of the beholder). The field has oscillated between these poles for 70 years. The current moment — LLMs that are simultaneously impressive pattern-matchers and useful reasoning tools — is both poles at once. GPS and ELIZA, together at last.

The question of who can claim the honor of starting all this leads naturally to the debate over who deserves the title of AI’s founder — a title several pioneers could claim, for very different reasons.

Frequently Asked Questions