Second AI Winter Explained: The Expert System Collapse (1987-2000)

The second AI winter has the strangest opening of any technology bust: it began with success. Expert systems — AI programs encoding human expertise as rules — actually worked, actually sold, and actually created a multi-hundred-million-dollar industry in the mid-1980s. Companies like Symbolics and Teknowledge were darlings of the tech press. Japan committed billions to the Fifth Generation Computer project. Then, within a few years, nearly all of it was gone: companies bankrupt, hardware obsolete, the word “AI” unspeakable in polite funding circles.

The second AI winter (roughly 1987 to the late 1990s, with effects lingering into the early 2000s) is the more instructive of the two winters for today’s reader, because it was a market failure, not just an academic one — and markets are where today’s AI lives. For the full context, see what AI winter is and the history of artificial intelligence.

Table of Contents

• The Expert System Boom: What Actually Worked
• The Billion-Dollar Bet
• The Knowledge Bottleneck
• The Lisp Machine Catastrophe
• Japans Fifth Generation: The Moonshot That Missed
• The Collapse: 1987 to 1993
• The Long Thaw: AI Under Aliases
• What Survived and Why It Mattered
• Lessons for the Current Boom
• Frequently Asked Questions

The Expert System Boom: What Actually Worked

To understand the collapse, first understand what was real. Expert systems encoded human expertise as if-then rules: if the patient shows these findings and these test results, then consider this conclusion. The pioneering systems — DENDRAL (chemistry), MYCIN (medical expert systems), XCON (configuring computer orders at DEC, reportedly saving the company tens of millions annually) — demonstrated genuine value in narrow domains.

The business logic was seductive: human experts are scarce and expensive; encode their knowledge once, deploy it infinitely. Consulting firms, manufacturers, and financial institutions bought in. By the mid-1980s, hundreds of companies were building or deploying expert systems, and the trade press projected a multi-billion-dollar market. AI had its first real industry — not research grants, but customers paying for products.

And the products weren’t fraudulent. XCON genuinely configured VAX orders. Medical expert systems genuinely matched expert performance on narrow tasks. The boom’s foundation was real achievement — which made the collapse more disorienting, because the technology worked and still failed commercially. That paradox is the key to the whole story.

The Billion-Dollar Bet

The hype escalated from “useful narrow tools” to “the future of computing” with remarkable speed. Japan’s Fifth Generation Computer Systems project (1982-1992) committed enormous government funding to leapfrog American computing with logic-programming-based AI machines. In the US, DARPA’s Strategic Computing Initiative poured over a billion dollars into AI, including expert systems and autonomous vehicles. Startups raised massive valuations; established companies launched AI divisions.

The projections assumed expert systems would generalize: from configuring computers to running factories, from identifying infections to managing enterprises. Each success in a narrow domain was extrapolated into a broad revolution. The press obliged with covers featuring robot brains; the researchers obliged with confident timelines; the funders obliged with checks.

What nobody priced in: the gap between “works in one domain with dedicated experts maintaining it” and “general-purpose technology.” That gap turned out to contain the entire difficulty — the knowledge engineering bottleneck, the maintenance burden, the brittleness. The billion-dollar bet was placed on a technology whose scaling properties were unknown, and the unknown turned out to be hostile.

The Knowledge Bottleneck

The fundamental technical cause: expert systems couldn’t acquire knowledge at viable cost. Building one required “knowledge engineers” to interview human experts, extract their decision-making as rules, encode those rules, test them, and maintain them. This process was slow (months to years per system), expensive (rare specialists doing painstaking work), and fragile (experts often couldn’t articulate their tacit knowledge — the intuition part).

Worse, the systems didn’t learn. Every new situation needed new hand-written rules. Rule bases grew into tangled, contradictory messes — the “brittleness” problem: expert systems were confident and wrong outside their narrow domains, with no sense of their own limits. Maintenance became a treadmill: the world changed, the rules didn’t, and updating them required the same expensive experts.

Contrast with what an AI model is today — systems that learn patterns from data automatically. The knowledge bottleneck was really a learning bottleneck: expert systems couldn’t learn, so knowledge had to be hand-installed at prohibitive cost. The field’s eventual answer wasn’t better knowledge engineering — it was abandoning hand-coded knowledge for machine learning. But that answer took a decade to become viable, and the winter filled the gap.

The Lisp Machine Catastrophe

The second winter’s most vivid business casualty was the Lisp machine industry. Expert systems were typically built in Lisp (the AI language of choice), and specialized Lisp machines — expensive workstations optimized for Lisp — were the premium platform. Companies like Symbolics, Lisp Machines Inc., and Texas Instruments’ Explorer division built a real hardware business serving AI labs and expert system developers.

Then the workstation wars killed them. General-purpose workstations from Sun Microsystems and others, running Unix, became powerful enough to run Lisp adequately — at a fraction of the cost. Why buy a $100,000 Lisp machine when a $20,000 Sun workstation runs your expert system almost as well? The specialized hardware market evaporated almost overnight. Symbolics — once valued enormously — collapsed into irrelevance.

This wasn’t really about AI failing; it was about a hardware business model being undercut. But the Lisp machine companies were AI’s most visible commercial face, and their collapse read as AI’s collapse. The winter’s narrative was written by its most spectacular bankruptcies, regardless of what the technology itself was doing. Perception became reality for funding purposes.

Japan’s Fifth Generation: The Moonshot That Missed

Japan’s Fifth Generation project deserves its own autopsy because it was the largest, most deliberate AI bet ever placed — and its failure shaped global perceptions. Launched in 1982 with massive government backing, it aimed to build Prolog-based parallel computers that would reason like humans, leapfrogging American dominance in conventional computing. The ambition was national-strategic: AI as industrial policy.

A decade later, the verdict was clear: the machines were built, but the AI revolution didn’t arrive. Logic programming proved less generally powerful than hoped; parallel Prolog machines couldn’t compete with the relentless advance of conventional microprocessors (Moore’s Law favored the generalists); and the applications never materialized at the promised scale. The project wound down in 1992, widely judged a failure.

The damage extended beyond Japan. The Fifth Generation’s high profile meant its failure was read globally as AI’s failure — proof that even billions in coordinated investment couldn’t make the promises real. It validated the skeptics and gave funders everywhere a cautionary tale. Moonshots that miss don’t just waste money; they poison the well for the next attempt.

The Collapse: 1987 to 1993

The unraveling was swift once it started. The expert system companies’ growth stalled as customers discovered the maintenance burden and limited scope. Valuations collapsed. The Lisp machine market died. DARPA’s Strategic Computing Initiative faced critical reviews and redirected funding. By the early 1990s, the AI industry — hundreds of companies at its peak — had largely vanished: bankrupt, acquired for scraps, or pivoted away from AI.

Academia felt it as funding dried up and students fled. The term “AI” became professionally hazardous — researchers rebranded as machine learning, pattern recognition, or informatics specialists. Conferences shrank. The ambitious were advised to work on anything else. The field’s public presence went from magazine covers to punchlines.

What made this winter psychologically worse than the first: people had seen it before. Veterans of the first winter watched the same hype cycle inflate and burst again, with the same promises, the same extrapolations, the same crash. The sense wasn’t just disappointment — it was déjà vu, the demoralizing feeling that the field was structurally incapable of learning from its own history.

The Long Thaw: AI Under Aliases

The 1990s weren’t empty — they were incognito. Machine learning research continued and advanced: support vector machines, Bayesian methods, reinforcement learning foundations, early neural network work (backpropagation improvements, convolutional networks for handwriting recognition). Data mining found commercial applications. The work that would become modern AI was being done — just not under the AI name.

Key developments of the “winter” years: the internet began generating massive datasets (the fuel future AI needed); GPUs emerged as parallel compute engines (the engine); statistical learning theory matured (the mathematics); and a few stubborn connectionists kept neural networks alive through the unfashionable years. The thaw wasn’t a single event but an accumulation — by the late 2000s, the ingredients for the deep learning revolution were quietly in place.

The alias period had an ironic benefit: freed from AI’s hype baggage, researchers pursued what worked rather than what impressed. Machine learning’s culture of benchmarks, rigorous evaluation, and modest claims — so different from AI’s grand pronouncements — was forged in the winter’s cold. When the thaw came, the field that emerged was more empirical, more humble, and more effective than the one that froze.

What Survived and Why It Mattered

What survived the second winter shaped everything after. Bayesian and statistical methods became the field’s mathematical backbone. The machine learning community’s evaluation culture (shared datasets, benchmark competitions) became the template for progress. Neural network research, kept alive by a handful of believers, was positioned for the deep learning explosion. And the hard lesson — that hand-coded knowledge doesn’t scale, but learning from data does — became the field’s central dogma.

The survivors were also psychologically distinctive: the researchers who stayed through the winter developed a permanent skepticism toward hype, a preference for empirical results over grand theory, and a certain stubborn independence. When the deep learning breakthroughs came, they came from this culture — careful, benchmark-driven, unimpressed by press releases. The winter selected for the traits the spring needed.

Institutionally, the winter taught diversification: don’t depend on one patron, one paradigm, or one application. The modern AI ecosystem — academic, industrial, open-source, global — is far more resilient than the concentrated, hype-driven field that froze. Whether it’s resilient enough for the next shock is the open question.

Lessons for the Current Boom

The second winter’s lessons for today are specific and uncomfortable. Beware the scaling assumption: expert systems worked in demos and narrow deployments but failed at scale — current AI’s impressive demos deserve the same scrutiny about deployment reality. Beware the business model: the Lisp machines died from economics, not technology — today’s AI economics (massive compute costs, uncertain revenue) deserve the same scrutiny. Beware the label dilution: when everything is “AI,” the term’s collapse takes the legitimate work down too.

But also note the differences: today’s AI learns from data (solving the knowledge bottleneck), has genuine mass adoption (not just enterprise pilots), and rests on diversified funding and applications. The second winter’s specific causes — hand-coded knowledge, specialized hardware dependence, narrow commercial base — don’t directly apply. New failure modes would be new, not repeats.

The deepest lesson: winters come from the gap between promises and delivery, not from technology failing. Manage the promises — or rather, distrust those who won’t — and the technology gets the time it needs to mature. The field that survived the second winter did so by keeping its head down and doing the work. That’s still the strategy.

Frequently Asked Questions