Before the AI Boom: A Look Back in Time!

AI the future, AI the present, but what about AI the past? I am a passionate teacher and have always enjoyed classroom discussions when teaching Deep Learning and Machine Learning concepts to my students. My doctoral work introduced me to Deep Learning, and I was intrigued by its history and the origins of the field. I owe a debt of gratitude to my PhD supervisor, the late Dr. Ravi Kulkarni, for introducing me to Deep Learning and for making our sessions so interesting. He would often begin by exploring the history of the topic we were about to discuss. This sparked my interest, and I picked up this wonderful habit from my professor.

My Classroom Talks

My classroom introductions to this subject often begin with a video case study. I ask my students to watch the BBC documentary Calculating Ada: The Countess of Computing, presented by another favourite mathematician of mine, Dr. Hannah Fry. The documentary offers insights into the life of Ada Lovelace, widely regarded as the first computer programmer, and introduces us to the Analytical Engine. It is a fascinating watch for anyone interested in the history of computing and artificial intelligence. I teach a diverse, heterogeneous audience, and this documentary does a wonderful job of introducing students to the subject while leaving them intrigued. More importantly, it encourages them to think about how far we have come and how some of the questions we ask today were being explored centuries ago.

Lady Ada Lovelace and Charles Babbage

Charles Babbage, often called the Father of the Computer, designed the Analytical Engine, a proposed general-purpose mechanical computing machine. Lady Ada Lovelace, who received a mathematical education under the guidance of her mother, tutors, and other teachers, became fascinated by the possibilities of this machine. She and Charles Babbage shared a remarkable intellectual relationship. In 1843, while translating an article by Luigi Federico Menabrea about Babbage’s Analytical Engine, Ada added extensive notes of her own. These notes included an algorithm for calculating Bernoulli numbers, widely regarded as the first published computer program intended for a general-purpose computing machine. One of her most famous observations was:

“The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform. It can follow analysis; but it has no power of anticipating any analytical relations or truths. Its province is to assist us in making available what we are already acquainted with. ”

— Ada Lovelace, Notes on the Analytical Engine, 1843. This statement has always fascinated me. My interpretation is that Ada was drawing attention to a fundamental question: if a machine operates according to instructions supplied by humans, can it truly originate something of its own? What would it take for us to call a machine intelligent? Would it need to surprise us with its originality, creativity, or agency without being explicitly guided by humans? These are questions I still find relevant today.

The Objection

Fast-forward to the period between 1939 and 1945. Alan Turing was working at Bletchley Park during World War II, contributing to the cryptanalysis of Enigma-encrypted German communications. His work involved machines, algorithms, and complex calculations. Yet his contribution to the history of computing extended far beyond codebreaking. In 1950, Turing published his landmark paper, Computing Machinery and Intelligence, in which he opened with a deceptively simple question:

“Can machines think?” Rather than attempting to settle the meaning of thinking directly, Turing proposed what became known as the imitation game, now widely associated with the Turing Test. In the same paper, he discussed what he called “Lady Lovelace’s Objection”, the argument that a machine could only do what we knew how to order it to do. Turing challenged the idea that this observation settled the question of machine intelligence. He explored how a machine might learn and produce outcomes that its designers had not explicitly anticipated. One of the ideas I find particularly interesting is that a machine can surprise its creators. Even when we design a system, we may not be able to predict every result it will produce. For me, this raises an important question: when a machine produces something unexpected, are we witnessing intelligence, creativity, or simply the consequences of a complex computational process?

The Dartmouth Workshop and the Birth of AI as a Research Field

In 1956, a landmark workshop at Dartmouth College helped establish Artificial Intelligence as a distinct field of research. The Dartmouth Summer Research Project on Artificial Intelligence was associated with John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon. The researchers explored whether aspects of learning and intelligence could be described precisely enough for a machine to simulate them. The early years of AI saw substantial interest in symbolic approaches: representing knowledge through symbols, rules, and logical operations. Programming languages such as Lisp, developed in the late 1950s, became important to AI research. Prolog emerged later, in the early 1970s, and became associated with logic programming. But AI’s history was never limited to symbolic reasoning. Researchers were also exploring mathematical models of learning, statistical methods, and artificial neural networks. The foundations of neural-network research predate the Dartmouth workshop, including the influential work of Warren McCulloch and Walter Pitts in 1943. Over the following decades, different approaches to machine intelligence developed alongside one another. Some focused on explicitly represented knowledge and logical reasoning; others focused on learning patterns from data. This brings us to another important part of the story: the history of statistical learning.

A Parallel Journey: The Evolution of Statistical Learning

While AI was developing as a research field, mathematics and statistics were providing tools to understand data, estimate relationships, and make predictions. Here is a simplified timeline of selected milestones:

  • Early nineteenth century: Least-squares methods and the foundations of regression.
  • 1936: Fisher’s influential work on linear discriminant analysis.
  • Mid-twentieth century: Development and wider use of logistic regression for categorical outcomes.
  • 1972: Generalised linear models formalised by Nelder and Wedderburn.
  • 1980s: Advances in computational methods, tree-based models, generalised additive models, and renewed interest in neural networks.
  • 1990s: The modern support vector machine formulation and broader development of statistical learning methods. These are selected milestones rather than a complete chronology. The development of AI and statistical learning was not one straight progression; multiple research traditions influenced one another over time.

Where Do Mathematics, Statistics, Computer Science, and AI Intersect?

Figure 1: A simplified illustration of the relationships between mathematics, statistics, mathematical statistics, computer science, data mining, machine learning, deep learning, and artificial intelligence. The diagram is a conceptual map rather than a strict classification of disciplines. The boundaries overlap, and different researchers may represent these relationships differently. When we look at the history of AI and statistical learning together, we can see that today’s systems draw on ideas developed over many decades. Symbolic reasoning, statistical methods, supervised and unsupervised learning, reinforcement learning, and deep learning represent different approaches to different kinds of problems. Modern generative AI builds on developments in deep learning, including transformer architectures, which have become central to many large language models. It is worth remembering that these approaches did not simply replace one another. They continue to coexist, and each has its own strengths and limitations.

My Two Cents: How Intelligent Are Our Machines Today?

Going back in time and reflecting on where we stand today in terms of machine intelligence, I find myself asking: Intelligent? Superintelligent? Or are we still figuring out what intelligence means when it comes to machines? With the emergence of generative deep learning, I believe we have made remarkable progress towards what we commonly perceive as creativity. Yet I think we still have important questions to answer about originality and agency. Let me explain what I mean. A generative deep learning model learns patterns and statistical regularities from a dataset containing images, videos, text, or other forms of data. It then uses what it has learned to generate new samples that share characteristics with the training data. The key idea is the learned representation of the data and the patterns captured by the model. Humans have selected the data, designed the learning process, and defined the objectives within which the model operates. And yet, the output can surprise us. This brings me back to Ada Lovelace’s observation about a machine’s ability to originate something, and to Turing’s exploration of machine intelligence and unexpected results. We are fascinated by the outputs of generative AI because they can be novel, convincing, and sometimes difficult to distinguish from human-created work. But does generating something unexpected mean that a machine is creative? Does producing something novel mean it is original? And does the ability to perform a sequence of tasks mean that it possesses agency? I don’t think these questions have simple answers. For me, they are an invitation to examine the boundaries of what our machines can do and what we mean when we describe them as intelligent.

AlphaGo: When a Machine Surprised the Experts

Another favourite case study I use with my students is the documentary AlphaGo, which follows DeepMind’s AI system during its 2016 match against Go champion Lee Sedol. Go is a strategy board game with an enormous number of possible positions. Its complexity makes it difficult to solve using straightforward enumeration of all possible moves. AlphaGo combined deep neural networks with tree-search techniques. Its development involved learning from expert human games and reinforcement learning through self-play. One of the most memorable moments came during Game 2, when AlphaGo played Move 37, a move that surprised expert commentators and Lee Sedol himself. The move was so unexpected that it challenged human intuitions about how the game should be played. AlphaGo went on to win the game and ultimately won the five-game match by four games to one. For me, Move 37 is a wonderful example of why machine learning is so fascinating. The system produced a move that human experts had not anticipated, demonstrating a strategy that appeared unconventional yet proved effective. But does that make the move creative? Was it original? Or did the system identify a strong possibility that human players had overlooked? I find this distinction fascinating. It is one thing for a machine to learn patterns from data and produce an unexpected result. It is another to determine what that result tells us about creativity, originality, or agency. And that is precisely why I believe we still have much to explore.

A Few Resources I Recommend

If this journey through the history of AI has piqued your interest, here are a few resources I enjoy sharing with my students.

Documentaries and Further Reading

  • Calculating Ada: The Countess of Computing — a documentary presented by Dr. Hannah Fry that explores Ada Lovelace’s life and contributions.
  • Mystery of Numbers — a BBC documentary series for those who enjoy mathematics and its history. Verify the exact programme title and availability before watching.
  • AlphaGo — a documentary about DeepMind’s Go-playing system and its historic match against Lee Sedol.
  • Computing Machinery and Intelligence — Alan Turing’s original 1950 paper.
  • Ada Lovelace’s 1843 notes — the historical text that includes her observations on the Analytical Engine.

    My Favourite Books

  • Deep Learning with Python — François Chollet, published by Manning.
  • The Elements of Statistical Learning: Data Mining, Inference, and Prediction — Trevor Hastie, Robert Tibshirani, and Jerome Friedman.

    My Favourite YouTube Channels and Courses

  • Dr. Balaraman Ravindran — Introduction to Machine Learning.
  • Alexander Amini and Ava Soleymani — MIT Deep Learning course resources.
  • Ajay Halthor — CodeEmporium.
  • Grant Sanderson — 3Blue1Brown.
  • Normalized Nerd.
  • Computerphile. These are some of the resources that have helped me learn, reflect, and find new ways to explain complex ideas in the classroom. I hope you find something here that sparks your own curiosity.

    Acknowledgements

    I would like to thank:

  • The late Dr. Ravi Kulkarni, my PhD supervisor, for introducing me to Deep Learning and inspiring my interest in the history behind the concepts we study.
  • Anant Shrivastava, for encouraging me to begin writing this blog.

    In Summary…

    This article has been a journey back in time, from Ada Lovelace’s reflections on the Analytical Engine to Turing’s questions about machine intelligence, from the early days of AI research to the statistical learning methods and generative models we use today. I want to leave you with a thought on examining the boundaries of what our machines can do and what we mean when we describe them as intelligent.