In collaboration with Google, the Computer History Museum (CHM) has made a significant piece of AI history available to the public by releasing the original source code for AlexNet on CHM’s GitHub repository.
What Is AlexNet?
AlexNet is a pioneering artificial neural network designed for image recognition. Developed in 2012 by University of Toronto graduate students Alex Krizhevsky and Ilya Sutskever under the guidance of Geoffrey Hinton, this network set the stage for the modern deep learning revolution in AI.
The Birth of Deep Learning
Geoffrey Hinton is often hailed as one of the founding figures of deep learning—a branch of AI that leverages neural networks. Early neural network research began in the late 1950s with simple three-layer models, like the perceptron developed by Frank Rosenblatt. However, these early designs quickly revealed their limitations, and by the 1970s, neural networks had fallen out of favor among many researchers.
A revival occurred in the 1980s when cognitive scientists at the University of California, San Diego, reintroduced neural networks under the banner of “connectionism.” After completing his Ph.D. at the University of Edinburgh, Hinton joined UCSD as a postdoctoral fellow. There, he collaborated with David Rumelhart and Ronald Williams to rediscover the backpropagation algorithm—a breakthrough that allowed multi-layer networks to learn complex features for tasks such as language and vision. This algorithm remains a cornerstone of modern deep learning.
By 1987, when Hinton moved to the University of Toronto, his work helped transform the institution into a hub of deep learning research. His influence extended to his students, including Yann LeCun—now a leading scientist at Meta—who later demonstrated the power of convolutional neural networks for tasks like handwritten digit recognition.
The Role of ImageNet and GPUs
Despite these advancements, neural networks struggled to consistently outperform other machine learning methods. Two key developments changed this landscape:
- Big Data: The advent of the Internet brought forth vast amounts of image data, exemplified by ImageNet. Initiated by Stanford professor Fei-Fei Li, ImageNet was a groundbreaking dataset that cataloged millions of images across thousands of categories using WordNet taxonomies and crowdsourced labeling via Amazon’s Mechanical Turk.
- GPU Computing: Training deep neural networks requires extensive computation, particularly for matrix multiplications. The emergence of programmable 3D graphics chips (GPUs), notably advanced by NVIDIA and its CUDA programming platform (released in 2007), provided the necessary computational boost.
In 2012, these elements converged in AlexNet—a deep neural network trained on the expansive ImageNet dataset using powerful NVIDIA GPUs—delivering results that redefined the field.
How AlexNet Came to Life
During the late 2000s, Hinton’s graduate students began exploring the use of GPUs for training neural networks, achieving early successes in speech recognition. Ilya Sutskever, one of these students, recognized that the performance of neural networks could scale with more data—a hypothesis perfectly tested with the release of ImageNet.
In 2011, Sutskever persuaded fellow student Alex Krizhevsky—renowned for his skill in optimizing GPU performance—to develop a convolutional neural network specifically for ImageNet. Leveraging an existing CUDA-based project called cuda-convnet, Krizhevsky expanded it to support multiple GPUs and adapted it for the much larger dataset. Working from his parents’ home with two NVIDIA cards, he iteratively refined the network until it outperformed all its competitors. The network was eventually named AlexNet in honor of Krizhevsky.
The groundbreaking paper on AlexNet, presented by Krizhevsky in October 2012 at a computer vision conference in Florence, marked a turning point. While some veteran researchers were skeptical, peers like Yann LeCun recognized its transformative potential. After AlexNet’s debut, neural networks quickly became the de facto approach in computer vision research, paving the way for subsequent advances in AI—from voice synthesis to game-playing algorithms, culminating in innovations like ChatGPT.
Releasing the AlexNet Source Code
Recognizing the historical significance of AlexNet, CHM reached out to Alex Krizhevsky in 2020 regarding the possibility of releasing the network’s original source code. Through his connection with Geoffrey Hinton—then at Google—CHM began a five-year collaboration with Google’s team to negotiate the release. They identified the specific version of AlexNet’s source code that was used in 2012, distinguishing it from various recreations available online.
Today, CHM proudly presents the official 2012 version of AlexNet’s source code, a pivotal artifact that helped transform AI as we know it. You can access this code directly on CHM’s GitHub page.
Acknowledgments
Special thanks go to Geoffrey Hinton for his insightful contributions and for reviewing the content, as well as to Cade Metz and Alex Krizhevsky for their clarifications. Appreciation is also extended to David Bieber and the entire team at Google for their instrumental role in securing the release of this historic source code.
References
- Fei-Fei Li, The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI. First edition, Flatiron Books, New York, 2023.
- Cade Metz, Genius Makers: The Mavericks Who Brought AI to Google, Facebook, and the World. First edition, Penguin Random House, New York, 2022.
This article originally appeared on the Computer History Museum blog.
