Book Notes
The Shortest History of AI
Toby Walsh
2025 – The Experiment – 207 pages
Dave’s Summary
Artificial intelligence is around us every day. It’s not going away. But don’t worry, it can’t think, and it is not about to do away with its human inventors. Toby Walsh’s book, “The Shortest History of AI,” traces its development, explains how it works, and outlines how it will continue to develop.
AI began at a 1956 meeting led by John McCarthy with a bold goal to build thinking machines. Early systems relied on rules and logic. They worked well in clear tasks like games, but failed in real life. This period showed how hard it is to build true intelligence because our world lacks the kind of clear rules that computers require.
AI went through cycles of hype and disappointment. Early excitement led to funding. But progress was slow because it was difficult to integrate human judgment with the rules that computers need.
Progress slowed, and support dropped during the AI winter. Expert systems ushered in a new wave by mimicking human rules in narrow domains. They worked well in small areas but failed to handle real-world complexity. The key issue was the knowledge gap. It was hard to capture human judgment in simple rules.
The field shifted when learning replaced strict coding. Neural networks, computational models inspired by the human brain, were developed. They use interconnected layers of artificial neurons to learn patterns, make predictions, and process complex data.
Transformers changed how machines process language. They break text into tokens, turn them into numbers, find patterns, and generate new text. This design powers GPT models. Each new version grew larger and more capable. GPT-1 was small. GPT-3 and GPT-4 became massive, trained on vast amounts of data from the internet.
ChatGPT made this tech widely available. It works like advanced auto-complete, predicting likely words and ideas. It can write, code, and answer questions. Still, it does not know the truth. It predicts what seems likely, which leads to errors or made-up facts.
Reinforcement learning improves results by incorporating human feedback. It rewards helpful answers and filters harmful ones. Yet limits remain. These systems can drift, make errors, or reflect biases in the training data.
AI also uses probability to handle uncertainty. Methods like Bayes’ rule help systems judge risk and make better choices. This powers tools like spam filters and recommendation systems.
Today, AI is advancing rapidly amid significant investment and global competition. Governments and companies are racing to lead. The technology brings gains but also risks like bias, job loss, and misuse. The future may bring machines that match or exceed human intelligence, but progress will take time and careful control.
5 key highlights
• AI began in 1956 with a clear goal to build thinking machines.
• Early rule-based systems worked in games but failed in real-world tasks.
• Learning from data replaced strict coding and drove modern progress.
• Transformers and large models power tools like ChatGPT.
• AI predicts likely answers, not true ones, which leads to errors.
Most important lesson
AI feels smart, but it does not understand the way we do. It can’t navigate the world the way our brain can. You should still question its answers and use your own judgment.
Memorable Highlights
You might be getting excited, then, about the robot butler, robot cleaner and robot cook soon to be turning up in your home thanks to all the recent advances in AI. I have bad news for you: I fear that it’s going to be a long time before we have robot butlers, cleaners and cooks. In part, this is because the home is a chaotic and messy environment.
GPT stands for Generative Pretrained Transformer. It is an ugly name. “Generative” means that the model can generate text. “Pretrained” means that the model is pretrained with no particular goal in mind other than to discover features in the training data.
In its aim to please, ChatGPT will always give you an answer, whether or not it is confident.
It was therefore a rather bold move for OpenAI to launch ChatGPT, knowing full well that it would frequently make things up. Sam Altman argued that “the world needs to get used to this. We need to make decisions together.” And it became the mantra of OpenAI that it should get its products into the hands of the public, despite their flaws, and iterate fast to deal with any problems. Unfortunately, hallucination isn’t a problem that is going to go away. Indeed, it’s not something you want to disappear completely. The only reason that ChatGPT can write a sonnet about falling in love with your laptop in the style of Shakespeare is because it can “hallucinate.”
What I take from the success of ChatGPT is that we’ve overestimated not machine intelligence but human intelligence. A lot of human communication is quite formulaic. We don’t engage our brains as often as we think we do. Much of what we say is formulaic. And these formulas have now been taught to computers.
Large language models like GPT-2 and GPT-3 can behave as if they have attention deficit disorder. You will recall that these models are random, producing different output each time they are run. And the problem with randomness is that the dice eventually go against you. Mathematicians call it gambler’s ruin.
Purchase the book here and I will receive a small commission at no extra cost to you. But, by doing this, you help support my reading habit! Also, I do not get paid to read or review books. All of the books I summarize were either purchased by me or borrowed from my local library.
Dave Edwards Media 