GPT-3.5 vs ELIZA – How did a 1960s chatbot beat ChatGPT?
Table of Contents
OpenAI has a throne to defend, as the world’s most advanced company in AI. That position is not an easy one to keep, and Google is coming for the crown. With the release of the Gemini AI model, parent company Alphabet challenged not only OpenAI’s GPT-4 Turbo flagship foundation model, but human expert intelligence, surpassing us for the first time in history. However, it seems that it may not take a tech giant to dethrone the king. Has ELIZA, an early AI chatbot for the 1960s already beaten OpenAI’s ChatGPT at the Turing test?
What is ELIZA?
Eliza (Stylized ELIZA) was one of the first iterations of what we know as an AI chatbot today. Called Chatterbots at the time, these were computer programs built to succeed at the Turing test, which tests “a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.” Proposed by Alan Turing in 1950, the test has long stood as a final frontier for computer science, allowing us to measurably distance man from machine. Until now.
Despite having very different architectures from modern AI systems, ELIZA displayed natural language processing capabilities. The training methods we use today, based on LLMs (Large language models) were completely impractical 60 years ago, as the world did not have the data infrastructure of a single tech giant company today. Of course, methods were less advanced, as was technology as a whole. However, what Joseph Weizenbaum created between the years of 1964 to 1967 may be more convincing than today’s methods.
Essential AI Tools
Did a 1960s Chatbot beat OpenAI’s ChatGPT at the Turing test?
In a test that one researcher from UC San Diego has described as “embarrassing” for OpenAI, this early AI ‘Chatterbot’ was able to beat today’s leading iteration in one very important respect.
The world has been waiting for the moment at which machine passes man in intelligence. This pivotal moment, at which AGI (Artificial General Intelligence) is born, is hotly debated and wrongly predicted each day. It seems, however, that we are further behind than we thought.
In a study of 652 participants, ELIZA, created almost 60 years ago today, was able to outperform OpenAI’s ChatGPT, fooling humans 27% of the time, compared to just 14% of GPT-3.5. The goal of the AI here is to trick a human into believing that they are chatting online with another human. A sufficiently adept AI, who has learned from the linguistic style and patterns of thousands and thousands of novels, poems, corporate documents, and scientific papers, will be able to mimic a human without scrutiny.
We’d be remiss to not point out that the most recent foundation model from OpenAI, GPT-4, actually scored 41%. This does beat ELIZA, and considering how many billions of dollars have been poured into Sam Altman’s AI giant, we’d sure hope so.
About the Author