Last Updated on
Every day there seems to be a new story about ChatGPT. The latest headline surrounds the AI bot’s potential to pass a standardized law assessment. But, can ChatGPT pass the bar exam?
ChatGPT has been turning heads ever since its release in November last year. In the last 24 hours, Buzzfeed even announced it will be using the AI model to write some of its content – pretty big news!
The developers at OpenAI have been showing us what this language model can do, and it’s scary. With companies implementing these models to replace human writers and now its potential to pass exams, what more can we expect from AI in the future? Let’s explore whether or not this AI language model can pass the bar.
What is the bar exam?
The bar exam is the final assessment for someone looking to become a licensed lawyer. In order to practice law in a particular state in the US, one must pass the state’s specific bar exam.
The test has a reputation for being very difficult, with overall pass rates sometimes reaching as low as 30%.
If you’ve ever wondered what the exam looks like, you’re not alone. The bar exam questions can take two different forms, multiple choice, and essay. The tests are usually split across many separate sessions sometimes taking several hours to complete.
Did ChatGPT pass the bar exam?
Professors from Illinois Tech and Michigan State College of Law put GPT-3.5 up against the multistate bar exam (MBE). In their research paper, they found that the model “achieves a passing rate on two categories of the Bar” and achieved similar marks to human “test-takers” on another.
The results are outstanding, and they believe that GPT-3.5’s answers show that the model has a “general understanding of the legal domain”. If you are interested in reading the entire research paper, you can find it here.
So we know that GPT-3.5 can pass the bar, but what about other law exams? Well, other law professors at the University of Minnesota also tested out OpenAI’s language model on another exam scenario. They made Chat GPT take four assessments covering different courses in the university’s law degree. The results were fascinating.
The chatbot only managed to produce mediocre scores of just a C+ but still passes nonetheless. The findings also seem to suggest that ChatGPT would be advanced enough to potentially secure a degree in law if it were a student at the college.
Did GPT-4 pass the bar exam?
OpenAI’s latest model, GPT-4, not only got admitted into law school but also completely aced the Bar exam. For context, the multimodal model scored in the 88th percentile for the LSAT or Law School Admission Test. And, for the Bar? Well, this model scored in the top 90th percentile. This is a massive achievement considering OpenAI’s later model, GPT-3 could only score within the bottom 10% of test-takers.
Final thoughts
There’s no doubt about it, OpenAI’s language models are pretty clever tools. Its ability to pass exams is very clear and its recent passing mark on the bar has caught many by surprise. If you found this article interesting, be sure to read another article here.
- NOW READ Can Chat GPT pass the Turing test?