Last Updated on
Every day there seems to be a new story about Chat GPT. The latest headline surrounds the AI bot’s potential to pass a standardized law assessment. But, can Chat GPT pass the bar exam?
Chat GPT has been turning heads ever since its release in November last year. In the last 24 hours, Buzzfeed even announced it will be using the AI model to write some of its content – pretty big news!
The developers at OpenAI have been showing us what this language model can do, and it’s scary. With companies implementing these models to replace human writers and now its potential to pass exams, what more can we expect from AI in the future? Let’s explore whether or not this AI language model can pass the bar.
What is the bar exam?
The bar exam is the final assessment for someone looking to become a licensed lawyer. In order to practice law in a particular state in the US, one must pass the state’s specific bar exam.
The test has a reputation for being very difficult, with overall pass rates sometimes reaching as low as 30%.
If you’ve ever wondered what the exam looks like, you’re not alone. The bar exam questions can take two different forms, multiple choice, and essay. The tests are usually split across many separate sessions sometimes taking several hours to complete.
Bar exam on Chat GPT
Professors from Illinois Tech and Michigan State College of Law put GPT-3.5 up against the multistate bar exam (MBE). In their research paper, they found that the model “achieves a passing rate on two categories of the Bar” and achieved similar marks to human “test-takers” on another.
The results are outstanding, and they believe that GPT-3.5’s answers show that the model has a “general understanding of the legal domain”. If you are interested in reading the entire research paper, you can find it here.
So we know that GPT-3.5 can pass the bar, but what about other law exams? Well, other law professors at the University of Minnesota also tested out OpenAI’s language model on another exam scenario. They made Chat GPT take four assessments covering different courses in the university’s law degree. The results were fascinating.
The chatbot only managed to produce mediocre scores of just a C+ but still passes nonetheless. The findings also seem to suggest that Chat GPT would be advanced enough to potentially secure a degree in law if it were a student at the college.
Final Thoughts
There’s no doubt about it, OpenAI’s language models are pretty clever tools. Its ability to pass exams is very clear and its recent passing mark on the bar has caught many by surprise. If you found this article interesting, be sure to read another article here.
- NOW READ Can Chat GPT pass the Turing test?