🗓️ Week 10 - Case Study: Use of LLMs in sensitive contexts
2025/26 Autumn Term
This lecture will mainly focus on a case study: How should the use of large language models (e.g ChatGPT) be handled in sensitive contexts (e.g legal, medical or child protection contexts)?
👨🏫 Lecture Slides
Either click on the slide area below or click here to view it in fullscreen. Use your keypad to navigate the slides. You can also find a PDF version on Moodle.
You can download the newspaper case study materials (left-over from last week) by using the link below (.zip archive containing a Jupyter notebook (.ipynb) and a serialized dataframe (.pkl) ):
Today’s main case study will look at the following articles:
UK legal cases
- Robert Booth (June 2025). “High Court tells UK lawyers to urgently stop misuse of AI in legal work”. The Guardian – (Booth 2025)
- Jamie Grierson (Oct 2025). “Barrister found to have used AI to prepare for hearing after citing fictitious cases”. The Guardian – (Grierson 2025)
- Khaled Moyeed (May 2025). “AI, Fake Cases, and the Courts: A Cautionary Tale”. Gunnerscooke LLP (Focus on the UK Ayinde case) – (Khaled Moyeed 2025)
- David Allen Green (May 2025). “A close reading of the ‘AI’ fake cases judgment” - Legal blogger’s detailed analysis – (Green 2025)
- Hibaq Farah (2023). “Court of appeal judge praises ‘jolly useful’ ChatGPT after asking it for legal summary”. The Guardian – (Hibaq Farah 2023)
US legal cases (for comparison):
- Khari Johnson (Sept 2025). “California issues historic fine over lawyer’s ChatGPT fabrications”. CalMatters – (Johnson 2025)
- Jared Perlo and Angela Yang (Oct 2025). “These people ditched lawyers for ChatGPT in court”. NBC News – (Perlo and Yang 2025)
- David Lat (Feb 2025). “A Major Law Firm’s ChatGPT Fail”. Original Jurisdiction – (Lat 2025)
UAE laws rewriting
- Chloe Cornish (Apr 2025). “UAE set to use AI to write laws in world first”. The Financial Times – (Cornish 2025)
Child protection:
- Josh Taylor (2024) . “AI ban ordered after child protection worker used ChatGPT in Victorian court case”. The Guardian – (Josh Taylor 2024)
Healthcare:
Josh Taylor. (July 2023). “AMA calls for stronger AI regulations after doctors use ChatGPT to write medical notes” . The Guardian – (Josh Taylor 2023)
If you want to read further on the use of LLMs in the legal system
Luke Taylor (2023). “Colombian judge says he used ChatGPT in ruling”. The Guardian – (Taylor 2023)
Amy Guthrie (2023). “Colombian Judge Uses AI Tool ChatGPT in Court Ruling”. Law.com International – (Amy Guthrie 2023)
Benjamin Weiser (2023). “Here’s What Happens When Your Lawyer Uses ChatGPT”. The New York Times – (Weiser 2023) (You can also access this article through LSE Library Search)
Dan Milmo (2023). “Two US lawyers fined for submitting fake court citations from ChatGPT”. The Guardian – (Milmo 2023)
Simon Gibson (2023). “Cautioning the legal sector: AI is a tool, not a panacea”. Legal Futures – (Simon Gibson 2023)
Pranshu Verma and Will Oremus (2023).“How lawyers used ChatGPT and got in trouble”. The Washington Post – (Pranshu Verma and Will Oremus 2023)
Leyland Cecco (2024). “Canada lawyer under fire for submitting fake cases created by AI chatbot”. The Guardian – (Leyland Cecco 2024)
🎥 Looking for lecture recordings? You can only find those on Moodle.
📚 Recommended Reading
- Check the end of slides for the list of references cited in the lecture.
- Check the 📔 Syllabus for this week’s complete list of indicative and recommended readings.
📟 Communication
- Post your reflections, questions, and links on Slack.
- Book office hours if you want to discuss your coursework with either me, Barry or Stuart.
📝 Coursework (to bring to next week’s lecture)
Find one or two examples data science/AI applications with ethical issues
- What kinds of issues are these?
- Where do they stem from?
Be prepared to discuss your examples with the rest of the class