⭐ First Summative Essay (worth 30%)
2023/24 Autumn Term
🎯 OBJECTIVE: Write a 1,500-word essay using Quarto Markdown and Zotero 1
⌛ DUE DATE: 20 December 2023.
🗺️ Context
Since the first weeks of this course, we have been discussing the fact that algorithms are not neutral and that their design, starting with which data feeds into them, is a series of not so objective judgement calls. These judgments calls have real life consequences. Thinking along these lines, in this essay, you will be asked to research the Ofqual’s A-levels “mutant algorithm” fiasco and then reflect on the potential dangers of ill-designed algorithms this case reveals. More than familiarising yourself with this particular scandal, you will also be asked to reflect on the ethics of replacing human judgment and decision by supposedly objective AI algorithms in high stakes sensitive contexts.
The Ofqual’s A-levels “mutant algorithm” fiasco is a case study of the dangers of poor algorithmic design (Paulden 2020) (Kelly 2021) (Hern 2020). In 2020, in response to the COVID-19 pandemic, both A-levels and GCSE exams were cancelled and Ofqual, the regulator of qualifications, exams and tests in England was tasked by the then secretary of state for education to ensure “ensure, as far as is possible, that qualification standards are maintained and the distribution of grades follows a similar profile to that in previous years”. Ofqual then produced an algorithm to compute the A-levels grades. After public outcry and criticism, the grades computed by the algorithm had to be withdrawn(Sabbagh 2020).
📖 Suggested readings
🗞️ In the news
Sabbagh, Dan. 2020. “Algorithm and blues: Gavin Williamson’s week of A-level chaos.” The Guardian, August.
Hern, Alex. 2020. “Ofqual’s A-level algorithm: why did it fail to make the grade?” The Guardian, August.
🎓 Academic
Kelly, Anthony. 2021. “A tale of two algorithms: The appeal and repeal of calculated grades systems in England and Ireland in 2020.” British Educational Research Journal 47: 725–741.
Paulden, Tim. 2020. “A Cutting Re-Mark.” Significance 17 (5): 4–5, October 2020.
Kizilcec, René F., and Hansol Lee, 2022. “Algorithmic fairness in education”. In The ethics of artificial intelligence in education (pp. 174-202). Routledge.
Feel free to use other sources as well, but make sure you cite them properly. Why not try using the recommendations from Week 10 lecture to find more sources?
🕸️ LSE Blogs
Scenario and your task
Consider you are writing an online article for LSE Blogs with the topic: “‘Mutant algorithms’? What did we learn from the Ofqual 2020 A-levels fiasco?”. You do not need to use this exact title, just make sure you are writing about the same topic.
Your target audience is a general audience of non-experts, so you should try to explain the main ideas and arguments clearly and concisely. You should also try to make your writing as engaging as possible so that your readers can easily follow your arguments and understand the main points.
📝 Instructions
What we are looking for
From the point of view of assessment, the main thing we are looking for here is: can you draw on the topics we have been discussing in class when reflecting and writing about the Ofqual “mutant algorithm” fiasco? (Weeks 01 - Week 11)
Expanding on that, we could say that we would be also looking to see the following in your essay:
- can you make connections between the ideas we have discussed and the case study?
- can you reflect beyond this specific case and think about the issues it uncovers and their wider implications?
- can you explain the main ideas and arguments clearly and concisely?
- can you use the right tone and style to engage your readers?
Unlike in Formative 3, you don’t have to use all the references we gave you in this summative topic but you should use at least one reference from the list and cite at least 7 references in your essay.
👉 Avoid making explicit references to the course (e.g., writing things like “As we saw in Week 05…”), as this would go against the spirit of the exercise, which is to write to a general audience. Instead, refer to the bibliography we have provided (as well as additional relevant bibliography you have collected) and try to make connections between the ideas we have discussed and the case study. The same goes for AI-generated text.
⚠️ Any ideas, arguments or results that were not produced by your mind must be cited in the references.
💡 Tips
Note that some of the suggested readings might be challenging to understand, as some of them rely on a lot of technical jargon and are written for a specialized audience. What is most important is that you get a sense of the main ideas and arguments and try to relate them to the techniques and things we have discussed in class.
👉 Why not organise a study group with your classmates to discuss these readings? You can create a shared 📚 Zotero library to share your notes and resources. Feel free to use Slack to organise your study group.
Group work is a great way to learn and to get to know your classmates and it is highly encouraged. Just make sure your essay contains your own original ideas and that you cite any sources you use. The same applies to AI tools (more on that in the next section).
🤖 Using AI help?
You are allowed to use Generative AI tools such as ChatGPT to help you write your essay. If you do use it, however minimal use you made, you are asked to report the AI tool you used and add an extra section to your essay to explain the extent to which you used it (this won’t count towards the word limit).
Note that, while these tools can be helpful, they tend to generate responses that sound convincing but are not necessarily correct. Another problem is that they tend to generate responses that are formulaic and repetitive; thus, limiting your chances of getting a high mark.
In effect, you are asked to explain the following:
- What AI tool did you use?
- How did you use it? For example, did you use it to generate ideas, write a draft, proofread your essay, etc.?
- How much of your essay was written by the AI tool? For example, did you feed it the entire prompt and it wrote the entire essay? Or did you feed it guided questions?
- If you didn’t edit the AI tool’s output, what was the output like? For example, did it produce a coherent essay?
- What did you do to make sure that the AI tool did not produce gibberish? and that the essay was not formulaic.
- Importantly, how did you ensure that the essay did not contain any plagiarism?
Look up our Generative AI Policy to see in more details what you are allowed or aren’t allowed to do when it comes to Generative AI use for this summative and for examples of how to reference use of Generative AI in your essay.
🧱 Structure
You are free to structure your essay however you want. Be creative here! There is no need to adhere to a rigid standard 5-paragraphs opinion piece. If you need help deciding on the main text blocks, consider the following order of ideas:
A brief introduction to the Ofqual A-levels fiasco and the main arguments and ideas you will be discussing in your essay.
A discussion of the main issues uncovered in the fiasco (e.g algorithmic unfairness and bias brought about by bad algorithmic design).
A broader discussion of the issues of algorithmic bias, fairness and accountability
Potential solutions to the issues raised
Whatever structure you decide on, however, pay very close attention to the logical and smooth flow of your ideas! When reading your essay, we should never wonder why you transitioned from one idea to the next, nor should you bring ideas out of the left field…
Formatting
Your essay must be written in Quarto Markdown. You can use the template provided in the lab. Check, also, the readings listed on Weeks 7 & 8 of the 📔 Syllabus.
Feel free to modify the layout and aesthetics of the template. You can also add images, tables, bullet points, etc. to your essay.
Do not exceed 1500 words (bibliographical references and the (optional) generative AI section do not count).
Additional instructions
Make your writing clear, do not hide your thoughts behind jargon. You are not writing an academic article. Your essay is emulating a communication you would send to work colleagues who have very different educational backgrounds. You can find tips on how to write clearly and make your argumentation coherent in the Resources on clean and logical writing section of the 📄 Resources on argumentation and logical fallacies page on the course website.
Do not plagiarise. It is not that difficult to spot that someone copied content from other sources and, frankly, it is very embarrassing if you get caught. Here is the link to the LSE regulation on plagiarism.
✅ Submission
- Render your Quarto Markdown file to HTML
- ⚠️ IMPORTANT ⚠️: Rename your HTML to
DS101A-2023-summative-essay1-<CANDIDATE_NUMBER>.html, replacing<CANDIDATE_NUMBER>with your candidate number.- For example:
DS101A-2023-summative-essay1-123456.html. - You might be deducted marks if you don’t follow this naming convention.
- For example:
- Upload this file to Moodle under the appropriate assignment.
✋ Getting Help
- If you have any questions about the assignment, please post them on
#help-assignmentschannel on Slack. - Book office hours.
- 🏅 Organise a study group with your classmates.
📑 Marking Scheme
The following is the marking scheme we will use to mark your essay. Note that full marks mean that you have met a particular criterion to an extremely high standard, beyond our expectations. If you did “everything right”, you should expect about 70% of the marks on each criterion.
| Detailed Marking Criteria | Marks |
|---|---|
| 1. The essay contains an introduction to the topic and it’s succinct and concise. | 5 |
| 2. Your writing style conforms to the blog post style, i.e your tone and writing style needs to be relatively informal without being clickbait-y (you can look at BBC, Guardian or LSE Blogs pieces for inspiration) | 4 |
| 3. You have adhered to the 1500-words limit | 1 |
| 4. Your Quarto Markdown formatting is appropriate for a blog post, e.g., style of headers, the inclusion of images, themes, etc. | 5 |
| 5. Your writing style is clear and free of excessive jargon and abbreviations. | 5 |
| 6. The essay concludes with lessons learned | 5 |
| 7. The essay has a logical structure and flow | 10 |
| 8. Your argumentation is coherent | 10 |
| 9. You have succintly and clearly explained the Ofqual “mutant algorithm” fiasco | 2 |
| 10. You have included a simple but thorough (technical) explanation of the “mutant algorithm” | 3 |
| 11. You have given a clear and in-depth explanation of the main issues raised by the Ofqual fiasco | 10 |
| 12. You have convincingly expanded the scope of your reflection beyond the case study (i.e Ofqual fiasco) we provided and built a solid argumentation around the issues uncovered by the case study and their wider implications | 25 |
| 13. Your argumentation is not only solid but creative as well | 10 |
| 14. You are widely read and make use of many judicious references (quantity matters but quality as well!) | 5 |
References
Footnotes
Revisit what you learned about Quarto markdown and Zotero from 💻 Week 04 class. Try using a custom theme to make your essay look more professional (have a look at this page of the Quarto documentation for details on how to customize themes and page looks).↩︎