LSE ME204 · Data Engineering Principles for the Social Sciences
🖥️ Week 01 Day 01 Lecture

Welcome to the Course

Dr Jon Cardoso-Silva

LSE Data Science Institute

🗓️ 13 Jul 2026

Welcome to ME204

10:00 – 10:20

Three weeks from now you will have built a working data pipeline of your own. This morning we cover what that involves and how the course gets you there.

Your lecturer

Dr Jon Cardoso-Silva

🌐 jonjoncardoso.github.io
📧 J.Cardoso-Silva@lse.ac.uk

Assistant Professor (Education)^*
AI and Education Fellow

^* Becoming Associate Professor from 1 Aug 2026.

Background:

• PhD in Computer Science (King’s College London)
• Industry experience in software development and as a lead data scientist

---

Highlighted research:

GENIAL and the LSE AI and Education Fellowship

• How generative AI can strengthen learning in programming courses, rather than replace the hard parts students still need to do
• Evidence and practical guidance for GenAI in data science education

📢 I will talk more about our findings after the break!

:::

:::

Your class teacher

Jonas Weinert
Class Teacher
📧 J.Weinert@lse.ac.uk

• PhD candidate in the Department of Health Policy at LSE.
• Studies how people form health-related behaviours during crises, from vaccination intentions to preventative care.
• Background in econometrics, causal inference, and computational economics.
• Has taught machine learning courses at the LSE Data Science Institute.
• Runs the afternoon labs.

📲 Who’s here?

10:20 – 10:50

Scan the QR code or go to menti.com

Keep Menti open throughout this section!

We will run through a set of quick questions so I can see who is in the room, what you know, and what tools you have used.

---

Mentimeter Live

1️⃣ A Few Words About Working with Data

10:50 – 11:15

You just told me what you think a data engineer does. Let’s see how that compares to what this course actually covers.

Data science is…

“[…] a field of study and practice that involves the collection, storage, and processing of data in order to derive important 💡 insights into a problem or a phenomenon. Such data may be generated by humans (surveys, logs, etc.) or machines (weather data, road vision, etc.), and could be in different formats (text, audio, video, augmented or virtual reality, etc.)”

Source of this definition: Shah, C. (2020). A hands-on introduction to data science. Cambridge University Press.

Emphasis and emojis are of my own making.

The mythical data science unicorn 🦄

knows everything about statistics

able to communicate insights perfectly

fully understands businesses like no one

is a fluent computer programmer

In reality…

We are all jugglers 🤹

• Everyone brings a different skill set.
• We need multi-disciplinary teams.
• Good data scientists know a bit of everything.
  • Not fluent in all things
  • They know when and where to work with others

The Data Science Workflow

The Data Science Workflow (of this course)

People in the field like to joke that 80% of the time and effort spent on a data science project goes to the tasks highlighted in the diagram above.

And this is what this course is about! You will learn some of the most common tools used during this data wrangling process.

---

The meme is real

The struggle is real.
by u/ali_azg in r/dataengineering

Preparing data for analysis doesn’t get as much attention as algorithms (what people usually think of when they hear the term data science), but it is an essential skill if you want to work with data.

What does this have to do with social science?

Data in social science is messy. Here are some examples from real research:

Real examples

• Census tables where borough boundaries changed between releases
• Survey responses with missing fields that mean different things (“not collected” vs “declined to answer”)
• Scraped policy documents in five formats from five government websites

Decisions the data cannot make for you

• Is this data source trustworthy?
• Is the table shaped for the question you are asking?
• Does a missing value mean the same thing across your dataset?
• Does your conclusion go further than the evidence allows?

💡 After the break, we will test that claim directly. You will ask an AI to answer a real data question, and we will see how it does.

2️⃣ Course Structure and the Week Ahead

11:15 – 11:35

Now that I know who’s in the room, let’s look at what you’re here to do.

Assessment

25%	Problem set	✏️ Midterm	Released: Thu 16 Jul 2026 Due: Tue 21 Jul 2026
75%	Website project	🌐 Final Project	Announced: Thu 23 Jul 2026 Due: Fri 31 Jul 2026, 5pm

Full dates and criteria: 📓 Syllabus.

Before This Course, You Already…

This course assumes some basic coding exposure:

• Know what variables are and how to create them
• Understand the different data types (str, int, float, bool)
• Know how to use conditionals (if, elif, else)
• Know how to use loops (for and while)
• Know how to use functions (def)
• Have an intuition for lists and dictionaries

Complete beginners have done well in this course before. Expect a steep first week if you are starting from scratch.

📗 Automate the Boring Stuff, chapters 1 to 5

Course syllabus

Let’s look at the course syllabus together:

📔 Syllabus

---

Everything you need is on Moodle, but we also have a website anyone can open so you can refer to it even after the course is over:

🌐 ME204’s website

:::

:::

This Week, Day by Day

Monday
today

pwd ls cd cat
Navigate a folder tree, write your first script in nano

Tuesday
📅 Week 01 Day 02

🖥️ Files as a machine sees them: Markdown, CSV, JSON, binary
💻 Your first Python: variables, lists, reading a file

Wednesday
📅 Week 01 Day 03

🖥️ REST APIs and curl in the terminal
💻 requests in a Jupyter notebook, fetch weather data from Open-Meteo

Thursday
📅 Week 01 Day 04

🖥️ Why tables beat loops: pandas and DataFrame
💻 Load yesterday’s weather data into a DataFrame
✏️ Midterm released

☕ Coffee Break

11:35 – 11:50

When we come back:

• You ask an AI to answer a real data question
• We compare your AI’s answer to the audited ground truth
• What I have learned from watching students use AI to learn
• How AI fits into your work in this course

3️⃣ Can’t AI Do All of This for Us?

11:50 – 12:20

Large language models have become a standard tool for knowledge workers, especially for tasks that involve writing code. Tech leaders routinely claim these tools can replace whole categories of work. But how much of a data analysis can they actually do on their own, and can you trust what they produce?

The Question for Your Groups

In groups, in whatever AI tool you use, no data handed over:

💬 “How has the frequency of heatwaves changed in London over the last 40 years?”

Try to get a plot of heatwave events per year. The AI has to find the data itself, decide how to define a “heatwave,” build the analysis, and hand you a chart.

The Ground Truth

Data source: Open-Meteo Historical Weather API (ERA5 reanalysis). Definition: 3+ consecutive days at or above 28°C (London’s Met Office threshold). Each event is one unbroken run of 3+ days. Period: 1986 to 10 Jul 2026.

What Did Your Groups Get?

Check your group’s numbers against the chart:

• Did your AI find a trustworthy data source on its own?
• Did its definition of “heatwave” match the one shown here?
• Did the shape of its trend match this chart, even roughly?
• How would you have known it was wrong, if you had not just been shown the real answer?

Where did your group’s attempt diverge from this chart, and why?

4️⃣ What I’ve Learned About AI and Learning

12:20 – 12:40

You’ve just tested whether AI can do the data work for you. Here’s what the research shows about what happens when students rely on it.

What Happens When You Let AI Do the Thinking

The evidence is a little mixed but I like to take the warning signs more seriously: general-purpose chatbots, without any educational design in them, may disrupt our capacity to learn well.

Practice scores went up, exam scores went down

Students who used GPT-4 freely scored +48% higher while they had it. When it was taken away, they scored −17% lower than students who never had it.

They got better at solving problems with AI, but worse at solving them alone. Most did not notice.

Bastani et al. (2025)

ChatGPT improved the writing but not the writer

When ChatGPT was available, revision narrowed to a back-and-forth between the chatbot and the text editor. Students stopped going back to the source material, stopped checking the rubric, and stopped reviewing what they had written. Students paired with a human expert kept doing those checks.

Fan et al. (2025)

Your brain works differently when you start with AI

Students who opened ChatGPT before attempting the task showed weaker brain activity in regions linked to memory and recall. Students who attempted the task first and then used AI did not show the same pattern.

The order matters: attempt first, then use AI to check or refine, and keep only what you can explain.

Kosmyna et al. (2025)

Strong today. How much sticks three weeks later?

Students who studied with ChatGPT had higher quiz scores right after the lab than students with no tools (about 83 vs 60). Three weeks later their quiz scores had dropped to about 66, while the no-tools group was still near 60. Studying with ChatGPT may help on the day. Whether those higher quiz scores last is less clear.

Akgün & Toker (2025)

Cognitive offloading, in one picture

Kosmyna et al. (2025), MIT Media Lab

Confident answers ≠ correct data work.

The studies say the same thing in different words:

• practice can go up while unaided performance falls
• polished output can hide missing knowledge
• who does the thinking matters
• quiz scores that look strong right after the lab can look ordinary three weeks later

For ME204: attempt → then AI to check or debug → keep only what you can justify.

Using AI in This Course

You may use AI for anything in your coursework. We mark for process over output. If you cannot explain a decision your submitted work makes, slow down.

Signs you have not engaged much

• A library we did not teach, without explaining why it fits the problem better
• A long, convoluted solution to a problem we solved in one line
• Long, repetitive explanations or text in the generic voice AI defaults to (I call these “AI writing sins”)
• We cannot tell from reading whether something came from you

What good work looks like

• Code that solves the problem and nothing else
• It is evident from reading your work that you could walk us through why you made each choice
• Your explanations are in your own words, not in “AI voice”
• If you used AI, it was to check or speed up work you had already started

5️⃣ This Afternoon

12:40 – 12:50

This afternoon’s lab

You will work with just the Terminal and tomorrow we will discuss your experience.

You will learn how to use your keyboard to do things you’d normally do with a mouse:

• pwd (see where you are in your computer),
• ls (view the list of files),
• cd (change directory), and
• cat (check the content of a text file).

By 💻 Week 01 Day 03 Lab you will use the terminal to fetch the same weather data you just asked an AI to find. By 💻 Week 01 Day 04 Lab you will have it in a table.

This is what it looks like

Whether or not you have seen a terminal before, you start from the same place this afternoon: four commands and a folder tree.

Let’s get set up on Nuvolos

We have a cloud environment called Nuvolos that has been set up for you to use during these 3 weeks of ME204.

The step-by-step guide is on Moodle: Guides → 1 Nuvolos First Time.

Thanks!

12:50

This afternoon: your first terminal session with Jonas.

💬 Keep notes about anything you may not have understood so we can sort out your questions tomorrow morning!

References

These slides were built with Quarto.

• Shah, C. (2020). A hands-on introduction to data science. Cambridge University Press.
• Schutt, R., & O’Neil, C. (2013). Doing data science (1st edition). O’Reilly Media.
• Heatwave data: Open-Meteo Historical Weather API (ERA5 reanalysis), 1986 to 10 July 2026.
• Sweigart, A. (2019). Automate the Boring Stuff with Python (2nd edition). No Starch Press.
• Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning. Proceedings of the National Academy of Sciences, 122(26).
• Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness. British Journal of Educational Technology, 56(2), 489–530.
• Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your Brain on ChatGPT. arXiv:2506.08872.
• Akgün, M., & Toker, S. (2025). Short-Term Gains, Long-Term Gaps: The Impact of GenAI and Search Technologies on Retention. arXiv:2507.07357.

LSE Summer School 2026 | ME204 Week 01 Day 01