LSE ME204 · Data Engineering Principles for the Social Sciences
🖥️ Week 02 Day 01 Lecture
10:00 – 10:10
Ten minutes for midterm questions. Longer support returns at 12:40.
🌍 Cities Fair coastal versus inland comparison
Choose three or four cities and explain what they share
🔑 OpenWeather key Your account, your requests
Sign up, open the API keys page, store the key in .env
📡 Path A or Path B Same research question, same marking rubric
🌙 Nighttime cooling Define the measure, then compute it
Decide which hours count as night for your cities
📓 Notebooks or scripts Collect, transform, analyse
NB01–NB03, or the matching .py scripts route in the brief
🖼️ Insight in the README State the finding up front
A chart, or a styled pandas DataFrame, embedded in the README
10:10 – 11:25
Essential chart types, common mistakes, and a first look at plotly for this course. A chart can be technically correct and still mislead, hide the data, or make the wrong comparison easy. We build first, then look at ways charts go wrong.
Let’s see how I wrote the code for this plot I showed you last week.
Data source: Open-Meteo Historical Weather API (ERA5). Definition: 3+ consecutive days at or above 28°C (Met Office London threshold). Period: 1986 to 10 Jul 2026.
In the course this year we want you to use plotly exclusively for your charts. Each argument names a column in the table.
fig = px.bar(
chart_df,
x="year",
y="heatwave_events",
color="period",
)
fig.show()
First rows of chart_df:
| year | heatwave_events | period |
|---|---|---|
| 1986 | 0 | complete year |
| 2003 | 2 | complete year |
| 2026 | 1 | 2026 (partial, to 10 Jul) |
How the arguments map:
x="year" sets the horizontal axisy="heatwave_events" sets the bar heightcolor="period" sets the bar colourSame fig = px.bar(...) call as before. Then adjust the labels with update_layout, and thin out the year ticks:
Same call again. Name an exact colour for each category so the palette matches the rest of your report:
Browse on the lectern or show screenshots:
Start from the claim, then pick a type. The Data Visualisation Catalogue is a good menu of options.
| Type | Catalogue page | Use when… |
|---|---|---|
| Bar | Bar Chart | Comparing categories or years |
| Line | Line Graph | Change along an ordered time axis |
| Scatter | Scatterplot | Relating two numeric columns |
The same data, two impressions.
On the left, the y-axis starts near the values, so a gap of one percentage point looks huge. On the right, the axis starts at zero, so you see the true scale.
Rule of thumb: bar charts that show counts or proportions should start at zero. If you zoom a line chart, say why in a note on the figure.
One short framing from Friends Don’t Let Friends Make Bad Graphs (Clara Qin Li and contributors).

Source: Friends Don’t Let Friends Make Bad Graphs, rule 1 (Clara Qin Li and contributors)
Rule we care about today: do not use a bar of means when the distribution matters.
Two groups can share the same mean and look identical as bars, while the points tell different stories (spread, outliers, two clusters).
💡 Read rule 1 on the Friends repo before you design a mean comparison.
Sure, this type of chart has a place, and I can see why it gets used in certain situations. But I find them very misleading. It is easy to rush and jump to conclusions that aren’t warranted.
About five minutes. On your phone or laptop, find a chart in the wild that is misleading, confusing, or hard to read (news, reports, papers, social posts).
In your post, answer in one or two sentences:
Post to the Discussion Forum.

11:25 – 11:40
After the break:
px.scatter, px.line, or px.barx and y11:40 – 12:40
Build a DataFrame in the shape Express expects: one column per visual channel, named in the call.
Same plot idea, two ways to call Express. Prefer the right-hand style.
Lists in the call
The numbers appear only in the plot call. The table is optional.
We prefer the declarative form:
x and y.For scatter, line, or bar, the habit is the same: one column for each thing you want on the chart, then name those columns in the call. The tables below are made up, and the titles are just placeholders.
| x_val | y_val | group |
|---|---|---|
| 8.1 | 6.4 | group A |
| 9.4 | 7.9 | group A |
| 11.2 | 9.1 | group B |
| 12.0 | 8.5 | group B |
| month | value | series |
|---|---|---|
| 1 | 12.0 | series A |
| 2 | 13.5 | series A |
| 1 | 9.0 | series B |
| 2 | 8.7 | series B |
| category | count | bucket |
|---|---|---|
| A | 4.2 | type 1 |
| B | 3.1 | type 1 |
| C | 5.0 | type 2 |
| D | 2.4 | type 1 |
Get the DataFrame into the shape the chart expects before you call plotly.express.
Daily rows
| date | tmax_c | heatwave_day |
|---|---|---|
| 2003-08-04 | 30.1 | True |
| 2003-08-05 | 31.0 | True |
| 2003-08-06 | 29.4 | True |
| 2003-08-07 | 24.2 | False |
| … | … | … |
Useful for defining events. Not the table for a yearly bar chart.
One row per bar
| year | heatwave_events | period |
|---|---|---|
| 2003 | 2 | complete year |
| 2022 | 3 | complete year |
| 2026 | 1 | 2026 (partial, to 10 Jul) |
px.bar(chart_df, x="year", y="heatwave_events", color="period")
Start from a daily table. Walk the steps one slide at a time.
daily.head()
| date | tmax_c | heatwave_day |
|---|---|---|
| 2003-08-04 | 30.1 | True |
| 2003-08-05 | 31.0 | True |
| 2003-08-06 | 29.4 | True |
| 2003-08-07 | 24.2 | False |
| … | … | … |
Keep only the rows where heatwave_day is True.
heat_days.head()
| date | tmax_c | heatwave_day |
|---|---|---|
| 2003-08-04 | 30.1 | True |
| 2003-08-05 | 31.0 | True |
| 2003-08-06 | 29.4 | True |
| … | … | … |
The cool day (False) is gone.
Source: When to use .copy()?
Pull the year out of the date string and store it as its own column.
heat_days.head()
| date | tmax_c | heatwave_day | year |
|---|---|---|---|
| 2003-08-04 | 30.1 | True | 2003 |
| 2003-08-05 | 31.0 | True | 2003 |
| 2003-08-06 | 29.4 | True | 2003 |
| … | … | … | … |
groupby collapses many rows into one row per year. Then rename the count column.
chart_df.head()
| year | heatwave_events |
|---|---|
| 2003 | 2 |
| 2022 | 3 |
| 2026 | 1 |
This is already a table px.bar can plot.
.aggSame collapse as the previous slide. .agg names the output column, and the tuple names which column to summarise, so you do not need ["heatwave_day"] first or a separate .rename.
as_index=False keeps year as an ordinary column for px.bar. If you omit it, year becomes the index and you need .reset_index() before you plot.
Need more than one summary later? Add another name in the same .agg(...) call, still with string aggregators such as "mean" or "median".
Optional colour channel: set a default, then overwrite 2026 with .loc.
| year | heatwave_events | period |
|---|---|---|
| 2003 | 2 | complete year |
| 2022 | 3 | complete year |
| 2026 | 1 | 2026 (partial, to 10 Jul) |
Now plot the chart:
plotly.express with a DataFrame, naming columns in the call (not parallel lists in x and y)[…]. I want a plot that […].”[x, y, z]. Help me tweak while aligning to the rules I previously specified.”x and y, reject the snippet and ask again for a DataFrame with column names in the callA title should state the takeaway, not only the variable names.
"Heatwave events by year""London heatwave events rose after 2000 (1986–2026, Met Office threshold)"If your midterm insight is a styled pandas table rather than a chart, treat that table like a figure. Use Styler.set_caption() for a title and subtitle that state the finding before the numbers.
caption = (
"<b>Coastal cities cool faster at night</b><br>"
"<span style='font-size:0.85em;font-weight:400'>"
"Mean overnight drop (°C), four cities"
"</span>"
)
(
summary_df.style
.format({"cooling_rate": "{:.1f}"})
.background_gradient(
subset=["cooling_rate"],
cmap="Blues",
)
.set_caption(caption)
)| city | coastal | cooling_rate |
|---|---|---|
| Brighton | yes | 4.2 |
| Bournemouth | yes | 3.8 |
| Reading | no | 2.1 |
| Oxford | no | 1.9 |
Save a screenshot or export of the styled table into figures/ and embed it in the README the same way you would embed a chart.
12:40 – 13:00
Open work and questions. Use the time for problems that are blocking you, not for new lecture content.
💬 Later questions: Discussion Forum on Moodle. During the lab, ask your class teacher.
Afternoon lab: chart practice and midterm work time.
LSE Summer School 2026 | ME204 Week 02 Day 01
LSE ME204 (2026)