💻 Lab Week 03, Day 04

🦸🏻‍♂️ Super Tech Support

🥅 Learning Objectives

By the end of this lab, you should be able to: i) Complete your final project with individual support and guidance. ii) Get your project ready for submission with proper documentation. iii) Resolve any remaining technical issues before submission. iv) Ensure your project meets all submission requirements.

This is your final opportunity to work on your 📦 Final Project with direct support from the teaching team. Bring your specific questions and project materials so we can help you most effectively.

⏰ Thursday, 31 July 2025 | 2:00pm - 5:00pm 📍 Location: CKK.2.06 (see LSE’s 🗺️ campus map)

🤨 How to keep working on data projects after this course?

1. Bookmark the 🌐 ME204 website. You can always revisit the course materials and guides here.

2. Make sure you have a copy of all the files you Nuvolos.

   GitHub is a good place to store your work. All of your GitHub repositories will still be available to you after the end of the course ¹.

   If you are still using the me204-study-notes repository I suggested back in 🖥️ Week 01, Day 03 lecture, you can just move all of your files there.

   Revisit the 5️⃣ Git & GitHub guide if you need a refresher.

3. Install all the necessary software on your own computer.

   Nuvolos allowed us to start working on code from day 1 of our course but if you want to keep doing this type of work afterwards, you will need to have the same software installed on your own computer.

   Take a look at the 6️⃣ Local Environment guide for instructions on how to do this.

Final Project Checklist

To help make this a smooth and effective troubleshooting session, please use this checklist to ensure your project is complete and ready for submission.

If you plan to continue working on it after the session, make sure you have clarified any remaining questions with the teaching team.

✅ Data Collection (NB01)

Pure data collection - no analysis or databases

• Good use of titles/subtitles to structure the notebook
• Clear and concise justification of data source (it’s clear what motivated you to collect this data in the particular way you did)
• API authentication working with proper error handling (if+else statements or try+except blocks)
• Data saved to ../data/raw/ folder in appropriate format (CSV or JSON or TXT files)
• All “magic numbers” are clearly justified (that is, if you collected “top 5” posts, explain why you chose 5 and not, say, 10)
• No analysis, visualisations, or database creation in this notebook (some basic checks on data is fine)

✅ Database Design (NB02)

ETL process - reads raw data, creates database

• Good use of titles/subtitles to structure the notebook
• Starts by reading files from within ../data/raw/
• At least 2 tables with proper primary/foreign keys
• There is an explanation for why you chose to organise the data in the way you did
• Data loaded into SQLite database with appropriate data types
• All filtering/processing decisions justified (e.g., “before saving to database, I removed the column [column name] from [this dataframe] because…”)
• Database contains all the needed data for NB03 analysis

✅ Analysis (NB03)

Reads only from database - no external files

• Good use of titles/subtitles to structure the notebook
• At the top, we see an explanation for what you’re curious to explore in the data (e.g., “This notebook shows the process I took to investigate [this thing] in the data. I was motivated by [this reason]…” or even, say, “I didn’t have a particular research question in mind at first, I was just curious, so I decided to do [this and that] simple explorations to see where that would take me…”)
• Starts by reading data from database only (no attempts to read from ../data/raw/ or any other source)
• Uses pandas functions like merge and groupby instead of loops (that is, either everything is done with SQL and/or pandas functions like merge and groupby are used)
• There’s enough context into what you were trying to observe from a given plot or summary table
• Key insights, the ones that will go to the website, have clear narrative titles and subtitles (e.g., title: “ULEZ has indeed contributed massively to reduce air pollution in London” | subtitle: “The volume of NOx emissions in London has decreased by [X] % since [YYYY] when [this happened]…”)
• All analytical choices justified (e.g., “I used the median instead of the mean because…”)
• There is an overall conclusion (even if the results are not particularly surprising)

✅ Website (docs/index.md)

Public-facing story for general audience

• Good use of titles/subtitles to structure the website
• Compelling narrative that guides readers through findings
• Methodology section with clear justification of choices
• Key visualisations/summary tables embedded with explanations
• Professional presentation with proper markdown formatting
• No technical jargon - accessible to general audience
• Clear connection between data and insights

✅ Documentation & README.md

Professional project setup

• README.md with clear setup instructions and API credentials
• All notebooks well-documented with markdown explanations
• Git repository properly organised with meaningful commits
• Reproducible workflow from start to finish

🎯 Final Check

Submission readiness

• All files committed and pushed to GitHub
• Website accessible via GitHub Pages URL
• No sensitive data (API keys, personal info) in repository
• Code runs without errors in all notebooks (to confirm, run Restart, then Clear All Outputs, and then Run All in each notebook)
• Ready for submission by 1st August 2025 at 6pm UK time
• I rewarded myself with a cookie or a special drink to celebrate my hard work!

---

📦 Final Project

Due Friday, 1 August 2025 at 6pm.

➡️ Project Requirements

Questions?

➡️ Ask on Slack

Footnotes

1. Your midterm and final project repositories are stored in the LSE GitHub organisation and we cannot promise to keep them available forever. It’s a good idea to have a copy of them in your personal GitHub account