🗓️ Week 01
Welcome to the course
LSE DS202 – Data Science for Social Scientists
04 Oct 2024
Your lecturer
Assist. Prof. (Education)
LSE Data Science Institute
lecturer
course convenor
- PhD in Computer Science (University of Twente, Netherlands)
- Background: Engineering, Databases, Health Informatics, ML for cybersecurity
- Formerly Research Associate at King’s College London and the University of Edinburgh (School of Informatics)
decision support systems
machine learning applications
databases
provenance
ethical AI/XAI
Teaching Assistants
Data Scientist
The Economist
MSc in Applied Social Data Science (LSE)
guest teacher
Data Scientist
Microsoft
MSc in Data Science (LSE)
guest teacher
Guest Lecturer
Data Science Institute
DPhil in Politics (Oxford University)
guest teacher
Data Scientist
MSc in Statistical Science (Oxford University)
guest teacher
Support Sessions
Research Officer
LSE Data Science Institute and LSE Cities
- 🦸🏻♀️ Runs weekly drop-in sessions for all DSI courses!
DS202A Weekly Drop-in sessions:
- Typically every Tuesday from 12.15 to 13.30 at the COL.1.06 (Visualisation studio) but check announcements and calendar invites for updates.
Administrative Support
Teaching and Learning Administrator
LSE Data Science Institute
Write an e-mail to Kevin:
- if you cannot find the lecture recording on Moodle
- when you need an extension for an assignment
(👉 check LSE’s extension policy) - to request a class group change
(you will be asked to provide a reason for this) - to inform us of any other issues that may affect your studies
The Data Science Institute
- This course is offered by the LSE Data Science Institute (DSI).
- DSI is the hub for LSE’s interdisciplinary collaboration in data science
- ⏭️ Let’s see a few activities that might be of interest to you
CIVICA Seminar Series
Industry “field trips”
Who are you? (cont.)
![]()
Course Brief
Course BriefWhat is this course about?
Focus: learn and understand the most fundamental machine learning algorithms
How: practical use of machine learning techniques and its metrics, applied to relevant data sets
![]()
Course Brief
What is this course about?
- Focus: learn and understand the most fundamental machine learning algorithms
- No neural networks, no deep learning, no large-scale data
- How: practical use of machine learning techniques and its metrics, applied to relevant data sets
- Some but not a lot of theory, math proofs and derivations
- Lots of coding, examples and exercises
More about 🧑🏻💻 Labs (90 min each week)
Each week, you will have a roadmap of what to do.
The roadmap will typically contain the following elements:
| Type of activity | Description |
|---|---|
| 🧑🏻🏫 TEACHING MOMENT | Your class teacher deserves your full attention |
| 🎯 ACTION POINTS | Time to follow the steps in the roadmap. Try it for a bit, but if you get stuck, call your class teacher. |
| 👥 IN PAIRS/GROUPS | You will benefit from completing that task with your peers more than doing it alone |
| 🗣️ CLASSROOM DISCUSSION | Your class teacher will facilitate a discussion about the task |
| 📝 SUBMISSION | Submit your work |
👉 Now, let’s navigate our 
Moodle page to see the 📓 Syllabus and to talk about ✍️ Assessments & Feedback.
Programming
- Programming Language:
- Integrated Development Environment (IDE) options:
- You choose:
- RStudio is the most popular IDE for R
- VS Code is a more general IDE, good for many programming languages. It is more lightweight than RStudio, but it requires more configuration.
Software and Tools
Teaching Philosophy
- My teaching approach is grounded in empiricism.
- I see learning as a transformative process, something that conduces to change, which is best facilitated by active, experience-focused, and exploration-driven activities.1
- In summary: learning by doing (or said, more bluntly😂, learning by trial and error) serves as the cornerstone of this course.
What does that mean in practice?
Image created with DALL·E via Bing Chat AI bot. Prompt: “An illustration of a person trying to solve a puzzle with pieces that have different symbols and formulas on them. The person is looking at a screen that shows the 📋 Getting Ready guide and has a smile on their face.”
- Frequently, we will present you with tasks that involve new concepts before diving into the corresponding theory or background knowledge.
- For example, I might ask you to consult the
tidyversedocumentation instead of explaining it directly.
- For example, I might ask you to consult the
- Reasoning: letting your ‘struggles’ guide the learning process.
- 👉 allow yourself to make silly mistakes and to ask ‘dumb questions’.
- You are very much encouraged to help and learn from each other.
- If this course is too easy for you, try to apply its concepts to your own data sets or to more complex problems and bring us your questions.
- If you feel this teaching style is not working, drop us an e-mail or discuss it during office hours (see 📟 Communication)
AI tools in this course
Do you use ChatGPT, GitHub Copilot, or other AI tools?
What do you think of generative AI tools?
The ![]()
GENIAL project
GENIAL project - We see many students using ChatGPT during lectures, labs, and assessments.
- Frankly, most university instructors are clueless as to whether this is helping or hindering your learning.
- So we did some research to try to figure out:
- How are students using generative AI tools in their studies?
- What are the benefits and drawbacks of using generative AI tools?
Participating Courses:
- DS105W (Data for Data Science)
- DS202W (Data Science for Social Scientists)
- ST456 (Deep Learning)
- PP422 (Data Science for Public Policy)
The ![]()
GENIAL project
You can read more about the GENIAL project on the project page.
What we have learned so far:
We haven’t fully analysed the data yet (lots of it!⛰️) but here’s what we can say for now about the good and bad aspects of using generative AI tools in education:
- Good: The students who made the most resourceful use of GenAI remained in control of their learning. They often gave the chatbots a lot of context (“I want to perform web scraping of this website with the library
scrapy, the code must contain functions – no classes – and I want to save the data in a CSV file.”) and would always check the code/output generated by GenAI against the course materials or reputable sources. They were able to identify when the AI was suggesting something that was not correct or not following best practices and would never blindly accept the AI’s suggestions.
The ![]()
GENIAL project
What we have learned so far:
We haven’t fully analysed the data yet (lots of it!⛰️) but here’s what we can say for now about the good and bad aspects of using generative AI tools in education:
- Bad: If you don’t master a subject, GenAI can make you feel like you do. This pattern was frequent, for example, among students who had gaps in their understanding of programming concepts. They would ask the AI to generate code for them, and the AI would produce code that seemed to work but that generated the incorrect response or was so complex, it was virtually impossible to edit.
Read more about it in our preprint:
Dorottya Sallai, Jonathan Cardoso-Silva, Marcos E. Barreto, Francesca Panero,Ghita Berrada, and Sara Luxmoore. “Approach Generative AI Tools Proactively or Risk Bypassing the Learning Process in Higher Education”, Preprint, July 2024.
☕️ Time for a break
Our first proper lecture will start in a few minutes.
“What really is data science? + R tips”
The academic possibilities
- Humans and machines nowadays generate A LOT of data ALL THE TIME
- It has become very cheap to collect and store this data
- This abundance of data opens up new possibilities for research & policy-making
New data to answer old questions:
- How do rumours spread?
- How can we predict unemployment rates accurately?
New questions enabled by new data/new technologies:
- Is social media a threat to democracy/public order?
- Is generative AI a threat to the job market?
We hope that in this reformulated version of the DS202 course, you will learn how to tackle similar questions that are relevant to your field of study.
Python vs R
- Python is a general-purpose programming language
- It is used for web development, scientific computing, data science, advanced machine learning tools (deep learning), etc.
- R is more niche. It is a programming language created for statistical computing
- You can do many other things with R, but it is mostly used for statistics and general data science (except for heavy Machine Learning)
R for Python users
Data types
- In R, you assign a variable using the operator
<-:
var <- 2- Some basic data types:
var <- "value" # A string. Single quotes are OK too
var <- 2.2 # A double (aka numeric)
var <- 2 # Also a double! 😱
# Want an integer? You have to be explicit:
var <- as.integer(2)
- Whereas in Python, assignments are done with
=:
var = 2- The python equivalent:
var = "value" # A string. Single quotes are OK too
var = 2.2 # A float
var = 2 # An int (🏅)R for Python users
R Vectors
- R is a vectorized language. Everything is a vector!
- Amongst other things, this means we can call
length()on any variable:
In the example below, var is a vector of length 1.
# This is a vector!
var <- 2.2
length(var)returns:
[1] 1Use the c( ) function to concatenate vectors:
# A vector of length 3
var <- c(2.2, 3.3, 4.4)
length(var)returns:
[1] 3R for Python users
R Vectors (cont.)
- ⚠️ R vectors can only have one data type!
This is straightforward:
# A vector of numbers
c(1, 2, 3, 4, 5)
# A vector of characters
c("a", "b", "c", "d", "e")
# A vector of booleans
c(TRUE, FALSE, TRUE, FALSE, TRUE)But beware! The code below is also valid:
my_vec <- c(2, "3", as.integer(4))It won’t throw an error, but once you inspect the type of the vector, you will see that typeof(my_vec) is a "character".
If you type:
my_vecYou will see:
[1] "2" "3" "4"R for Python users
Lists (not the same as vectors)
- If you need to keep elements of different data types, create a list instead.
my_list <- list(2, "3", as.integer(4))Now, if you type:
my_listYou will see:
[[1]]
[1] 2
[[2]]
[1] "3"
[[3]]
[1] 4- We see a list with a length of 3
- Each element of the list is shown after the double brackets,
[[ ]] - The first element of the list (
[[1]]) is a vector of size 1 ([1]) that contains the number2, etc.
R for Python users
Lists are more flexible than vectors (they are also slower to process)
Vectors are always flat:
# Me trying to do something complicated
my_silly_vector <- c(1, c(2, c(3, 4)), 5)
my_silly_vectoryields a simple vector:
[1] 1 2 3 4 5Lists preserve the structure:
# Let's try converting some vectors into a list
my_silly_list <- list(1, list(2, c(3, 4)), 5)
my_silly_listThis produces a list of length 3 (not 5) with a more complex, nested structure:
[[1]]
[1] 1
[[2]]
[[2]][[1]]
[1] 2
[[2]][[2]]
[1] 3 4
[[3]]
[1] 5R for Python users
Obs: Python does not have vectors, only lists
- If you run:
elements = [2, "3", 4]- You will get a list of length 3, with elements of different data types:
type(elements)listlen(elements)3elements[2, '3', 4](preserved structure)
R for Python users
Loops are not that different
for (i in 1:10) {
print(i)
}while (i < 10) {
print(i)
i <- i + 1
}(R needs the curly brackets)
for i in range(1, 11):
print(i)
while i < 10:
print(i)
i += 1
(Python needs the indentation)
R for Python users
Custom functions definition, compared
my_function <- function(x) {
return(x + 1)
}my_function(2)In R, the return keyword exists, but it is optional. Whatever is at the last line of the function will be returned.
my_function <- function(x) {
x + 1
}
def my_function(x):
return x + 1
my_function(2)Base R vs tidyverse
R has a base set of functions that come with the installation of the language
The base functions are OK - they are just not awesome.
The
tidyverseis not part of the base R installation, but it is a very popular packageIt is actually a collection of several packages that make it easier to manipulate data (+ databases + plotting + modelling + etc.)
This is what we will use in this course. (We suffered tremendously teaching base R two years ago)
Note to Python users
Think of the tidyverse as what pandas is to Python
Base R vs tidyverse
Example: reading a csv file
# Base R
my_data <- read.csv("my_file.csv")
# tidyverse
my_data <- read_csv("my_file.csv")Base R vs tidyverse
Example: selecting columns
# Base R
my_data <- my_data[, c("col1", "col2")]
# tidyverse
my_data <- select(my_data, col1, col2)Base R vs tidyverse
Example: filtering rows
# Base R
my_data <- my_data[my_data$col1 == 1, ]
# tidyverse
my_data <- my_data %>% filter(col1 == 1)Example: combing columns together
# Base R
my_data$col3 <- my_data$col1 + my_data$col2
# tidyverse
my_data <- my_data %>% mutate(col3 = col1 + col2)Base R vs tidyverse
Example: grouping and summarizing
Say we have a random dataset:
# Generate a random my_data
my_data <- data.frame(col1 = sample(1:3, 100, replace = TRUE), col2 = rnorm(100))If we want to calculate the mean of col2 for each value of col1:
# The Base R way
my_data <- aggregate(my_data, by = list(my_data$col1), FUN = mean)
# Overtime, you will see that the tidyverse way becomes more intuitive
my_data <- my_data %>% group_by(col1) %>% summarize(mean(col2))