🗓️ Week 01
Welcome to the course
LSE DS202 – Data Science for Social Scientists
24 Jan 2025
Your lecturer
Assist. Prof. (Education)
LSE Data Science Institute
lecturer
course convenor
- PhD in Computer Science (University of Twente, Netherlands)
- Background: Engineering, Databases, Health Informatics, ML for cybersecurity
- Formerly Research Associate at King’s College London and the University of Edinburgh (School of Informatics)
decision support systems
machine learning applications
databases
provenance
ethical AI/XAI
Teaching Assistants
Data Scientist
The Economist
MSc in Applied Social Data Science (LSE)
guest teacher
Data Scientist
Microsoft
MSc in Data Science (LSE)
guest teacher
Guest Lecturer
Data Science Institute
DPhil in Politics (Oxford University)
guest teacher
Data Scientist
MSc in Statistical Science (Oxford University)
guest teacher
Support Sessions
Research Officer
LSE Data Science Institute and LSE Cities
- 🦸🏻♀️ Runs weekly drop-in sessions for all DSI courses!
DS202W Weekly Drop-in sessions:
- Typically every Wednesday from 10.00 to 11.30am at the COL.1.06 (Visualisation studio) but check announcements and calendar invites for updates.
Administrative Support
Teaching and Learning Administrator
LSE Data Science Institute
Write an e-mail to Kevin:
- if you cannot find the lecture recording on Moodle
- when you need an extension for an assignment
(👉 check LSE’s extension policy) - to request a class group change
(you will be asked to provide a reason for this) - to inform us of any other issues that may affect your studies
The Data Science Institute
- This course is offered by the LSE Data Science Institute (DSI).
- DSI is the hub for LSE’s interdisciplinary collaboration in data science
- ⏭️ Let’s see a few activities that might be of interest to you
CIVICA Seminar Series
Industry “field trips”
Who are you? (cont.)
![]()
Course Brief
Course BriefWhat is this course about?
Focus: learn and understand the most fundamental machine learning algorithms
How: practical use of machine learning techniques and its metrics, applied to relevant data sets
![]()
Course Brief
What is this course about?
- Focus: learn and understand the most fundamental machine learning algorithms
- No neural networks, no deep learning, no large-scale data
- How: practical use of machine learning techniques and its metrics, applied to relevant data sets
- Some but not a lot of theory, math proofs and derivations
- Lots of coding, examples and exercises
More about 🧑🏻💻 Labs (90 min each week)
Each week, you will have a roadmap of what to do.
The roadmap will typically contain the following elements:
| Type of activity | Description |
|---|---|
| 🧑🏻🏫 TEACHING MOMENT | Your class teacher deserves your full attention |
| 🎯 ACTION POINTS | Time to follow the steps in the roadmap. Try it for a bit, but if you get stuck, call your class teacher. |
| 👥 IN PAIRS/GROUPS | You will benefit from completing that task with your peers more than doing it alone |
| 🗣️ CLASSROOM DISCUSSION | Your class teacher will facilitate a discussion about the task |
| 📝 SUBMISSION | Submit your work |
👉 Now, let’s navigate our 
Moodle page to see the 📓 Syllabus and to talk about ✍️ Assessments & Feedback.
Programming
- Programming Language:
- Integrated Development Environment (IDE) option:
- In this course, we use:
- VS Code is a more general IDE, good for many programming languages. The flip-side is that it requires a bit more configuration.
Teaching Philosophy
- My teaching approach is grounded in empiricism.
- I see learning as a transformative process, something that conduces to change, which is best facilitated by active, experience-focused, and exploration-driven activities.1
- In summary: learning by doing (or said, more bluntly😂, learning by trial and error) serves as the cornerstone of this course.
What does that mean in practice?
Image created with DALL·E via Bing Chat AI bot. Prompt: “An illustration of a person trying to solve a puzzle with pieces that have different symbols and formulas on them. The person is looking at a screen that shows the 📋 Getting Ready guide and has a smile on their face.”
- Frequently, we will present you with tasks that involve new concepts before diving into the corresponding theory or background knowledge.
- For example, I might ask you to consult the
pandasorscikit-learnornumpyor Python documentation instead of explaining it directly.
- For example, I might ask you to consult the
- Reasoning: letting your ‘struggles’ guide the learning process.
- 👉 allow yourself to make silly mistakes and to ask ‘dumb questions’.
- You are very much encouraged to help and learn from each other.
- If this course is too easy for you, try to apply its concepts to your own data sets or to more complex problems and bring us your questions.
- If you feel this teaching style is not working, drop us an e-mail or discuss it during office hours (see 📟 Communication)
AI tools in this course
Do you use ChatGPT, GitHub Copilot, or other AI tools?
What do you think of generative AI tools?
The ![]()
GENIAL project
GENIAL project - We see many students using ChatGPT during lectures, labs, and assessments.
- Frankly, most university instructors are clueless as to whether this is helping or hindering your learning.
- So we did some research to try to figure out:
- How are students using generative AI tools in their studies?
- What are the benefits and drawbacks of using generative AI tools?
Participating Courses:
- DS105W (Data for Data Science)
- DS202W (Data Science for Social Scientists)
- ST456 (Deep Learning)
- PP422 (Data Science for Public Policy)
The ![]()
GENIAL project
You can read more about the GENIAL project on the project page.
What we have learned so far:
We haven’t fully analysed the data yet (lots of it!⛰️) but here’s what we can say for now about the good and bad aspects of using generative AI tools in education:
- Good: The students who made the most resourceful use of GenAI remained in control of their learning. They often gave the chatbots a lot of context (“I want to perform web scraping of this website with the library
scrapy, the code must contain functions – no classes – and I want to save the data in a CSV file.”) and would always check the code/output generated by GenAI against the course materials or reputable sources. They were able to identify when the AI was suggesting something that was not correct or not following best practices and would never blindly accept the AI’s suggestions.
The ![]()
GENIAL project
What we have learned so far:
We haven’t fully analysed the data yet (lots of it!⛰️) but here’s what we can say for now about the good and bad aspects of using generative AI tools in education:
- Bad: If you don’t master a subject, GenAI can make you feel like you do. This pattern was frequent, for example, among students who had gaps in their understanding of programming concepts. They would ask the AI to generate code for them, and the AI would produce code that seemed to work but that generated the incorrect response or was so complex, it was virtually impossible to edit.
Read more about it in our preprint:
Dorottya Sallai, Jonathan Cardoso-Silva, Marcos E. Barreto, Francesca Panero,Ghita Berrada, and Sara Luxmoore. “Approach Generative AI Tools Proactively or Risk Bypassing the Learning Process in Higher Education”, LSE Public Policy Review, 3(3), p. 7, 2024.
☕️ Time for a break
Our first proper lecture will start in a few minutes.
“What really is data science? + Python tips”
The academic possibilities
- Humans and machines nowadays generate A LOT of data ALL THE TIME
- It has become very cheap to collect and store this data
- This abundance of data opens up new possibilities for research & policy-making
New data to answer old questions:
- How do rumours spread?
- How can we predict unemployment rates accurately?
New questions enabled by new data/new technologies:
- Is social media a threat to democracy/public order?
- Is generative AI a threat to the job market?
We hope that in this reformulated version of the DS202 course, you will learn how to tackle similar questions that are relevant to your field of study.
Python vs R
- Python is a general-purpose programming language
- It is used for web development, scientific computing, data science, advanced machine learning tools (deep learning), etc.
- R is more niche. It is a programming language created for statistical computing
- You can do many other things with R, but it is mostly used for statistics and general data science (except for heavy Machine Learning)
Some Python basics
Data types
- In R, you assign a variable using the operator
<-:
var <- 2- Some basic data types:
var <- "value" # A string. Single quotes are OK too
var <- 2.2 # A double (aka numeric)
var <- 2 # Also a double! 😱
# Want an integer? You have to be explicit:
var <- as.integer(2)
- Whereas in Python, assignments are done with
=:
var = 2- The python equivalent:
var = "value" # A string. Single quotes are OK too
var = """I want to write
sentence without caring for line breaks"""
# Python also has an additional option (triple double quotes!!!) to simplify the handling of strings that contain newlines
var = 2.2 # A float
var = 2 # An int (🏅)
var = float(2) # A floatIn Python, less is more! Always, be explicit when using the greedier data types…
Python basics
Python lists
- We can put basic data types (i.e strings, integers, floats) in collections of data (e.g lists, dictionaries, tuples)
# I am creating a list of integers
l = [1, 2, 3, 4]
lreturns:
[1, 2, 3, 4]You could use the append method to add elements to a list:
l.append(5) # You're adding an element to the list l in-place
lreturns:
[1, 2, 3, 4, 5]You could also use the extend method to add several elements in one go:
l.extend([6,7,8]) # You're adding elements to the list l in-place
lreturns:
[1, 2, 3, 4, 5, 6, 7, 8]Yet another way to elements to a list is as follows:
l2=[5,7,9] #defining a list as usual
l2+=[8,0,4] #adding elements to the list
l2returns:
[5, 7, 9, 8, 0, 4]Python basics
Python lists (cont.)
- Lists don’t need to contain a single data type
mixed_type=[2.0,7,"bananas"] # this line is entirely valid!- You can define lists of lists and lists of lists of lists, etc…
nested_list=[[1,2,3],[4,5,6]] #An example of nested list- You can access the elements of lists and nested lists as follows:
l=[1, 2, 3, 4, 5, 6, 7, 8]
l[0]returns:
1nested_list=[[1, 2, 3], [4, 5, 6]]
nested_list[0][-1]
nested_list[1][1]returns:
3
5Python basics
Other types of data collections
- Aside from lists, you also have tuples
my_tuple=(1,2,3,4)
my_tuplereturns
(1, 2, 3)my_tuple[1]returns
2What do you think is the difference here?
Tuples are immutable!
Is there a way to update tuples? Yes!
First method
my_tuple+=(9,) #one way of adding an element to the tupleSecond method
#another way way of adding an element to the tuple
temp=list(my_tuple) #convert the tuple to list
temp+=[9] #append element to list
my_tuple=tuple(temp) #convert the list back to tuplePython basics
Other types of data collections
Aside from lists and tuples, you also have dictionaries and other more complex data collection types (for these, see the documentation).
A Python dictionary is a collection of key-value pairs, where each key corresponds to its associated value. For example:
# Me trying to do something complicated
my_silly_dictionary={"first_name":"Jane","last_name":"Doe","city":"London"}
my_silly_dictionaryreturns:
{'first_name': 'Jane', 'last_name': 'Doe', 'city': 'London'}You can access dictionary elements as follows:
my_silly_dictionary["first_name"]returns
'Jane'You can add an element to the dictionary as follows:
my_silly_dictionary["country"]="UK"returns
{'first_name': 'Jane', 'last_name': 'Doe', 'city': 'London', 'country': 'UK'}Python basics
Some basic operations
- If you run:
type(my_silly_dictionary)
type(var)you will get the type of your Python object.
The above returns:
<class 'dict'>
<class 'float'> #since var=float(2.0) You can get the length of a collection of data (list,dictionary,tuple) with the len function.
len(l) #l=[1, 2, 3, 4, 5, 6, 7, 8]
len(my_tuple) #my_tuple=(1,2,3)
len(my_silly_dictionary) #my_silly_dictionary={'first_name': 'Jane', 'last_name': 'Doe', 'city': 'London'}returns
3
8
3Python basics
Something, we need to perform operations repeatedly
We have loops (for or while loops):
result = []
for i in range(100000):
result.append(i * 2)result = []
i = 0
while i < 100000:
result.append(i * 2)
i += 1(Note that Python needs indentation and you absolutely can’t mix tabs and spaces!)
And you have list comprehensions (as well as dictionary comprehensions)
result = [i * 2 for i in range(100000)]# dictionary that associates a number with its square
squares = {x: x**2 for x in range(10)}
print(squares)returns
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}# code that produces a dictionary that only contains the pair numbers from the original dictionary
original_dict = {"a": 1, "b": 2, "c": 3, "d": 4}
filtered_dict = {k: v for k, v in original_dict.items() if v % 2 == 0}
print(filtered_dict){'b': 2, 'd': 4}Python basics
Custom functions definition
def my_function(x):
return x + 1
my_function(2)In R, the return keyword exists, but it is optional. Whatever is at the last line of the function will be returned.
my_function <- function(x) {
x + 1
}Python basics
Custom functions definition
Let’s define functions based on the loops and list comprehension from before. We’ll do some code profiling!
import cProfile
def for_loop_example():
result = []
for i in range(100000):
result.append(i * 2)
def while_loop_example():
result = []
i = 0
while i < 100000:
result.append(i * 2)
i += 1
def list_comprehension_example():
result = [i * 2 for i in range(100000)]
# Profile each function
print("Profiling for loop:")
cProfile.run("for_loop_example()")
print("\nProfiling while loop:")
cProfile.run("while_loop_example()")
print("\nProfiling list comprehension:")
cProfile.run("list_comprehension_example()")pandas and scikit-learn (briefly)
- Python has a base set of functions and libraries that come with the installation of the language (e.g
os,collections,math, etc. - see the Python documentation for details)
The
pandas,numpyandscikit-learnlibraries are not part of the standard Python libraries, but they are very popular and very actively maintained packages.These packages contain most of the functionality needed to handle datasets, manipulate them (
pandasmainly), perform statistical operations on them and apply machine learning models on themThese are the libraries we will rely on most in this course.
Note to R users
Think of the pandas as what tidyverse is to R and to some extent of scikit-learn, as what tidymodels (and perhaps caret) are to R.
A touch of pandas
Example: reading a csv file
import pandas as pd
my_data=pd.read_csv("my_file.csv")Example: selecting columns
# accessing a column in the dataframe
my_data['col']
my_data.col #note that for this to work column "col" should not have any spaces in its name
my_data[[col1,col2]]A touch of pandas
Example: filtering rows
Filtering when the values are integers
df.query("col==2") #case where the column type is an integerFiltering when the values are strings
df.query("col=='python'") #case where the column type is a stringExample: concatenating dataframes
Say we have two random datasets:
df1 = pd.DataFrame({
"Name": ["Alice", "Bob"],
"Age": [25, 30]
})
df2 = pd.DataFrame({
"Name": ["Charlie", "David"],
"Age": [35, 40]
})If we want to concatenate both dataframes vertically (i.e name and age stay the columns) then:
# Concatenate the DataFrames
result = pd.concat([df1, df2], ignore_index=True)
print(result)which returns