---
title: "🗓️ Week 10 - The Basics of Text Mining"
subtitle: "Theme: Applications"
author: "Dr. Ghita Berrada"
format: 
  html:
    from: markdown+emoji
    page-layout: full
    toc: true
    toc-depth: 2
self-contained: true
---

Welcome to Week 10!

```{r, echo=FALSE, message=FALSE, warning=FALSE}
library(tidyverse)
library(tidymodels)
library(quanteda)
library(quanteda.textplots)

```

# Part 1: Representing text as structured data 

The code chunks below will be explained during the lecture.

```{r}
df_speeches <- read_csv("w10-data/speeches.csv")
df_mps <- read_csv("w10-data/mps.csv")
```


Let's explore the data together a bit:

```{r}
# Live demo
```

## 1.1 The notion of a **corpus**:

```{r}
corp_speeches <- quanteda::corpus(df_speeches %>% select(-speech_html), text_field="speech_raw_text")
quanteda::docnames(corp_speeches) <- df_speeches$`speech_id`
```


The notion of a **token**:

```{r}
# Live demo
```

```{r}
plot_df <- summary(corp_speeches)

max_x_break <- round(max(plot_df$Tokens), -3)

g <- (
  ggplot(plot_df, aes(x=Tokens))
  + geom_histogram(binwidth=100, fill="#C63C4A")
  + theme_minimal()
  
  + scale_x_continuous(breaks = seq(0, max_x_break, max_x_break/10))
  
  + labs(x="Number of Tokens",
         y="Count",
         title="Typically, how many tokens are there in those speeches?",
         subtitle = "Corpus: speeches at the House of Commons in November 2023")
  
  + # Prettify plot a bit
  theme_bw() +
  theme(plot.title=element_text(size=rel(1.5)),
        plot.subtitle = element_text(size=rel(1)),
        axis.title=element_text(size=rel(1.3)),
        axis.title.x=element_text(margin=margin(t=10)),
        axis.title.y=element_text(margin=margin(r=10)),
        axis.text=element_text(size=rel(1.2)))
)

g
```
## 1.2 More on tokens

```{r}
tokens_speeches <- quanteda::tokens(corp_speeches)
```

Exploring:

```{r}
# Live demo of types
```

**The notion of n-grams**

```{r}
tokens_ngrams(tokens_speeches)[[1]]
```


## 1.3 The Document-Term Frequency Matrix

```{r}
dfm_speeches <- quanteda::dfm(tokens_speeches)
dfm_speeches
```

```{r}
# To bring it to a familiar object
dfm_as_data_frame <- quanteda::convert(dfm_speeches, to="data.frame")
dim(dfm_as_data_frame)
```
**Things you can do with a dfm:**

```{r}
dfm_speeches %>% quanteda::topfeatures()

#docfreq(dfm_speeches)
```

🗣️ **ARE THESE USEFUL?**

What would you do with these features?

```{r}
quanteda.textplots::textplot_wordcloud(dfm_speeches)
```
## 1.4 TF-IDF

There will be an explanation about a technique called TF-IDF, then we'll discuss what the following resulting data frame represents:

```{r}
dfm_tfidf(dfm_speeches, scheme_tf = "prop")
```


# Part 2: Feature Engineering for Machine Learning

## 2.1 Rebuilding

Let's re-build the corpus more carefully:

```{r}
# Get rid of punctuation or stopwords

tokens_speeches <- 
  # Get rid of punctuations
  quanteda::tokens(corp_speeches, remove_punct = TRUE) %>% 
  # Get rid of stopwords
  quanteda::tokens_remove(pattern = quanteda::stopwords("en"))

tokens_speeches <- tokens_ngrams(tokens_speeches, n=1:2)

dfm_speeches <- quanteda::dfm(tokens_speeches)
quanteda.textplots::textplot_wordcloud(dfm_speeches)
```


## 2.2 PCA

```{r}
df_word_frequency <- convert(dfm_speeches, to="data.frame")


pca_recipe <-
  recipe(~ ., data = df_word_frequency) %>%
  update_role(doc_id, new_role="id") %>% 
  step_normalize(all_predictors()) %>% 
  step_pca(all_predictors())

pca_prep <- prep(pca_recipe)

# Add PC1 & PC2
df_pca <- bake(pca_prep, df_word_frequency)
```



```{r out.width=7, out.width=4, echo=FALSE}
ggplot(df_pca, aes(PC01, PC02)) +
  geom_point(alpha=0.2, size=rel(2.5)) +
  labs(title = "There is a huge blob in the middle! I wonder what that represents.", 
       subtitle = "There are some clear outliers here",
       caption = "Figure 2. Distribution of samples using\nPC01 and PC02 as coordinates.") +
  
  #scale_y_continuous(breaks=seq(-7.5, 7.5, 2.5), limits=c(-7.5, 7.5)) +
  
    # Prettify plot a bit
  theme_bw() +
  theme(plot.title=element_text(size=rel(1.2)),
        plot.subtitle = element_text(size=rel(1)),
        axis.title=element_text(size=rel(1.3)),
        axis.title.x=element_text(margin=margin(t=10)),
        axis.title.y=element_text(margin=margin(r=10)),
        axis.text=element_text(size=rel(1.2)))
```

## 2.3 Clustering

Let's build it together! K-means? DBSCAN?

```{r}

```

## 2.4 Explaining clusters with textplot_keyness!

Using `textstat_keyness()`, you can compare frequencies of words between target and reference documents. Here we want to compare what makes Conservative party member party speeches (our "target") different from other party member (essentially Labour) speeches...

```{r}

```

## 2.5 Topic Modelling

There are other algorithms (e.g Latent Dirichlet Allocation) you can use!
Find out more about this in [here](https://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/).


