---
title: "🗓️ Week 10 - The Basics of Text Mining"
subtitle: "Theme: Applications"
author: "Dr. Jon Cardoso-Silva"
format: 
  html:
    from: markdown+emoji
    page-layout: full
    toc: true
    toc-depth: 2
self-contained: true
---

Welcome to Week 10!

```{r, echo=FALSE, message=FALSE, warning=FALSE}
library(tidyverse)
library(tidymodels)
library(quanteda)
library(quanteda.textplots)

```

# Part 1: Representing text as structured data 

The code chunks below will be explained during the lecture.

```{r}
df_speeches <- read_csv("speeches.csv")
df_mps <- read_csv("mps.csv")
```


Let's explore the data together a bit:

```{r}
# Live demo
```

## 1.1 The notion of a **corpus**:

```{r}
corp_speeches <- quanteda::corpus(df_speeches %>% select(-speech_html), text_field="speech_raw_text")
quanteda::docnames(corp_speeches) <- df_speeches$`speech_id`
```


The notion of a **token**:

```{r}
# Live demo
```

```{r}
plot_df <- summary(corp_speeches)

max_x_break <- round(max(plot_df$Tokens), -3)

g <- (
  ggplot(plot_df, aes(x=Tokens))
  + geom_histogram(binwidth=100, fill="#C63C4A")
  + theme_minimal()
  
  + scale_x_continuous(breaks = seq(0, max_x_break, max_x_break/10))
  
  + labs(x="Number of Tokens",
         y="Count",
         title="Typically, how many tokens are there in those speeches?",
         subtitle = "Corpus: speeches at the House of Commons in November 2023")
  
  + # Prettify plot a bit
  theme_bw() +
  theme(plot.title=element_text(size=rel(1.5)),
        plot.subtitle = element_text(size=rel(1)),
        axis.title=element_text(size=rel(1.3)),
        axis.title.x=element_text(margin=margin(t=10)),
        axis.title.y=element_text(margin=margin(r=10)),
        axis.text=element_text(size=rel(1.2)))
)

g
```
## 1.2 More on tokens

```{r}
tokens_speeches <- quanteda::tokens(corp_speeches)
```

Exploring:

```{r}
# Live demo of types
```

**The notion of n-grams**

```{r}
tokens_ngrams(tokens_speeches)[[1]]
```


## 1.3 The Document-Term Feature Matrix

```{r}
dfm_speeches <- quanteda::dfm(tokens_speeches)
dfm_speeches
```

```{r}
# To bring it to a familiar object
dfm_as_data_frame <- quanteda::convert(dfm_speeches, to="data.frame")
dim(dfm_as_data_frame)
```
**Things you can do with a dfm:**

```{r}
dfm_speeches %>% quanteda::topfeatures()

#docfreq(dfm_speeches)
```

🗣️ **ARE THESE USEFUL?**

What would you do with these features?

```{r}
quanteda.textplots::textplot_wordcloud(dfm_speeches)
```
## 1.4 TF-IDF

There will be an explanation about a technique called TF-IDF, then we'll discuss what the following resulting data frame represents:

```{r}
dfm_tfidf(dfm_speeches, scheme_tf = "prop")
```


# Part 2: Feature Engineering for Machine Learning

## 2.1 Rebuilding

Let's re-build the corpus more carefully:

```{r}
# Get rid of punctuation or stopwords

tokens_speeches <- 
  # Get rid of punctuations
  quanteda::tokens(corp_speeches, remove_punct = TRUE) %>% 
  # Get rid of stopwords
  quanteda::tokens_remove(pattern = quanteda::stopwords("en"))

tokens_speeches <- tokens_ngrams(tokens_speeches, n=1:2)

dfm_speeches <- quanteda::dfm(tokens_speeches)
quanteda.textplots::textplot_wordcloud(dfm_speeches)
```


## 2.2 PCA

```{r}
df_word_frequency <- convert(dfm_speeches, to="data.frame")


pca_recipe <-
  recipe(~ ., data = df_word_frequency) %>%
  update_role(doc_id, new_role="id") %>% 
  step_normalize(all_predictors()) %>% 
  step_pca(all_predictors())

pca_prep <- prep(pca_recipe)

# Add PC1 & PC2
df_pca <- bake(pca_prep, df_word_frequency)
```



```{r out.width=7, out.width=4, echo=FALSE}
ggplot(df_pca, aes(PC01, PC02)) +
  geom_point(alpha=0.2, size=rel(2.5)) +
  labs(title = "There is a huge blob in the middle! I wonder what that represents.", 
       subtitle = "There are some clear outliers here",
       caption = "Figure 2. Distribution of samples using\nPC01 and PC02 as coordinates.") +
  
  #scale_y_continuous(breaks=seq(-7.5, 7.5, 2.5), limits=c(-7.5, 7.5)) +
  
    # Prettify plot a bit
  theme_bw() +
  theme(plot.title=element_text(size=rel(1.2)),
        plot.subtitle = element_text(size=rel(1)),
        axis.title=element_text(size=rel(1.3)),
        axis.title.x=element_text(margin=margin(t=10)),
        axis.title.y=element_text(margin=margin(r=10)),
        axis.text=element_text(size=rel(1.2)))
```

## 2.3 Clustering

Let's build it together! K-means? DBSCAN?

```{r}

```

## 2.4 Explaining clusters with textplot_keyness!

```{r}

```

## 2.5 Topic Modelling

There are other algorithms you can use!

```{r}

```