---
subtitle: "LSE ME204 Week 02 Day 04 - Lecture Material"
title: "Camden Crime Data: an Exploratory Analysis"
author: "Dr. Jon Cardoso-Silva"
date: "2023-07-20"
date-format: "DD MMMM YYYY"
format: 
    html:
        df-print: kable
        self-contained: true
        link-external-icon: true
        toc: true
        theme: simplex
        code-fold: true
        page-layout: full
        fig-width: 10
        fig-height: 6
        
server: shiny
---

_(Use this markdown file as a template for your project later)_

# ⚙️ Setup

<details><summary>Click here to see how the data looks like</summary>

```{r checkwd, message=FALSE, warning=FALSE, echo=FALSE}
# If you're not in the right directory, change it with setwd()
getwd()
```

```{r setup, message=FALSE, warning=FALSE, echo=FALSE}
library(dplyr)
library(readr)
library(forcats)
library(lubridate)
library(ggplot2)

# New ones today
library(rmarkdown)
library(shiny)

df <- read_csv("../../data/camden_crime_reduced_file.csv")
```

The data we will explore in this page is saved in a data frame `df` of dimensions:

```{r df-dims}
dim(df)
```

This is what a single row of the data looks like:

```{r show-row}
df[1,] %>% t()
```
</details>

# 🚁 Big Picture

The following plots and summary tables try to provide potential readers with an idea of **what is in the data?**

Let's start with a simple summary of the number of records in the data frame per ward:

```{r barplot-count}
#| fig-height: 8

# Define data and mapping
g <- ggplot(data=df, mapping=aes(y=fct_rev(fct_infreq(ward_name))))

# Use a geom
g <- g + geom_bar() 

# Customise scales
g <- g + scale_x_continuous(limits=c(0, 55000), 
                            breaks=seq(0, 60000, by=10000),
                            labels=scales::unit_format(unit = "K", scale = 1e-3))

# Customise titles and labels
g <- g + labs(title = "Crime Records in Camden's 27 Wards: A Closer Look",
              subtitle = "Distribution of Total Crime Records by Ward (2015-2023)",
              x = "Number of Crime Records",
              y = "Ward Name")
  
# Customise the theme
g <- g + 
  theme_minimal() +
  theme(axis.title = element_text(size=rel(1.3)),
        axis.text.y = element_text(size=rel(1.2)),
        axis.text.x = element_text(size=rel(1.4)),
        axis.title.x = element_text(margin=margin(t=15)),
        axis.title.y = element_text(margin=margin(r=15)),
        plot.title = element_text(size=rel(1.6), color="#D9230F"),
        plot.background = element_rect(fill="#FCFCFC", color=NA))

plot(g)
```

Now, let's explore the trend of crime incidents over time. The dataset spans from 2015 to early 2023, with the most recent incident recorded on 1 April 2023. In particular, we will focus on the wards with the highest number of records, as they reveal an intriguing pattern in this analysis:


```{r highlight-holborn}
#| fig.height: 5

plot_df <- 
  df %>% 
  group_by(epoch, ward_name) %>% 
  count() %>% 
  mutate(highlight_ward=if_else(ward_name == 'Holborn and Covent Garden' | ward_name == 'Bloomsbury',
                                ward_name, 'Other'))

# Data and Mapping
g <- ggplot(plot_df, 
            aes(x=epoch, y=n, 
                color=highlight_ward, 
                linewidth=highlight_ward,
                linetype=highlight_ward,
                alpha=highlight_ward,
                group=ward_name))
  
# Geoms
g <- g + geom_line()

# Customise scales
g <- g + 
  scale_x_date(date_breaks = "1 year",
               date_labels = "%Y",
               limits=c(lubridate::ymd("2014-12-20"), NA)) +
  scale_colour_manual(name="Ward", 
                      values=c("Holborn and Covent Garden"="#d95f02",
                               "Bloomsbury"="#7570b3",
                               "Other"="#d7d7d7")) +
  scale_linetype_manual(name="Ward", 
                        values=c("Holborn and Covent Garden"="dashed",
                                 "Bloomsbury"="dotdash",
                                 "Other"="solid")) +
  scale_linewidth_manual(name="Ward",
                         values=c("Holborn and Covent Garden"=0.8,
                                 "Bloomsbury"=0.8,
                                 "Other"=0.5)) +
  scale_alpha_manual(name="Ward",
                     values=c("Holborn and Covent Garden"=1,
                              "Bloomsbury"=1,
                              "Other"=0.8))
  

# Titles
g <- g + labs(title="'Holborn and Covent Garden' and 'Bloomsbury' have always led the records",
              subtitle = "Maybe those are the most populous/touristy areas of Camden?",
              y="Number of Records per Day",
              x="Date")

# Themes
g <- g + 
  theme_minimal() +
  theme(axis.title = element_text(size=rel(1.3)),
        axis.text.y = element_text(size=rel(1.2)),
        axis.text.x = element_text(size=rel(1.4)),
        axis.title.x = element_text(margin=margin(t=15)),
        axis.title.y = element_text(margin=margin(r=15)),
        plot.title = element_text(size=rel(1.6), color="#D9230F"),
        plot.background = element_rect(fill="#FCFCFC", color = NA),
        legend.background = element_rect(fill="#fcfcfc", color="#dedede"),
        legend.position = c(0.21, 0.76))

g
```

Alright, let's shift our focus from the location and explore the types of crimes reported. The table below provides a summary of the frequency of each type of crime reported between 2015 and 1 April 2023.

```{r crime-category-stats}
df %>% 
  group_by(category) %>%
  count() %>%
  arrange(desc(n))
```

**The top 3 represent nearly 54% of the occurrences!**

```{r top3-summary}
# Let me switch to the new fancier pipe style for a change
# and give you an example of a complex pipe that summarises data
#
# I don't even need to store the name of the top 3
df |>
  group_by(category) |>
  count() |>
  arrange(desc(n)) |>
  ungroup() |>
  group_by(type=if_else(row_number() <= 3, "Top 3", "Other Categories")) |>
  summarise(percentage=100*sum(n)/nrow(df)) |>
  arrange(desc(percentage)) |>
  mutate(percentage=sprintf("%.2f%%", percentage))
  
```



# 📈 Exploratory Analysis

After providing an overview of the data to potential readers (total number of records, breakdown per ward, and temporal dimension), let's delve into the analysis by posing more questions _to_ the data.

One initial question that arises is: 

> **What is the proportion of unresolved cases (those without an `outcome_date`)?**


```{r unresolved-cases, warning=FALSE}
#| fig.height: 4

plot_df <- 
  df %>% 
  group_by(epoch_month=lubridate::floor_date(epoch, unit="month"),
           has_outcome=if_else(is.na(outcome_date), "NO", "YES")) %>% 
  count() 

g <- ggplot(plot_df, aes(x=epoch_month, y=n, fill=has_outcome, color=has_outcome))

g <- g + geom_col(alpha=0.9, width=17)

# Colours from https://colorbrewer2.org/#type=diverging&scheme=RdYlBu&n=3
g <- g +
  scale_x_date(date_breaks = "1 year",
               date_labels = "%Y",
               limits=c(lubridate::ymd("2014-12-20"), NA)) +
  scale_y_continuous(limits=c(0, 5000), 
                            breaks=seq(0, 5000, by=1000),
                            labels=scales::unit_format(unit = "K", scale = 1e-3)) +
  scale_fill_manual(name="Case has an outcome?", 
                      values=c("YES"="#dedede",
                               "NO"="#D9230F")) +
  scale_colour_manual(name="Case has an outcome?", 
                      values=c("YES"="#dedede",
                               "NO"="#D9230F")) 


g <- g + labs(title="On average ~20-25% of cases do not have an outcome",
              subtitle = "Are they unsolved or is it just a reporting issue?",
              y="Number of Records per Day",
              x="Date")

# Themes
g <- g + 
  theme_minimal() +
  theme(axis.title = element_text(size=rel(1.3)),
        axis.text.y = element_text(size=rel(1.2)),
        axis.text.x = element_text(size=rel(1.4)),
        axis.title.x = element_text(margin=margin(t=15)),
        axis.title.y = element_text(margin=margin(r=15)),
        plot.title = element_text(size=rel(1.6), color="#D9230F"),
        plot.background = element_rect(fill="#FCFCFC", color = NA),
        legend.background = element_rect(fill="#fcfcfc", color="#dedede"),
        legend.position = c(0.75, 0.99),
        legend.direction = "horizontal") 

g
```

Your task for today (in the morning and in the afternoon) is to **POSE MORE QUESTIONS TO THE DATASET** and **CREATE NEW PLOTS**. Instead of bringing in more advanced content, let's practice!

> **Write a new question here**

```{r}
# Produce your plot here
```

> **Keep adding new questions**

```{r}
# Produce a new plot here
```



# 🗒️ Conclusions

Write down 