🗓️ Week 02 - Data types and the concept of tidy data

2023/24 Autumn Term

Author

What does data look like? And how do we organise it? This week’s lecture will discuss the concept of tidy data and how to organise data, even if in a spreadsheet. We will also discuss the different data types and how to deal with them.

👨‍🏫 Lecture Slides

Either click on the slide area below or click here to view it in fullscreen. Use your keypad to navigate the slides. You can also find a PDF version on Moodle.

🎥 Looking for lecture recordings? You can only find those on Moodle.

References used in the lecture slides this week

Abeysooriya, Mandhri, Megan Soria, Mary Sravya Kasu, and Mark Ziemann. 2021. “Gene Name Errors: Lessons Not Learned.” PLoS Computational Biology 17 (7): e1008984.

Baraniuk, Chris. 2015. “The Number Glitch That Can Lead to Catastrophe.” BBC Future, May. https://www.bbc.com/future/article/20150505-the-numbers-that-lead-to-disaster.

BBC News. 2014. “Gangnam Style Music Video ’Broke’ YouTube View Limit.” BBC News, December. https://www.bbc.co.uk/news/world-asia-30288542.

Beales, Richard. 2013. “Blame Microsoft.” New York Times (Online). 2013. https://www.proquest.com/blogs-podcasts-websites/blame-microsoft/docview/2215333604/se-2.

Felbo, Bjarke, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. “Using Millions of Emoji Occurrences to Learn Any-Domain Representations for Detecting Sentiment, Emotion and Sarcasm.” In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 1615–25. Copenhagen, Denmark: Association for Computational Linguistics. https://doi.org/10.18653/v1/D17-1169.

Gibbs, Samuel. 2014a. “Is the Year 2038 Problem the New Y2K Bug?” The Guardian (London). https://www.theguardian.com/technology/2014/dec/17/is-the-year-2038-problem-the-new-y2k-bug.

———. 2014b. “Y2K Bug Triggers Army Conscription Notices Sent to 14,000 Dead Men.” The Guardian (London). https://www.theguardian.com/technology/2014/jul/11/y2k-bug-us-army-conscription-1800s-pennsylvania.

Hern, Alex. 2020. “Covid: How Excel May Have Caused Loss of 16,000 Test Results in England.” The Guardian. https://www.theguardian.com/politics/2020/oct/05/how-excel-may-have-caused-loss-of-16000-covid-tests-in-england.

Kwak, James. 2013. “The Importance of Excel.” "The Baseline Scenario" Blog, February. https://baselinescenario.com/2013/02/09/the-importance-of-excel/.

Leek, Jeff. 2016. “Non-Tidy Data.” "Simply Statistics" Blog. https://simplystatistics.org/posts/2016-02-17-non-tidy-data/.

Oren, Nir. 2019. “If You Think the Millennium Bug Was a Hoax, Here Comes a History Lesson.” The Conversation. http://theconversation.com/if-you-think-the-millennium-bug-was-a-hoax-here-comes-a-history-lesson-129042.

Thomas, Martyn. 2019. “The Millennium Bug Was Real – and 20 Years Later We Face the Same Threats.” The Guardian (Online). https://www.theguardian.com/commentisfree/2019/dec/31/millennium-bug-face-fears-y2k-it-systems.

Uenuma, Francine. 2019. “Years Later, the Y2K Bug Seems Like a Joke–Because Those Behind the Scenes Took It Seriously.” Time. https://time.com/5752129/y2k-bug-history/.

Vincent, James. 2020. “Scientists Rename Human Genes to Stop Microsoft Excel from Misreading Them as Dates.” The Verge, August. https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates.

Wickham, Hadley. 2014. “Tidy Data.” Journal of Statistical Software 59 (10). https://doi.org/10.18637/jss.v059.i10.

Wickham, Hadley, and Garrett Grolemund. 2016. R for Data Science: Import, Tidy, Transform, Visualize, and Model Data. First edition. Sebastopol, CA: O’Reilly. https://r4ds.had.co.nz/.

Williams, Martyn. 2000a. “Computer Problems Hit Three Nuclear Plants in Japan.” CNN. https://edition.cnn.com/2000/TECH/computing/01/03/japan.nukes.y2k.idg/index.html.

———. 2000b. “Y2K Bug Hits Heating System in Korean Apartments.” CNN. https://edition.cnn.com/2000/TECH/computing/01/03/korea.heat.y2k.idg/index.html.

Ziemann, Mark, Yotam Eren, and Assam El-Osta. 2016. “Gene Name Errors Are Widespread in the Scientific Literature.” Genome Biology 17 (1): 1–3.

✍️ Coursework (Formative)

Collect data from Wikipedia and make it tidy

Go to Wikipedia, click on the Main Menu (top left), and click on the “Current events” link.
Create a new spreadsheet and add all “Ongoing events” to it.
Collect data for each event: the topic, key dates, period, location, the event, the number of people affected, etc. (feel free to get creative here).
Consider collecting metadata too: when was the event added to Wikipedia, who added it, etc.
Now, reorganise your spreadsheet to ensure only one column for each variable and a row for each observation. That is, convert it to tidy data.
Save your spreadsheet as a CSV file and upload it to Moodle under the “Week 02 Formative - Tidy data” assignment in the “✍️ Coursework and Assessment” section.
- This will help you build a portfolio of your work throughout the course. We might ask you to refer to this material during your summative assessments.

📚 Recommended Reading

Check the end of slides for the list of references cited in the lecture.
Check the 📔 Syllabus for this week’s complete list of indicative and recommended readings.

📟 Communication

Post your reflections, questions, and links on Slack (depending on your purpose, use either of the #general, #help-classes or #help-lectures channels - you might also want to use the #interesting-articles channel).