🗓️ 2023/24 Winter Term – Activities

What will happen in the GENIAL project between January and March 2024?

This page contains all the activities that we have planned for the Winter Term of the 2023/24 Academic Year, as well as information about the prizes that students can win by participating in the project.

♻️ Important changes

After the first Term of the project, we have decided to make some changes to our methodology following our observations and feedback from participating students of what worked and what didn’t. We noticed, for example, that reserving classroom time for independent work with generative AI tools was not very effective. Although we spent a good amount of time tweaking the class plan to make it fit with the use of generative AI tools, it didn’t always fit with the course content.

Here we list the main changes that we are making to the project this Term.

What remains the same:

• We will still run a comprehensive survey at the start of the Term to gauge students’ prior knowledge and experience with generative AI tools. Estimated time to complete: 10-15 minutes.
• Every week, we will ask students to complete a very brief survey where they will share their chat logs and answer 2-3 brief questions on their experience with generative AI tools. Estimated time to complete: 3 minutes.
• We will still run a final survey at the end of the Term to gauge students’ overall experience with generative AI tools. Estimated time to complete: 10-15 minutes.
• Course convenours will still have autonomy on the content and structure of their courses. We will not interfere with their teaching or assessment methods.
• We will NOT give any guidance on the use of ChatGPT on the first half of the courses (Weeks 01-05). Students will be asked to use it however they feel like it - the purpose of this is to observe how students use the tool, whether they are already proficient with it or not.
• There will still be an ‘intervention’ on Week 06, in which we will communicate to students:
  • the common criticisms of generative AI tools in education
  • ‘prompt engineering’ tricks to get the most out of generative AI tools
• From Week 07 - Week 11, instructors will be free to provide tips about effective use of generative AI tools when they find fit or when students ask for it.

What changes this Term:

• The study will no longer operate on an opt-in basis. Instead, all students enrolled in the participating courses will be automatically enrolled in the study.
  • There will be an opt-out option for students who do not wish to participate in the study.
• We will focus exclusively on ChatGPT, as it is the most popular generative AI tool among our participants and it is easier to export chat logs. ¹
• Students will be tasked with creating a dedicated ChatGPT chat for the course every week. This chat is no longer restricted to activities in the classroom. Students can use it to ask questions while preparing for, during, or reviewing the week’s content.
• Students will have to use ChatGPT with GPT 3.5, and we will not allow GPT 4 chats.
  • The reason for this is that exporting chat logs from GPT 4 did not always work reliably in our tests. We encountered a message stating ‘Sharing conversations with images is not yet supported,’ which reflects a restriction on exporting when plugins such as DALL-E are used in the chat. Since plugins are active by default and a user might use one without realising it, we have decided to stick with GPT 3.5 for now.
• We will run three focus group sessions with participants in key moments of the Term.
  • These are tentatively scheduled for: Week 03, Week 05 and Week 09.
  • These sessions will be separated by course and will last 45-60 minutes each.
  • We will design questions that will help us understand more deeply how the students connect the content of the course with their use of generative AI tools.
• We will not run any observational experiments this Term.
  • We will no longer ask students to work independently with generative AI tools during class time.

🗒️ Timeline

The table below shows the planned activities for the Winter Term of the 2023/24 Academic Year. Our methodology has changed since the Autumn Term 2023/24. While in the Autumn Term, we allocated classroom time for independent work with generative AI tools, in the Winter Term, we are embracing a broader approach that extends beyond the classroom. Our primary objective remains barely unchanged, though: To explore how undergraduate students (in our disciplines) use generative AI tools for learning and under what circumstances these tools serve as catalysts for learning.

Note to outsiders: LSE’s Academic Terms are composed of 11 weeks of teaching, typically with a reading week in the middle (Week 06), when there are no classes or lectures scheduled. The Winter Term starts in January and ends in March.

Table 1. Activities of the GENIAL project during Winter Term 2023/24. Key activities are highlighted in bold font and colours.

Week	Date	Activity
W01	15 Jan 2024	📩 GENIAL Newsletter 01 (Welcome message + link to initial survey + instructions on how to access AI chatbots)
W01	15 Jan 2024 - 19 Jan 2024	🧑🏻‍🏫 Students will be told about the purpose of the project in their first lectures/classes. - Lecturers will reserve ~10 minutes of their lecture for the initial survey - Class teachers will request students to create a dedicated chat window for all course-related interactions using an AI chatbot - It’s essential that any course-related use of an AI chatbot takes place within this designated chat window - Instructors communicate to students that they won’t be allowed to give them tips on the use of AI
W02	22 Jan 2024	📩 GENIAL Newsletter 02 (Communicate progress with of the initial survey + reminder to keep a chat log)
W02	22 Jan 2024 - 26 Jan 2024	🧑🏻‍🏫 Lectures and classes run as usual (At the start of each small class, class teachers provide students with a link to export their chat logs of the week)
W03	29 Jan 2024	💻 Internal data analysis (GENIAL researchers start looking at the chat log data in search of patterns and insights)
W03	29 Jan 2024	📩 GENIAL Newsletter 03 (info about first round of focus groups + weekly reminder to keep a chat log)
W03	29 Jan 2024 - 2 Feb 2024	🧑🏻‍🏫 Lectures and classes run as usual (At the start of each small class, class teachers provide students with a link to export their chat logs of the week)
W03	29 Jan 2024 - 2 Feb 2024	🎙️ Focus Groups (round 1) - Researchers will interview participants about how genAI usage impacts their course learning. - The session will be audio recorded - Each participating course (DS105W, DS202W, ST456) will have its own focus group session - The timing of the sessions will be communicated in the weekly newsletter - Each session will be limited to the first 10 participants who arrive first
W04	5 Feb 2024	📩 GENIAL Newsletter 04 (weekly reminder to keep a chat log)
W04	5 Feb 2024 - 9 Feb 2024	🧑🏻‍🏫 Lectures and classes run as usual (At the start of each small class, class teachers provide students with a link to export their chat logs of the week)
W05	12 Feb 2024	📩 GENIAL Newsletter 05 (info about upcoming W06 activities + weekly reminder to keep a chat log)
W05	12 Feb 2024 - 16 Feb 2024	🧑🏻‍🏫 Lectures and classes run as usual (At the start of each small class, class teachers provide students with a link to export their chat logs of the week)
W06	19 Feb 2024	📩 GENIAL Newsletter 06 (reminder of the W06 open lecture + weekly reminder to keep a chat log)
W06	19 Feb 2024 - 23 Feb 2024	🧑🏻‍🏫 Open Lecture: Using AI chatbots for Learning - Core members of the GENIAL project will deliver a lecture to all students - We will explain how the AI chatbots work and what is prompt-engineering - We will discuss the current discourse around AI as well as the criticisms of its use in education and in general - We will share our findings from your logs and participation in the focus groups. - We will showcase the uses of AI chatbots made by participants so far that made their learning experience better - We will show cases where participants shared that the use of AI made them worse at learning
W07	26 Feb 2024	📩 GENIAL Newsletter 07 (summary of open lecture + info about focus groups + weekly reminder to keep a chat log)
W07	26 Feb 2024 - 1 Mar 2024	🎙️ Focus Groups (round 2) - Researchers will interview participants about how genAI usage is impacting their course learning - The session will be audio recorded - Each participating course (DS105W, DS202W, ST456) will have its own focus group session - The timing of the sessions will be communicated in the weekly newsletter - Each session will be limited to the first 10 participants who arrive first
W07	26 Feb 2024 - 1 Mar 2024	🧑🏻‍🏫 Lectures and classes run as usual - From now on, instructors will be free to give prompt engineering tips to students. - At the start of each small class, class teachers provide students with a link to export their chat logs of the week
W08	4 Mar 2024	📩 GENIAL Newsletter 08 (prompt engineering tips + weekly reminder to keep a chat log)
W08	4 Mar 2024 - 8 Mar 2024	🧑🏻‍🏫 Lectures and classes run as usual - Instructors will be free to teach tips on the effective use of AI, students can also ask for prompt-engineering tips - At the start of each small class, class teachers provide students with a link to export their chat logs of the week
W09	11 Mar 2024	📩 GENIAL Newsletter 09 (prompt engineering tips + weekly reminder to keep a chat log)
W09	11 Mar 2024 - 15 Mar 2024	🧑🏻‍🏫 Lectures and classes run as usual - Instructors will be free to teach tips on the effective use of AI, students can also ask for prompt-engineering tips - At the start of each small class, class teachers provide students with a link to export their chat logs of the week
W10	18 Mar 2024	📩 GENIAL Newsletter 10 (info about final round of focus groups + prompt engineering tips + weekly reminder to keep a chat log)
W10	18 Mar 2024 - 28 Mar 2024	🎙️ Focus Groups (round 3) - Researchers will interview participants about how genAI usage is impacting their course learning - The session will be audio recorded - Each participating course (DS105W, DS202W, ST456) will have its own focus group session - The timing of the sessions will be communicated in the weekly newsletter - Each session will be limited to the first 10 participants who arrive first
W10	18 Mar 2024 - 28 Mar 2024	🧑🏻‍🏫 Lectures and classes run as usual - Instructors will be free to teach tips on the effective use of AI, students can also ask for prompt-engineering tips - At the start of each small class, class teachers provide students with a link to export their chat logs of the week
W11	25 Mar 2024	📩 Final Newsletter + the Final survey (info about final survey + weekly reminder to keep a chat log)
W11	25 Mar 2024 - 15 Mar 2024	🧑🏻‍🏫 Lectures and classes run as usual - Instructors will be free to teach tips on the effective use of AI, students can also ask for prompt-engineering tips - At the start of each small class, class teachers provide students with a link to export their chat logs of the week

🎟️ Student prizes

Students who actively participate in the project will be eligible to win one of the following prizes:

• 1x £100 Amazon voucher
• 3x £50 Amazon voucher

Every time a student engages with a GENIAL activity, they will receive a raffle ticket. The more activities they engage with, the more raffle tickets they will receive. At the end of the term, we will randomly select the winners of the prizes.

This is a list of the activities that will earn students a raffle ticket:

• Initial survey (W01) - 3 raffle tickets
• Weekly survey (W02-W10) - 1 raffle ticket per week
• Focus group (W03, W07, W10) - 5 raffle ticket per participation in focus group
• Final survey (W11) - 3 raffle tickets

Footnotes

1. We will not be using GitHub Copilot in this Term. GitHub Copilot’s autocomplete feature is extremely useful for programming in some of the participating courses, and it does have a good adoption rate among our participants. However, the only effective way we could think of collecting usage would be to ask students to record their screens to capture their use of GitHub Copilot. This would be nice but we do not have a full team of researchers to watch all these videos at the moment. We will revisit this in the future.