Pharm D Program
PHM305H1: Medication Therapy Management 4
|Read the Full PDF Report|
I joined D3QA because I wanted to collaborate with data scientists to assess whether a pre-testing intervention can improve the performance of struggling students through closer look at outcomes. This will potentially help the teaching team design an adaptive learning module in subsequent years.
The overarching goal is improvement of student performance during simulations. Learners are currently provided with customized resources and learning activities prior to the simulation to scaffold and direct students towards studying strategies/content areas.
Intervention: pre-module quizzes (designed to test students on content they don’t know yet) which are designed to prime students for the post-module quizzes (and simulations)
The goal for the redesign was to confirm whether students who perform poorly on a pre-test (prior to the lecture and simulation) show improvement on post-tests and simulations.
This will help the PHM305 team design an adaptive learning module in subsequent years.
Three questions were explored:
- Can the pre-module and post-module quiz scores be used to identify students who are at risk of failing the simulation?
- Is there a correlation between a student’s performance on a pre-test and their performance on a post-test and final simulation?
- In which ways can I identify gaps in student’s content knowledge ?
The following data sources were used:
- Pre-module quiz scores
- Post-module quiz scores
- Simulation scores
Use of Data to Inform Design Iteration and Instruction
I found that all students achieved a comparable overall score on the post-quiz independently of their performance in the pre-quiz. Furthermore, students with the lowest performance on the pre-quiz appeared to benefit the most from the intervention.
The image shows the distribution of Lab 3 Honours/Pass/Fail (H/P/F) grades according to the pre-quiz categories.
Future questions to consider include:
- Can prior experience working in a pharmacy be used instead of using the pre-quiz as a proxy for prior knowledge?
- What is the reason for the increase in the performance of students in the lowest pre-quiz categories?
I will build in adaptive learning components to the pre-quiz.