Return to D3:QA Showcase

Naomi Steenhof

Course/Program Pharm D Program PHM305H1: Medication Therapy Management 4​ This course focuses on developing and enhancing skills needed to optimize the pharmacist’s scope of practice in providing effective patient care in Ontario.

Read the Full PDF Report

Design Context

I joined D3QA because I wanted to collaborate with data scientists to assess whether a pre-testing intervention can improve the performance of struggling students through closer look at outcomes.​ This will potentially help the teaching team design an adaptive learning module in subsequent years.

Instructional Challenge

The overarching goal is improvement of student performance during simulations. ​Learners are currently provided with customized resources and learning activities prior to the simulation to scaffold and direct students towards studying strategies/content areas.

Design Strategy

Intervention: pre-module quizzes (designed to test students on content they don’t know yet) which are designed to prime students for the post-module quizzes (and simulations)​ 

​The goal for the redesign was to confirm whether students who perform poorly on a pre-test (prior to the lecture and simulation) show improvement on post-tests and simulations.​ 

This will help the PHM305 team design an adaptive learning module in subsequent years.

Three questions were explored:

• Can the pre-module and post-module quiz scores be used to identify students who are at risk of failing the simulation?
• Is there a correlation between a student’s performance on a pre-test and their performance on a post-test and final simulation? ​
• In which ways can I identify gaps in student’s content knowledge ?

The following data sources were used:

• Pre-module quiz scores ​ 
• Post-module quiz scores ​ 
• Simulation scores  

Use of Data to Inform Design Iteration and Instruction

I found that all students achieved a comparable overall score on the post-quiz independently of their performance in the pre-quiz​. Furthermore, students with the lowest performance on the pre-quiz appeared to benefit the most from the intervention. 

The image shows the distribution of Lab 3 Honours/Pass/Fail (H/P/F) grades according to the pre-quiz categories.

Next Steps

Future questions to consider include:

• Can prior experience working in a pharmacy be used instead of using the pre-quiz as a proxy for prior knowledge?
• What is the reason for the increase in the performance of students in the lowest pre-quiz categories?

I will build in adaptive learning components to the pre-quiz.

Report

Click to download full report PDF in a new tab