Steenhof – Pharmacy

Return to D3:QA Showcase

Naomi Steenhof


Pharm D Program

PHM305H1: Medication Therapy Management 4
This course focuses on developing and enhancing skills needed to optimize the pharmacist’s scope of practice in providing effective patient care in Ontario.

icon of a report with text and a bar graphRead the Full PDF Report

Design Context

I joined D3QA because I wanted to collaborate with data scientists to assess whether a pre-testing intervention can improve the performance of struggling students through closer look at outcomes.​ This will potentially help the teaching team design an adaptive learning module in subsequent years.

Instructional Challenge

The overarching goal is improvement of student performance during simulations. Learners are currently provided with customized resources and learning activities prior to the simulation to scaffold and direct students towards studying strategies/content areas.

Design Strategy

Intervention: pre-module quizzes (designed to test students on content they don’t know yet) which are designed to prime students for the post-module quizzes (and simulations)​ 

The goal for the redesign was to confirm whether students who perform poorly on a pre-test (prior to the lecture and simulation) show improvement on post-tests and simulations.​ 

This will help the PHM305 team design an adaptive learning module in subsequent years.

Three questions were explored:

  • Can the pre-module and post-module quiz scores be used to identify students who are at risk of failing the simulation?
  • Is there a correlation between a student’s performance on a pre-test and their performance on a post-test and final simulation? ​
  • In which ways can I identify gaps in student’s content knowledge ?

The following data sources were used:

  • Pre-module quiz scores ​ 
  • Post-module quiz scores ​ 
  • Simulation scores  

Use of Data to Inform Design Iteration and Instruction

I found that all students achieved a comparable overall score on the post-quiz independently of their performance in the pre-quiz​. Furthermore, students with the lowest performance on the pre-quiz appeared to benefit the most from the intervention. 

The image shows the distribution of Lab 3 Honours/Pass/Fail (H/P/F) grades according to the pre-quiz categories.

Graph showing distribution of Lab 3 H/P/F grades according to the pre-quiz categories

Distribution of Lab 3 H/P/F grades according to the pre-quiz categories

Next Steps

Future questions to consider include:

  • Can prior experience working in a pharmacy be used instead of using the pre-quiz as a proxy for prior knowledge?
  • What is the reason for the increase in the performance of students in the lowest pre-quiz categories?

I will build in adaptive learning components to the pre-quiz.


Click to download full report PDF in a new tab