Labster Virtual Labs Final Report

Final Report

Executive Summary – PDF Download
Full Report – PDF Download

By Hedieh Najafi, hedieh.najafi@utoronto.ca, Laurie Harrison, laurie.harrison@utorotno.ca and Will Heikoop, will.heikoop@utoronto.ca

Online Learning Strategies

University of Toronto

April 2018

 

Overview

This report explains the design and implementation processes and outlines insights gained from a virtual lab integration pilot project, Active Learning Opportunities Through Virtual Lab Curricular Integration, at the University of Toronto where biology related virtual laboratories were integrated into undergraduate courses.

Laboratory experience in biology-related undergraduate courses may not always be available on-site, thus justifying initiatives that examine the educational value of virtual labs. Existing research shows that immersive simulations may provide an alternative option that facilitates equivalent learning outcomes and supports the possibility of improved overall learning experiences for students (De Jong et al., 2013). To provide access to biology virtual labs to students at the University of Toronto, we collaborated with a group of instructors who were eager to integrate virtual labs in their undergraduate courses. In this initiative, Online Learning Strategies (OLS) portfolio staff and Information Technology staff at U of T worked closely with this group of faculty members and provided curriculum design and technical integration support.

Labster Virtual Lab Curriculum Integration Pilot Project

In 2016, project team members at U of T selected virtual labs created by Labster (https://www.labster.com/), an established company specializing in interactive virtual lab development. Previously, Labster has developed online laboratory platforms and virtual reality-based labs for multiple elite institutions globally. Labster has a rich collection of simulations in biology in line with the interests of the first group of U of T faculty members who joined the pilot phase of this project as collaborators. While Labster is also a leader in developing 3D simulations, the labs selected in the pilot phase of the project were 2D and no additional technology other than a computer or tablet was required once the instructor provided access to the labs. In this report we explain the logistics of curriculum design and implementation and share insights gained from an evaluation project that OLS staff conducted throughout the integration process.

Institutional Lead and Collaborating Instructors

The pilot phase of this project was undertaken by five U of T instructors at St. George campus, three OLS staff members who provided administrative coordination, and a specialist from Labster. Instructors participating in the project represent courses across several program areas (Table 1), including Biomedical Engineering, Cell and Systems Biology, and Human Biology program areas.  The Labster specialist provided consultations for Virtual Lab content integration, logistics, and assessment components. The project team included:

  • Project lead: Dr. Laurie Harrison, Director, Online Learning Strategies
  • Collaborating instructors:
    • Dr. Chris Garside, Assistant Professor, Teaching Stream, Department of Cell and Systems Biology
    • Dr. William Ju, Associate Professor, Teaching Stream, Human Biology Program
    • Dr. Dawn Kilkenny, Assistant Professor, Teaching Stream, Institute of Biomaterials & Biomedical Engineering
    • Dr. Maria Papaconstantinou, Associate Professor, Teaching Stream, Human Biology Program
    • Dr. Andrey Shukalyuk, Institute of Biomaterials & Biomedical Engineering Teaching Laboratory Coordinator
  • Labster (partner organization): Sam Butcher, Scientific Collaboration Specialist

This multi-expertise group closely collaborated for over a year, starting March 2017, to identify relevant labs, design course curricula, address IT integration aspects, implement Labster virtual labs in four undergraduate courses, and evaluate the pilot phase of the project. Labster provided a one-year licence for 250 students in each fall semester of 2017-2018 academic year. Additional courses will continue to integrate Labster virtual labs in the summer of 2018.

 

Table 1

Undergraduate courses included in the pilot phase of the project

Department Course name Level Class size Labs used Instructor
Institute of Biomaterials & Biomedical Engineering, Faculty of Applied Science & Engineering BME350, Biomedical Systems Engineering I: Organ Systems 3rd year 36 students · Smooth muscle

· Diabetes

 Dawn Kilkenny
Cell and Systems Biology, Faculty of Arts & Science BIO270, Animal Physiology I 2nd year 200 students · Smooth muscle

· Cellular respiration

· Diabetes*

· Carbohydrates*

· Enzyme kinetics*

· Protein denaturation*

 Chris Garside
Human Biology, Faculty of Arts & Science HMB360H1F, Neurogenomics 3rd year 96 students · iPSCs & Viruses

· Next Generation Sequencing

 William Ju
Human Biology, Faculty of Arts & Science HMB441sH1F, Genetics of Human Disease 4th year 50 students · Next generation sequencing

· Medical genetics

· Gene regulation

 Maria Papaconstantino

* Optional labs

Instructor Capacity Building

Beginning in April 2017, the project team met five times to plan, execute, and evaluate the Integration project. OLS staff members were in contact with the collaborating instructors and Labster specialists to follow up with their progress of IT integration and pedagogical planning and to identify and address potential issues. The following meetings were coordinated and organized by OLS staff, as well as ongoing communication on project activities:

  • April 2017: Introduction and orientation
  • May 2017: Curriculum development and contextualizing virtual labs within courses
  • August 2017: Addressing technical integration of Labster virtual labs within Blackboard; Sharing new course syllabi
  • October 2017: Mid implementation round table to share feedback and early lessons learned
  • February 2018: Post implementation round table to discuss learning benefits of Labster virtual labs and potential next steps

Faculty development and capacity building with regards to IT integration and curriculum design was ongoing and provided on a needs basis. However, other than initial setting up of the Labster virtual lab links in Blackboard, the collaborating instructors required minimal support and assistance in implementing the labs. IT integration support was provided by an OLS staff member who acted as a liaison and was in contact with both the instructors and Labster support personnel.

Curriculum design support was facilitated through alignment templates, explained in more detail in the evaluation section of this report. The goal of curriculum design support was to ensure that Labster virtual labs, as learning activities, were explicitly contextualized within the courses.

Integration and Support

Information Risk and Risk Management Assessment was conducted in June 2017 through two virtual meetings between U of T and Labster technical leads. In these meetings Labster’s IT security and data management practices were examined for congruence with U of T standards of practice. The results of this consultation were satisfactory to U of T.

The process for LTI integration included several steps beginning with Labster providing licence key. LMS operations team was in charge of the installation process and testing on QA servers and determined the resources are fully interoperable. Instructor and student support documentation were also reviewed, with some differences between the documentation and the current LTI implementation noted.

Another point of attention for the collaborating instructors and the administrators was the accessibility of the labs. The university requested that Labster provide accessibility information regarding the virtual labs. Labster shared a self-assessment accessibility report in the form of a Voluntary Product Accessibility Template (VPAT). While not identical to Accessibility for Ontarians with Disabilities Act (AODA) in terms of specific requirements, the VPAT does provide evidence of their general compliance with accessibility guidelines. Labster noted several tasks they undertook in their development roadmap over the past academic year:

  • Testing the status quo for keyboard navigation
  • Improved voice narration: Ivona to Neospeech due to a better and improved voice quality to help hearing impaired students
  • Testing improvement of color contrast (Completed in October 2017)

The Product Development team also deployed a new Contrast Color ratio for all of our simulations. This means that the LabPad color contrast is 4:5:1 ratio, which meets accessibility guidelines.

Through their course LMS website, instructors could access Labster dashboard data that provided a summary of students activity for each lab. Figure 1 shows a generic example of dashboard data available to the instructors. Initially, the project team anticipated a more detailed activity log with information such as students’ response to individual questions within a lab. In practice the log provided by Labster lacked such granularity. Students activity information provided in the log showed the number of attempts, time spent, performance in terms of score obtained, and progress through a lab.

Figure 1. Labster dashboard.

An Information Technology team at U of T supported the technical aspects of Labster labs integration into undergraduate courses. The team allocated 40 hours to the following tasks

  • Backend implementation of API integration
  • User acceptance testing
  • Adding the key open access to Labster functionality
  • Reviewing security and data privacy conducted by Information Security and Enterprise Architecture (ISEA) team
  • Liaison with Labster and detailed testing of functionality

Preparation and necessary follow up for each team meeting was provided by the OLS team and totalled approximately 60 hours committed to faculty development support. TA support and training was part of usual training time and no additional time was required. Student support was not required either as the students did not need extra support to use Labster virtual labs.

Project Evaluation

A research study and an evaluation project were designed to investigate the impact of Labster virtual labs curriculum integration on students’ learning and to examine the process of curriculum design and implementation, respectively. The research project was designed and conducted by the collaborating instructors. Analysis of data collected in the research study is underway at the time of writing this report and the research team will present the findings in conferences and/or academic journals.

OLS staff members led an evaluation project to assess various aspects of the pilot project. The evaluation project was approved as program evaluation, quality assurance, or quality improvement project and thus was exempt from research ethics review. Within the context of this evaluation project, OLS staff worked with the collaborating instructors throughout the process of integrating Labster virtual labs into their courses. The outcomes of the evaluation project are high-level insights regarding opportunities for use of virtual labs at UofT that we present in this report. The evaluation project included three phases: Design, implementation, and reflection and redesign.

Design Phase

In the Design Phase, OLS staff met with the instructors three times between May 2017 and September 2017 to clarify how their selected Labster virtual labs fit with relevant learning outcomes, course activities, and course assessments. The instructors completed an alignment template in June 2017 and later answered a survey that collected their expected integration goals with regards to students’ learning benefits, and their expected resources and support for successful virtual lab integration. OLS team met with the instructors once more in August to review their final virtual labs integration plans and to provide technical information regarding linking the virtual Labs into Blackboard and setting up students’ access to the selected Labster virtual labs. The instructors also shared their course syllabi from 2016-2017 and 2017-2018 academic years.

Regarding anticipated positive impact of virtual labs integration in the selected courses, the instructors indicated these areas of potential impact: depth of understanding (four instructors), motivation and retention (three instructors), and access (two instructors). As for resources needed to successfully integrate virtual labs into their courses, the instructors identified Labster related guides and supporting documents for the students, TA support, and tech support.

The instructors agreed to use similar language across all course to introduce Labster virtual labs, and to connect the labs to assessments and other course activities. One instructor also included an additional document with more elaborate information regarding the timeline of lab completion and the relevance of labs to a lecture and to an on-site lab. Each instructor selected between two to three labs. Grading policy varied as a combination of 1) participation-based with a minimum required grade obtained in the virtual labs, 2) use in a pre-lab report, and 3) inclusion of lab-related questions in mid-term and final exams. To integrate the virtual labs into their courses, the instructors slightly modified their grading policy compared to 2016-2017 academic year.

Implementation Phase

When meeting in November 2017, the instructors shared their experiences and observations with Labster virtual lab integration for labs implemented so far. No major obstacles or problems regarding students’ use of the labs were reported in this meeting. At the end of the session, the instructors also completed a short open response implementation reflection template. Course implementation aligned well with course design across all courses and no changes were made to the original implementation plans as indicated in course syllabi. Labster virtual labs served different purposes across different courses:

  • Pre-lab activity
  • Lecture substitute
  • Extending current teaching
  • Providing lab work opportunity in seminar courses with no existing lab
  • Opportunity to work on a lab that did not currently exist
  • Refreshing students knowledge before teaching new material

Labster labs include theoretical information which one of the instructors found to be of high quality. Moreover, in those courses that on-site labs were not available, the instructors noted that the students had the opportunity to connect their classroom learning with their experience in the integrated Labster virtual labs. The instructors indicated their plan to include Labster virtual lab questions in midterm or final exam and the students were informed that the virtual lab contents were exam eligible. Informal early student feedback ranged from virtual labs being exciting to being enjoyable, and in some cases, repetitive.

The instructors raised a number of concerns regarding the integrated labs. Labster virtual labs environment felt different from on-site labs and it took some time to adjust to the non-physical environment regarding usability and conduct. Also, the instructors expressed their preference for a more granular and detailed student activity log that showed the results of each question in Labster virtual labs quizzes. The currently available data dashboard (Figure 1) provides the overall performance of a student in a given lab.

Reflection and Redesign

In the final phase of the evaluation project, the instructors shared highlights of their experience after the fall 2017 courses had ended. In this focus group meeting held in February 2018, OLS staff facilitate the discussion using open ended thematic questions to gain an understanding of the overall impact of the integrated Labster virtual labs on students’ learning and to examine potential future steps. We present the insights gained from this meeting in six themes: Rationale for virtual lab integration, observed impact on students learning, desired changes in Labster virtual labs to increase impact, informal students’ feedback, curricular integration, and sustainability.

Rationale for virtual lab integration. Instructors noted several reasons for attempting the Labster virtual lab curriculum integration, the most foundational being the possibility to extend current teaching. For example, the instructors indicated that digital resources have already changed the way students approach learning. With a plethora of electronic resources available to them, students can complement their course work and can clarify concept that they may not have deeply grasped during lectures. Instructors can, thus, use digital resources such as Labster virtual labs to purposefully deepen students learning and to deliberately connect those resources to the course learning objectives. Moreover, virtual labs can enhance students teaching and learning experience for concept that are either complex or are too resource intensive to teach in on-site labs or face to face. Labster was the first interactive virtual environment that became available to the instructors to pilot in their courses. Increasing student motivation was another reason for virtual lab integration.

Observed impact on students’ learning. The instructors highlighted the benefits of self-paced Labster virtual labs for students with accessibility requirements. Students can use these virtual labs as many times as they want and are not required to complete a lab within a certain time limit. One instructor emphasized that virtual labs can be completed at times and in places convenient to the students, thus providing them with a desired flexibility in planning for their learning. Information about the number of attempts and students’ progress in each lab is available to the instructors through Labster dashboard.

A pedagogically significant feature of Labster virtual labs was the embedded theory links within each lab. The instructors highlighted the importance of the connections between theoretical concepts and hands on labs experiments. This direct connection provides contextualized justification for the theoretical underpinning of labs. One instructor specifically pointed out to the high quality of the theoretical component of the Labster virtual labs. Another learning benefit of Labster virtual labs was facilitating students’ deeper understanding of course key concept concepts through animations. Overall, the instructors agreed that Labster virtual labs could be integrated within their courses to extend lectures, to complement midterm and final exam, and to offer much-desired flexible learning resource for the students.

Desired changes in Labster virtual labs to increase impact. With their first Labster virtual labs curriculum integration completed, the instructors reflected on how these labs can enhance students learning in a more effective way. At this time, Labster virtual labs are non-customizable. The instructors expressed their interest in having more granular control over the labs so that they can select the most relevant section of the virtual labs to complement and extend their instruction. Labster virtual labs also include quiz questions that accompany each lab. The instructors suggested these questions could be modifiable to connect more seamlessly with course content and learning goals. During the fall 2017 semester students taking Labster integrated courses shared informal comments about their experience with Labster virtual labs with their instructors. A desired improvement to the labs was the ability to save their work, exit the lab, and resume it at a later time. Currently, upon exiting a Labster virtual lab, students need to start over from the first step.

As we mentioned in the previous sections of the report, Labster virtual labs contained high quality theory pages that students could study to better understand the link between theory and lab experiments. The instructors suggested that additional fields in data dashboard could provide more detailed information on whether and how often the student visited the theory pages. For one course, the instructor questioned if Labster would develop labs on specific topics that are not currently available in the collection. Opportunities for co-development of the labs were not discussed with Labster at this time. Finally, in the fall 2017 semester, the instructors could only access 2D Labster labs that did not necessarily facilitate the development of students’ tactile knowledge of performing in on-site labs. The instructors inquired whether the upcoming 3D virtual lab option would meet this need.

Informal student feedback. The collaborating instructors designed and conducted a study to investigate learners’ perception of Labster virtual labs integrated in their courses. At the time of writing this report, data analysis for that study is underway and results will be shared in conferences and/or published in relevant journals. During the fall 2017 semester, however, the students shared their informal feedback and thoughts with their instructors. In our post-implementation focus group, the instructors recounted these high level comments anonymously. From a technical point of view, students responded positively to the integrated labs.

Overall, the students thought Labster virtual labs were aligned with their learning as supplementary resources, lecture substitute, and pre-lab activities. Another positive feedback was related to the Labster virtual labs being available free of charge within these courses. Students also mentioned minor design issues of concern as they related to lab safety protocols. For example, the students noticed that contrary to an on-site lab, in one Labster virtual lab they could discard sharp objects with non-sharp objects.

Labster virtual labs integration in course material and assessments. We asked the collaborating instructors to share their newly designed course syllabi for the fall 2017 semester with us. Syllabi documents included the following examples of how the instructors integrated virtual labs in their course. In one course, a virtual lab substituted a lecture session where the instructor was attending a conference. In the same course, the students used another virtual lab as a pre-lab activity. For one seminar style course, Labster virtual labs were the only lab-type learning activity and provided a unique opportunity for the students to link theory and lab experiments. In another course, the instructors used two virtual labs as self-directed complementary learning activity so that students could deepen their understanding of complex course topics. Students later shared their experience with Labster virtual labs in a class meeting.

Labster virtual labs that were integrated into the participating courses were included in mid term and/or final exam questions. An interesting observation, though not verified by data, was that conducting or repeating the Labster virtual labs closer to exam time may lead to better performance on virtual lab related questions.

Sustainability. A major concern about the sustainability of Labster virtual lab curriculum integration was the cost associated with the labs. Students were in favour of no-cost access to the Labster virtual labs used in the selected courses in the 2017-2018 academic year. Moving forward, the instructors wondered if it would be possible to select the Labster virtual labs that they need for their courses, instead of the current model wherein all labs in Labster catalogue need to be purchased as a whole collection. Available support and instructor time were not prohibiting factors in integrating Labster virtual labs into the courses.

Knowledge Dissemination

The collaborating instructors and OLS staff members have already shared interim insights form this work in the following events:

In the coming months, the results of the evaluation project and findings of the study conducted by the collaborating instructors will be shared in at additional upcoming events.

Conclusion

The collaborating instructors provided positive feedback about the enriched experience that Labster virtual labs created for learners. They also reflected an overall positive response from their students. The instructors highlighted the main advantages of integrating Labster virtual labs in their courses as: An opportunity to extend existing teaching, providing students with lab experience where on-site labs are not available, and as a complementary self-directed learning resource for the students to deepen their understanding of the relation between theory and lab experiment.

The administrative overhead for the process of design and implementation of the Labster virtual labs integration in biology related courses at U of T entailed was quite manageable at the institution level. The instructors did not report technical issues hindering the curriculum design or implementation process and the seamless integration into the learning management system was seen as highly valuable.

With regards to the sustainability of Labster virtual labs, the instructors’ primary concern related to pricing model, which is currently based on access to the full Labster catalogue. The instructors noted that specific labs were relevant to their course learning outcomes, but the cost of licence to full catalogue per student may be prohibitive. There is also interest in customizable and adaptable lab materials that can be more closely aligned with specific course learning outcomes and learners’ needs.

 

Reference

De Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305-308.

 

For more information visit our Active Learning Opportunities Through Virtual Lab Curricular Integration Project site or contact online.learning@utoronto.ca.

Active Learning Opportunities Through Virtual Lab Curricular Integration Final Report by Online Learning Strategies Portfolio is licensed under a Creative Commons Attribution 4.0 International License.

This project was supported by an Innovation Grant from eCampusOntario