The team processes have been supported by library experts as well as educational technology professionals. We have learned that it “takes a village” to build a MOOC, which in turn requires that instructors make their design goals and plans visible and open to input from their colleagues. Anecdotal reports suggest the instructors appreciate the opportunity to tackle the design phase by participating in a guided workshop process. Elements of the Course Re/Design Institute offered annually by the Centre for Teaching Support & Innovation have been adapted for the MOOC context. Our experience has been that tried and true methods can effectively inform exploration in the new scaled environment. Introduction of foundational concepts such as “backward design” (Wiggins, 1998) and “significant learning” (Fink, 2003) has proved a successful strategy for pushing the boundaries beyond video presentation elements to effectively engage learners. The MOOC design process provides an entry point to applying principles of effective course design in a highly exploratory context, resulting in the twin benefits of increased awareness of course design principles, plus innovation in the online learning environment at scale.

Many of the MOOC instructors are award-winning faculty members who have considerable influence among their peers, both within their academic programs and across divisions. The spotlight on the MOOC initiative in combination with our other online learning projects drew considerable interest from many advisory committees and governance bodies within the institution. The internal audiences for presentations by these outstanding instructors included the Academic Board – Committee on Academic Policy and Programs; Teaching, Learning and Technology Advisory Committee; and the Open UToronto Advisory Committee. The formation of a network of instructors, supported by an informal community of practice, represents a significant gain in expertise and capacity within the sphere of online instruction at the University of Toronto.

Priority 3: Engage in research and evaluation activities

Scholarly research activity related to the learner characteristics, participation and outcomes accelerated during our second year of MOOC exploration, as several projects proposals received external funding. The Introduction to Psychology MOOC, as well as Making Sense of Data had already benefited from grant resourcing through the Bill & Melinda Gates Foundation that included a research component. In the fall of 2013, three additional Gates research projects were funded as part of the MOOC Research Initiative, coordinated by Athabasca University:

·      Beyond and Between “Traditional” MOOCs: Agile and Just-in-Time Learning
Jennifer Campbell and Alison Gibbs

·      Secondary School Students and MOOC’s: A Comparison between Independent MOOC Participation and Blended Learning
Dilip Soman and Rosemary Evans (UTS)

·      Hatch, match, and dispatch: Examining the relationship between student intent, expectations, behaviours and outcomes in six Coursera MOOCs at the University of Toronto
Carol Rolheiser, Chris Teplovs, Stian Haklev, Hedieh Najafin and Bodong Chen

These and other research initiatives were supported through development of an institutional MOOC data repository service. As part of the Open UToronto initiative, in collaboration with the Academic and Collaborative Technology portfolio, infrastructure was developed to support data analytics, research and evaluation projects using MOOC databases and clicklog archives. The service includes provision of cleaned and formatted data samples to researchers, while ensuring ethics protocols are addressed. Securelong-term storage is available for the purpose of research and evaluation.

As a result of the increased levels of research activity that occurred during the 2013-2014 academic year, community interest converged in a coordinated event to showcase this scholarly work. In April of 2014, Online Learning Strategies portfolio and the Centre for Teaching Support & Innovation co-hosted a MOOC Research Symposium featuring presentations of findings by five research teams. The event drew approximately 70 attendees, with a high level of discussion and engagement occurring throughout the day.

In addition to the Gates-funded research, our MOOC instructors and their colleagues have published a number of other papers. A collection of these MOOC research publications is posted on the Open UToronto web site, including two papers examining the use of MOOC materials in the inverted classroom model. Of particular interest was the finding from the Computer Science discipline indicating that learning as measured by final exam performance increases significantly when this format is used (Horton, Craig, Campbell, Gries, & Zingaro, 2014). A full list of publications is provided in Appendix 5.

Resourcing of institutional research support, including data storage and curation has been essential to our scholarly achievements related to use of the MOOC model. Several additional research projects are currently in progress, on topics including:

·      Evidence of development of higher order statistical skills in statistical literacy, reasoning, and thinking using broadly accepted evaluation frameworks, specific to the discipline area;

·      Impact of MOOC experience on perceptions and sense of efficacy in the sphere of library advocacy

·      Understanding the experiences and impact of instructors participating in the development and running of MOOCs; and

·      Use of analytics to better understand the relationship between student intent, expectations, behaviours and outcomes. 

Looking ahead to the Coming Year

As we look ahead to the coming year, we see the opportunity for continued exploration of a number of facets of online learning through the Open UToronto MOOC initiative. A key area where the MOOC model provides a catalyst for reflection and innovation is the use and re-use of digital content, including:

o Creation of on-demand open course content resources for flexible access;

o Re-use of MOOC content in differentiated fully online degree-creditcourses (enhanced through additional interactivity and instructor contact)

o Continuing exploration of a range of models beyond the “traditional” MOOCs, including flipped classrooms, outreach initiatives and research.

In order to support UofT’s broader strategic targets, several target actionables have been identified for the coming year:

o Piloting of University of Toronto alumni cohort MOOCs in collaboration with Alumni Relations.

o Using MOOC content to support additional flipped classroom active learning opportunities in gateway courses.

o Partnering with peer institutions in design and development efforts, either through community of practice activities or complementary MOOC offerings.

o Exploring the possibility of a revenue stream through non-degree program verified certificates.

In summary, we anticipate further extension of the types and uses of MOOCs through our continued partnership with Coursera and EdX in the 2014-2015 academic year. Our activities will include development of new MOOCs, as well as exploration of the potential for re-use of existing resources. The University of Toronto will continue to take a leadership role in the emerging research on scalable online learning, re-use of content in a range of instructional contexts, and application of learning analytics to improve the effectiveness of pedagogical design.

For more information on Open UToronto MOOC initiatives contact Laurie Harrison. 

References:

Stewart, B., 2013. What’s Next? After the MOOC Hype. Presentation at MOOC Research Initative Conference, December 2013. Retrieved from:http://www.slideshare.net/bonstewart/mooc- research-conference

Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass.

Horton, D., Craig, M., Campbell, J., Gries, P., & Zingaro, D. (2014). Comparing outcomes in inverted and traditional CS1 (pp. 261–266). ACM Press.

Pappano, L. (2012). The Year of the MOOC. The New York Times. Retrieved fromhttp://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html?pagewanted=al

Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco, CA: Jossey-Bass.