Learning & Education Advancement Fund+ Project Theme:
Generative Artificial Intelligence in Teaching and Learning (LEAF+ Gen AI)
Program Overview
The LEAF+ Gen AI program, funded by the Vice-Provost Innovations in Undergraduate Education (OVPIUE), addressed the theme of generative AI in teaching and learning. Given the potential impacts, opportunities and challenges of this emergent technology for the ways our course instructors and students engage in learning, the OVPIUE was interested in projects to help inform the community’s use in pedagogical contexts.
Eleven successful project teams formed a cross-disciplinary network and met periodically to share insights as new approaches using or responding to AI in teaching and learning were prototyped. This early exploration served as a conduit for feedback to OVPIUE on instructors’ experiences piloting new strategies and methods. See:
LEAF+ Programming
The LEAF+AI instructor cohort engaged in a series of comprehensive interactions with the facilitation team spanning from May 2023 to January 2024. Three sessions took the form of inclusive workshops attended by the entire cohort. Additionally, during the summer of 2023, specialized Special Interest Group (SIG) sessions were conducted, allowing team members to selectively participate based on the relevance of topics to their respective projects.
The workshops for the entire cohort were strategically designed to address crucial aspects of program onboarding, facilitate the exchange of methodologies and resources, and explore opportunities, challenges, and lessons learned in the domain of generative artificial intelligence. Concurrently, the SIG sessions delved into nuanced areas of focus, namely digital literacy and ethics, language learning application development.
Instructors further benefitted from a continuum of support facilitated by the facilitation team. Each participant was afforded individualized check-ins with the facilitation team, ensuring a personalized and responsive support framework throughout the entire program duration. The progression of the workshops is captured in the table below:
Table 1: Events delivered during the LEAF+ project.
| Events | Date | Objective |
| Workshop 1 | May 02, 2023 | LEAF+Gen AI Orientation |
SIG Groups:
|
Summer, 2023 | Focused discussions, troubleshooting and sharing based on themes gleaned from project types. |
| Workshop 2 | November 13, 2023 | Roundtable sharing: Opportunities and challenges. |
| Workshop 3 |
January 29, 2024 | Reflection on experience and lessons learned. |
Project Insights
All project leads were required to complete a detailed report regarding various aspects of their experience in implementing projects. Drawing from the feedback provided, common themes and insights related to integration of generative AI in the teaching and learning context are summarized below. These insights will be valuable in ongoing faculty development planning to support instructors and learning technology professionals across the university community.
Transformative nature of generative AI
Instructors highlighted the transformative nature of AI influencing their teaching and research through the following:
- Affordance for personalized learning experiences
- Agility needed to navigate rapid change as generative AI is integrated into industry tools, such as media production, writing, communication and design
- Interdisciplinary nature of generative AI opening critical and creative possibilities
- Value of domain-specific knowledge and expertise in shaping effective AI integration
- Importance of gathering student feedback on their experiences
Challenges in Teaching Students
The project teams also garnered insights regarding challenges in teaching students in a generative AI enabled environment:
- Student understanding of the underlying theory to inform their projects and critically assess AI outputs
- Support in citing sources and credit work that is not their own
- Reliance on Gen AI to support learning reducing the value of pedagogical experience
- Staying current given rapid evolution of the gen AI field
- Anxiety about sustainability and ethics, concerns about carbon footprint
- Content rights issues – copyright, intellectual property
- Erosion of language learning
- Increasing the distance between expert and novice language users
Enabling U of T instructor use generative AI
When asked for recommendations on resources or strategies to support instructors in use of Generative AI in teaching practice, the following suggestions were offered by project leads:
- University-wide platform for generative AI tools to handle tool subscriptions, reduce cost concerns, and create fairness in student access
- Resources including examples of successful AI integration, assignments designed for AI, updatable archives, toolkits, and best practices
- Training for instructors on critical understanding, practical skills and prompt engineering for effective AI use, and possibly discipline-specific guidance
- Workshops, brainstorming sessions, and ongoing discussions on both the opportunities and challenges
Program Outcomes
Read full details about the project outcomes below:
AI for Image Research in Art and Architecture
Deployment of LLM-based Personal Coding Assistants that Balance Helpfulness and Directness
Designing Differentiated Instruction in the Foreign Language Classroom Using ChatGPT
Exploring Ethical and Creative Uses of Generative AI to Support Equity-Deserving Students
Instructor-AI Collaborative Content Generation Project
Learning to Teach and Teaching to Learn in the Generative AI Landscape
Strengthening Co-op Student’s Communication Skills for Job Search and Workplace Success
Understanding the Limits of AI-Based Image Generators with DALL·E 2 and Midjourney
AI for Image Research in Art and Architecture
Project Lead: Cathryn Copper, University of Toronto Libraries, Eberhard Zeidler Library
The project resulted in an online guide supporting a first-year computation and design course assignment, now publicly available as a library research guide for generative AI in design. The guide covers key aspects, including model functioning, concept exploration, prompt engineering, critical evaluation, and citation of generative AI. An accompanying digital exhibition, “a thousand words,” showcased AI-generated images from student projects, fostering a guessing game that highlighted nuances and limitations in AI image generation. The dissemination of findings through a conference paper and presentation emphasized increased student confidence in AI image generators, enhanced critical thinking, and advocated for AI’s co-creator role in architecture, stressing collaboration between faculty and librarians in AI integration.
Deployment of LLM-based Personal Coding Assistants that Balance Helpfulness and Directness
Project Lead: Tovi Grossman, Faculty of Arts & Science, Computer Science
CodeAid, an LLM-powered programming assistant, was introduced with five key features to support programming assignments and reinforce concepts. In a deployment with 700 students, CodeAid garnered 372 users asking 8132 questions. Notably, women utilized CodeAid more frequently than men. Students posed inquiries primarily for programming assistance, code debugging, code writing, and code explanation. General Question was deemed the most useful feature, while Help Fix Code received the lowest rating. Overall, CodeAid demonstrated 79% correctness and 86% helpfulness, notably improving after transitioning to GPT-3.5 from OpenAI Codex. The system successfully avoided displaying direct code solutions, with 43% providing natural language responses and 24% including pseudo-code. Student feedback highlighted appreciation for CodeAid’s 24/7 availability, contextual assistance, and ease of formulating questions, but concerns included dependency, trust, and reliability. Survey results revealed varying usage patterns, with some students exclusively using CodeAid and others combining it with ChatGPT. Educators praised CodeAid’s pedagogical approach but expressed concerns about incorrect responses and student misuse, emphasizing the need for customization and a monitoring dashboard in future AI tools.
Designing Differentiated Instruction in the Foreign Language Classroom Using ChatGPT
Project Lead: Chiu-Hung Chen, University of Toronto Mississauga, Language Studies
The integration of this project into second and third-year Chinese language courses aimed to cater to diverse motivations and proficiency levels among students. Tasked with creating self-directed language and culture projects, students utilized ChatGPT with guidance on effective use. This initiative empowered them to choose a personal interest, develop a study plan, monitor progress, and present at semester-end. Outcomes included: 1) Development and testing of prompts for initial ChatGPT interactions, 2) Evaluation of student prompts with constructive feedback, and 3) Tailored prompts and feedback accommodating diverse language proficiency levels in a varied language classroom.
Exploring Affordances of Generative AI Tools for Pre-service (language) Teacher Education: From lesson development to assessment
Project Lead: Ji-young Shin, University of Toronto Mississauga, Language Studies
In this project, the focus was on exploring the affordances of generative AI for language learning and language teaching, while another four areas will be investigated in the upcoming 2024 winter term. Qualitative analyses of student reflections and performances in a second language instruction course revealed five identified affordances of generative AI, specifically ChatGPT: as an interactive conversation partner, language input source, facilitator, teaching assistant, and collaborator. Additionally, quantitative analyses of user experience surveys highlighted positive perceptions, particularly regarding functionality, with a notable emphasis on interactive and immediate feedback.
Exploring Ethical and Creative Uses of Generative AI to Support Equity-Deserving Students
Project Lead: Elaine Khoo, University of Toronto Scarborough, Centre for Teaching and Learning
Contexia, a Chrome extension developed as a digital tool, offers students immediate assistance in comprehending challenging sections of text by allowing them to highlight specific words or phrases, triggering Contexia to provide simplified explanations within its text window. This tool has proven beneficial for academic reading, eliminating the need to leave the reading page or consult external translation resources. Meanwhile, the exploration of critical AI literacy has resulted in guidelines, sample assignments, and pedagogical designs to empower students to make informed decisions about using Generative AI (GenAI). Additionally, efforts to engage students as junior scholars investigating GenAI topics have yielded shared samples of work, guidelines for ethical GenAI tool usage, and suggestions for instructors to explore creative and ethical approaches with their students. These outcomes collectively contribute to fostering a deeper understanding and responsible use of AI in higher education contexts.
Project Leads: Malayna Bernstein, Claire Battershill, Seamus Ross, Matt Ratto; Faculty of Information
Our overarching project goal was to explore the potential of generative AI in fostering new creative and analytic opportunities for educators. Despite recognizing the challenges posed by systems like ChatGPT, our aim was to collectively envision new possibilities. To support this, we introduced a framework focusing on three modes of engagement—Writing With AI, Writing For AI, and Writing About AI—emphasizing the connection between text-based generative AI systems and writing. The engagement of 160 participants from various disciplines in three workshops, followed by surveys, revealed that faculty members and instructors felt empowered, gaining enhanced understanding and planning to incorporate new teaching exercises and critical discussions with students.
Instructor-AI Collaborative Content Generation Project
Project Leads: Joseph Jay Williams, Nathan Laundry; Faculty of Arts & Science, Computer Science
We deployed the JoltEd Chrome extension in the CSC428 course on human-computer interaction. Students engaged in a qualtrics survey to reformulate explanations of how user interviews are applied in HCI research. Using JoltEd, they first utilized a simplify prompt unrelated to their area of interest and then created a personalized example linked to their field. Subsequently, students provided both quantitative and qualitative reflections on the tool’s impact on their perceptions of concept relevance and value. Informal qualitative analysis indicated that many students perceived the personalized explanation as enhancing the concept’s relevance and value to them. While the quantitative analysis, with limitations in statistical power, revealed an increase in perceived value for personalized examples compared to the simplify condition, no significant change was observed in relevance perceptions. The overarching research question addressed the use of LLMs to improve student perceptions of concept relevance and value through personalized explanations and examples.
Intelligent Support for Enhancing Problem Solving, Learning, and Metacognitive Skills in CS Courses: A Novel Tool Combining Large Language Models and Reinforcement Learning
Project Lead: Michael Liut, University of Toronto Mississauga, Mathematical and Computation Sciences
The project successfully achieved two main objectives. Firstly, it enhanced student support in problem-solving through the introduction of a novel approach leveraging large language models, thereby personalizing the learning experience. Secondly, a software tool was developed to offer intelligent and personalized assistance to students across various disciplines during problem-solving activities. The versatility of this tool positions it as a valuable asset applicable to the learning process in any course. These accomplishments signify a significant advancement in tailoring educational support through innovative technologies.
Learning to Teach and Teaching to Learn in the Generative AI Landscape
Project Leads: Dan Guadagnolo, Sarah Cherki El-Idrissi, Kate Maddalena, Michael Nixon, Steve Szigeti; University of Toronto Mississauga, Institute of Communication, Culture, Information and Technology
The project successfully achieved several milestones, including the creation of networked modules on Generative AI. Additionally, a significant overhaul was implemented in the 100-level curricular experience, impacting approximately one-third of UTM’s annual matriculating class. The project also produced valuable resources for instructors seeking to integrate awareness of generative AI tools into their teaching practices. Notably, the developed modules are designed to be sustainable and platform-agnostic, ensuring their effectiveness even with the emergence of new online services. These accomplishments collectively contribute to advancing awareness and understanding of Generative AI in the educational context.
Strengthening Co-op Student’s Communication Skills for Job Search and Workplace Success
Project Leads: Lynn Tucker, Susan Soikie, Cynthia Jairam-Persaud; University of Toronto Scarborough, Arts & Science Co-op
We sent two calls for participation with INStage to support practice and supplement traditional assessment. INStage allows students to conduct virtual simulations with a panel of avatars and receive immediate feedback on their communication performance. During the Summer term, calls for participation generated interest among students, but only a small percentage completed all required components. In the subsequent fall term, calls for participation, coupled with surveys in a co-op course, revealed that 35% of participants identifying as international students, students with disabilities, or both benefited from the platform. International students experienced a 33% increase in confidence in discussing work and volunteer experiences but showed no change in communicating academic achievements. Conversely, students with disabilities reported no change in confidence but expressed a desire for more opportunities to discuss disabilities during the recruiting process. Despite variations in comfort levels with AI, 80% of students found the simulation questions helpful and expressed a willingness to recommend the platform, with an increasing perception of AI responsiveness over time. A notable 5% preferred practicing with a real person instead of AI.
Understanding the Limits of AI-Based Image Generators with DALL·E 2 and Midjourney
Project Lead: Alec Jacobson & Chenxi Liu, Faculty of Arts & Science, Computer Science
We created a new assignment sequence for third-year undergraduate students in our Computer Graphics course on Generative AI. The success of text-to-image generation tools is due in part to their proprietarily large scale. Students learned about how these models work from our lecture and via firsthand experience by pressing these models in directions where they work best and then pressing them into directions that reveal their (possibly negative) biases. This project is poised for strong student impact by giving a curated experience to generative AI tools that are often out of reach (due to costs) and poorly understood (due to misleading hype in popular media). This way, students can form their own opinions, rather than simply being influenced by the news, and take more active roles in the ever-evolving AI landscape.
LEAF+ Facilitation Team
- Laurie Harrison (Director, Digital Learning Innovation, ITS)
- Will Heikoop (Coordinator, Digital Learning Innovation, ITS)
- Tegan Mannisto (Teaching Initiatives Coordinator, OVPIUE)
- Alex Olson (Senior Research Associate, Centre for Analytics & AI Engineering)
Contact digital.learning@utoronto.ca for more information.