Why did we start doing online team-based learning workshops?
A few years ago, some of the professors using InteDashboard for Team-based Learning (“TBL”) started doing so in online classes. We started to realize that one of the benefits of using technology to empower key parts of the TBL process was that it was entirely digitized and could be replicated in an online modality. Around the same time, the Team-based Learning Collaborative (“TBLC”) formed a community of practice devoted to online TBL.
In 2017, I tried teaching online with TBL in two workshops for incoming Duke business school students to see if it could be done logistically. If teaching TBL face-to-face requires a lot of administrative process coordination, online TBL takes this challenge to the next level. That said despite a clunky experience and lots of mistakes, the results were positive. I gave a poster on this experiment at the eLearning Forum Asia and presented a poster at the TBLC conference.
Last year we started kicking around the idea of running TBL professional development workshops online. As part of what I like to refer to as our “pedagogy marketing” we have done over 20 TBL train-the-trainer workshops at various conferences, training organizations, schools and universities around the world. We always wondered about how to scale this up and reach more educators. In collaboration with some of our customers (Dr. Julie Estis from the University of South Alabama and Dr. Jennifer Styron from Eastern Virginia Medical School), we launched our first online TBL workshop in 2018. The response was positive, so we started doing online TBL workshops every month and started refining both the process and technology involved.
Why online team-based learning?
There are several reasons to consider using TBL online. Here is what we have heard:
• Face-to-face TBL educators shifting online: Some of the early folks using online TBL were nursing schools particularly for their graduate nursing programs. Many of their students were practising nurses so they were unable to meet on a consistent basis. Some of these professors were using TBL in person and then brought that same methodology online.
• In-depth, blended global programs: There are some blended learning global programs that want to provide an engaging online learning experience. For these educators, a “flat” online class with just video lectures or webinars would not be enough. They look to online TBL to provide engagement among learners at a deeper level. This may be particularly relevant for experienced learners and working adults that can learn a lot from each other.
• Prepare students before they reach campus: We have some institutions that wanted to better prepare incoming students before they arrive to reduce the stress of adapting to intensive programs such as medical school. Others have wanted to provide career skills training, so students can hit the ground running particularly in short master’s programs.
• Learn more about TBL in general: We have seen some educators wanting to learn more about TBL but have difficulty obtaining the time or resources to attend an in-person event about TBL.
• Learn more about technology-enabled TBL: This one surprised me the most. We, several educators, participate in a workshop about online TBL to see if they wanted to use our InteDashboard software in their face-to-face classroom.
Online team-based learning workshop challenges
We have found that ad hoc professional development workshops delivered with team-based learning (“TBL”) can be effective and engaging learning experiences. It has taken some time and a few technology modifications, but we have figured out how to do so relatively smoothly in face-to-face conferences, workshops and even in an admissions session for a business school. However, the online workshop presented some unique challenges compared to other situations:
• Geographic differences: Participants are typically from at least four continents, sometimes five.
• Language differences: Some participants have different native languages; even the native English speakers have different accents between Americans, Australians, British, Indians and Singaporeans.
• No show rates: We have made these TBL workshops free. However, there has been a high no show rate of up to 50% in some cases.
• Latecomers: Sometimes participants show up late which means they may miss key instructions and team assignments.
• TBL familiarity: Some participants have been using TBL for years and others are brand new.
• Online familiarity: Some participants have been teaching online since nearly the beginning of online teaching and others are completely new to the modality.
• Technical savviness: Serving educators and professors means there is a mix of disciplines and experience from computer science to liberal arts and a wide range of technical comfort, but most are typically digital immigrants or even digital refugees.
• Equipment and device variations: Some people will be on a desktop in the office with good video and audio while others may be on an iPad or cell phone with less effective screen sharing capabilities.
• Environment variations: For some, this is a 6 am event taken at home, others are in the office, and others are on the couch at the end of the day with dogs barking in the background.
• One-off workshop: Doing ad hoc events can be trickier as most people are new to the process and technologies employed. If this were a regular class that met each week, it would get much easier after the first session.
Online team-based learning workshop objectives
We had three learning objectives which are described below.
At the end of this session you should be able to:
1. Define online team-based learning (“TBL”)
2. Experience TBL as a student in an online synchronous modality
3. Describe how to implement TBL in an online modality
Before the workshop
Team-based learning, like most online learning, requires a fair amount of educator preparation to design and scaffold the learning experience. Here’s what we did before the workshop:
Prework
Like most blended team-based learning workshops we provided pre-work for participants to review before the workshop. Two of the pre-work items were based on prior presentations I had given so there was not much incremental effort. The learner guide took a little more effort this time as we wanted to provide more preparation and scaffolding for learners on what to expect during the workshop.
1. Required Prework: Online Team-Based Learning Best Practice Development was a presentation I gave at the Transform MedEd conference hosted by the Lee Kong Chian School of Medicine, NTU Singapore and the School of Medicine, Imperial College London. It is available here.
2. Optional Prework: Team-Based Learning (“TBL”) Overview was a presentation I gave to Embry-Riddle Aeronautical University faculty. This was for those new to TBL. It is available here.
3. Learner Guide: Implementing TBL in an Online Environment was a presentation created for this workshop to prepare learners by providing information on what to expect during the workshop. Preparation is an important element of any online learning. It is available here. This presentation covered:
• Web conference access information
• Chat introduction instructions
• Agenda with timing
• Acknowledgements
• Objectives
• One-page overview of team-based learning
• Zoom breakroom information (Zoom was the web conference tool we used)
• How to ask for help
• Login instructions for InteDashboard (InteDashboard was the software tool we build and used to empower team-based learning with technology)
• Individual readiness assurance test (“IRAT”) instructions
• Team readiness assurance test (“TRAT”) instructions
• 4S application exercise instructions for multiple choice
• 4S application exercise instructions for free response
Drafting individual readiness assurance test (“IRAT”) questions
For this workshop, we prepared eight multiple-choice questions (“MCQ”) for the RAT. A general rule of thumb for in-person TBL is one minute per question plus five minutes. Our estimate was 13 minutes if this were a face-to-face workshop, so we increased this to 20 minutes to account for greater login time and transition to the web conference virtual team rooms.
We also made the decision to use standard MCQ selection rather than confidence-based testing which is sometimes referred to as point spreading. We felt that it would be better to keep things simple than to introduce a new form of MCQ to learners.
Drafting (or copying) team readiness assurance test (“TRAT”) questions
We used the same questions for the IRAT and TRAT which is very common in TBL, so we did not have to create additional questions.
4S application exercises questions
We created three application exercises for this workshop. We chose two free-response applications and one multiple-choice application. Since a benefit of using TBL to conduct a workshop about TBL is that learners experience the methodology as a student, we thought this would be a good way to have them experience both types of applications and observe how the ensuing discussion might be facilitated. We allocated 30 minutes based on a very simple estimate of 10 minutes per application. The applications we selected were:
1. Identify at least three benefits of using TBL online (free response).
Note: We usually ask teams to “rank order the top three” as opposed to “identify at least three” in our face-to-face workshops. The ranking is a higher-order task that tends to challenge the teams more and take a little longer. We might try ranking in our next workshop. Ranking can also be easier to facilitate because an educator or trainer can more easily compare teams’ top choice versus a list of choices and ask teams to explain their rationale.
2. What is the biggest challenge you would face in transitioning from a face-to-face TBL class to an online TBL class? (multiple choice).
A. Academic integrity
B. Attitudes of faculty
C. Online facilitation skills
D. Process coordination
E. Technology coordination
F. Other (please note in the comments)
Note: We included five distractors as well as a free-response “write-in” option. We asked teams to select a single best answer but may consider allowing teams to select multiple answers in the future.
3. Identify at least three changes you would make when transitioning from a face-to-face TBL class to an online TBL class (free response).
Technical setup
We had two technologies we needed to set up: a web conference tool and the team-based learning tool.
1. Web conference technology
We used the Zoom web conference tool to conduct our plenary sessions and team breakout session. We chose Zoom because we were able to do screen sharing relatively easily and Zoom has the ability to do team breakout sessions in virtual rooms. We had to make sure that our Zoom account had the breakout rooms feature activated.
2. Team-based learning platform technology
We used InteDashboard which is a team-based learning software tool to administer the Individual Readiness Assurance Test, the Team Readiness Assurance Test with Immediate Feedback and the Application Exercises. Our setup involved setting up a class for the workshop, enrolling students and setting up the IRAT, TRAT and application exercises.
3. Enrolling students
To enrol students we used InteDashboard’s generic users feature. This feature was developed specifically for one-off workshops and presentations where tracking specific student names and emails were not required or desirable. These generic codes are:
• Include a name plus a team number plus a student number
• Formula: CourseCode-Team Number “s” “student number”
• Example: TESTCLASS-1s2 would be team 1, student 2
• Example: TESTCLASS-2s3 would be team 2, student 3
• The login and passwords are identical
We have found this to be a relatively effective way to enrol students in ad hoc TBL workshops.
4. Adding questions
To add questions we used InteDashboard’s import feature. This feature allowed us to import questions by copying and pasting from an Excel spreadsheet. I find keeping a question bank in an Excel spreadsheet useful because I can see all my questions on the same screen at once and easily move them around in order or change distractors.
Team formation and participant survey
Team formation is typically an important element of TBL. Teams in TBL classrooms are almost always created by instructors and created with the intention of creating a diverse mix of team members.
Previously we have tried to pre-form teams. We did this by sending a participant survey in advance of the workshop and used the results of the survey to form diverse teams. What we found for ad hoc workshops was that the no-show rate can be quite high – as much as 50% on some occasions. This resulted in some teams that were full and others that may only have one person. To further complicate things, some participants had a “screen name” on Zoom that was different from the name they registered with, which made assigning breakout rooms challenging. This combined with latecomers made some of the logistics management difficult.
Instead, we have switched to forming teams on the spot as they enter the Zoom web conference. Consistent with online learning best practices, we target smaller teams of 3-4 (versus 5-7 for a typical face-to-face TBL classroom).
During the workshop
Introductions (minus 15 minutes)
We “opened the doors” of the workshop 15 minutes before the initial start time. As participants joined the web conference we asked them to type a chat introduction. This helped with one principle of online learning which was getting to know each other. It also has the practical effect of getting participants to practice accessing and using the chat function on their web conference software. As learners entered the web conference we started assigning them to teams without a set methodology and without communicating which team they were on.
Overview (10 minutes)
We started the workshop about five minutes after the scheduled start. I gave a short overview of what learners should expect using the learner guide. After that, we assigned participants to teams by using the team breakout room feature in Zoom. Once we activated the breakout rooms, participants were off in their virtual team rooms.
Online Individual Readiness Assurance Test (“IRAT”) (20 minutes)
Once in the team rooms, one of my colleagues served as a team host in each room and walked participants through a series of introductions, assignment of login details and helped participants to log in to InteDashboard to take their individual quiz known as an IRAT. While participants were completing the IRAT, I was using a real-time dashboard item analysis to track progress. In this workshop, it looked like I may need to be ready to clarify questions 1, 3 and 8.
Online Team Readiness Assurance Test (“TRAT”) with immediate feedback (15 minutes scheduled + 3 extra minutes = 18 minutes)
After the IRAT, the TRAT was started. Teams began by selecting a “team reporter” who would lodge a response on InteDashboard on behalf of their team. Teams would then discuss each question. When they agreed upon a response, the team reporter would submit it to InteDashboard and the team would get immediate feedback if they were correct or not. If correct, they would see a green check and go onto the next question with full points. If not, they would discuss and resubmit for partial credit (which declines) until they get the correct answer.
Like the IRAT, I monitored progress with my real-time dashboard. After the TRAT, it looked like question three might be the biggest trouble spot with the teams that got it incorrect both thinking B was the correct answer initially.
The TRAT took a little longer than the expected 15 minutes, so we gave the teams four extra minutes, but they only need three of them. In total it took 18 minutes. In the future, I would still leave the official time at 15 minutes to keep teams focused on finishing. I feel like if I gave them 18 minutes, there is always one time that would still need more time.
Not surprisingly the teams did better on the TRAT than the IRAT. They scored 81% as a team versus only 48% as individuals.
Clarifications
After the TRAT, we asked teams to submit any clarification requests through InteDashboard’s “request clarification” feature. After teams had a chance to submit clarifications, we brought everyone back to the plenary room and I provided a series of clarifications.
After finishing the theory part of the TBL process in the first hour, we moved on to the application phase with a series of three team application exercises. We had planned for 30 minutes. However, the teams all finished in 20 minutes, so we moved on to a facilitated discussion about the results. The facilitated discussion took about 25 minutes which allowed us to end the session 15 minutes early after 1 hour and 45 minutes of total time.
1. Free response application: facilitator description
The first application was facilitated by just highlighting some key points made by each of the team. Consistent with the TBL methodology, all the results were released to teams simultaneously. This application asks educators to identify some of the key benefits of online TBL, which were answered as follows:
· Global courses, flexibility in location
· Collaboration in online courses
· Engagement
· Real-time feedback
2. Multiple choice application: teams explain
Our second application was a multiple-choice application that required teams to choose what they thought the biggest challenge would be in implementing TBL online. We had someone from each team describe why their team reached that decision.
3. Free response application: E-gallery walk
Our third application was a question about what changes they would make if they were transitioning an existing TBL class to an online TBL class. At this point, I knew that we were comfortable with time and may even have some extra time, so I decided to run a gallery walk. In this technique, teams take a few minutes and review the responses and then vote for the team with the best response other than their own. Teams learn by seeing how others tried to solve a similar problem as well as by having to critically evaluate what makes a good answer in deciding which team to vote for.
Evaluation
Immediately after the workshop, we sent participants a feedback survey. Overall the responses were quite positive. Participants especially enjoyed discussions with other TBL practitioners during the team-based activities, indicating an average satisfaction score of 88%.
Acknowledgements
The author would like to acknowledge the members of the Team-Based Learning Collaborative Online Community of Practice and the 16 other co-authors of Off-to-On: Best Practices for Online Team-Based Learning whitepaper. I have learnt enormously from their deep insights and experience with online TBL.
The author would also like to acknowledge Dr. Julie Estis of the University of South Alabama and Dr. Jennifer Styron of Eastern Virginia Medical School who bravely hosted and facilitated our first online TBL workshop in 2018 and contributed their expertise in online TBL.
The author would like to acknowledge the contribution of my colleagues at CognaLearn who patiently support our efforts to incorporate features into InteDashboard to support online TBL and helped to beta test its use with live TBL workshops.
Disclosure
The author is the Commercial Founder of and has a financial interest in CognaLearn. CognaLearn is the company that developed InteDashboard, which is TBL software developed in collaboration with Duke-National University of Singapore Medical School. InteDashboard™ is a TBL software platform that was used and described in this post.
