WEBVTT

1
00:00:04.930 --> 00:00:29.540
BSPH CTL Teaching Toolkit: Okay, thank you all for joining us today for this second to last picnic Pd workshop session for this year. Today, we're going to cover the motivated learner providing assessment choice to activate student motivation. And I'll go ahead and pass it over to Lauren, Carrie and Dave. Love. Thank you.

2
00:00:30.140 --> 00:00:58.209
Lauren Dana: Thank you, Emily, so I just wanted to quickly introduce ourselves. So I'm Lauren, Dana. I'm an instructional designer at the center for teaching and learning. I'm joined by my colleague Carrie, who is also an instructional designer at the center for teaching and learning, and we're also lucky enough today to have Dave Love, who teaches the course, seafood and public health, global trade nutrition and the environment, and is based at the center for a livable future. So he's 1 of our wonderful faculty. So welcome, Dave.

3
00:00:59.070 --> 00:01:01.129
Dave Love: Yes, thank you. Nice to be here.

4
00:01:02.370 --> 00:01:12.579
Lauren Dana: Great. So at the end of this session, what we're hoping that you'll be able to do is understand how choice inspires student motivation through educational theory.

5
00:01:13.510 --> 00:01:18.910
Lauren Dana: examine different strategies on how to implement choice in and of assessments.

6
00:01:19.650 --> 00:01:25.560
Lauren Dana: and finally depending on time, create or revise an assessment that incorporates student choice.

7
00:01:27.090 --> 00:01:35.900
Lauren Dana: So before we jumped into the content of the presentation, I just want to start with a short reflection, and I wanted to offer you 3 reflection options.

8
00:01:36.050 --> 00:01:45.030
Lauren Dana: So option one. I want you to think of a time in college or graduate school when you were most engaged in an assignment. What made it meaningful for you.

9
00:01:45.180 --> 00:02:03.939
Lauren Dana: Option? 2. How have your own interests or backgrounds shaped your approach to learning or teaching or option? 3. If you're teaching currently, or have in the past. If someone asked your students what assignment or project best represents your course, what would they say? And why?

10
00:02:04.070 --> 00:02:11.030
Lauren Dana: So please choose the option that speaks to you. Just take about 2 min to quietly think about this, or jot down some notes.

11
00:02:11.190 --> 00:02:20.440
Lauren Dana: I do have some music, so when the music ends, that'll be the signal to wrap up. And hopefully, if you're comfortable, you can share your thoughts in the chat or just raise your hand to speak.

12
00:02:20.770 --> 00:02:24.009
Lauren Dana: I'm gonna start our music now.

13
00:02:44.740 --> 00:03:02.019
Lauren Dana: Okay, so I think that was a little shorter than 2 min. But okay, the music ran out. So would anyone like to share in the chat or raise their hand about any one of the options. Option 1, 2, or 3. So I'll just give everyone a moment, either type in the chat, or if there's any

14
00:03:02.220 --> 00:03:03.420
Lauren Dana: hands.

15
00:03:15.760 --> 00:03:44.039
Lauren Dana: and I can actually, I'll even start us off with some of you are typing. I know I was lucky enough to work actually with Renee. I just got my master's in performance, learning, and development, and I worked with Renee on a capstone project that was relevant to my current work. So, even though it was for school, I was able to take a look at some of the processes at Ctl, and how they could be improved, and so that for my option one, I just found it really meaningful that I could work on something outside

16
00:03:44.200 --> 00:03:45.120
Lauren Dana: of

17
00:03:45.460 --> 00:03:51.909
Lauren Dana: out with my work outside. I could use my actual work to work on my course outside, which was really helpful.

18
00:03:53.220 --> 00:04:00.959
Lauren Dana: Looks like Bsph toolkit. Emily has a memorable semi which you have to choose the context where to apply a structure to.

19
00:04:01.700 --> 00:04:09.260
Lauren Dana: Oh, Dave, that's great using real world data sets for projects. Right? So you're using real data, even if it's a theoretical or hypothetical project.

20
00:04:09.450 --> 00:04:10.400
Lauren Dana: Great.

21
00:04:13.370 --> 00:04:16.070
Lauren Dana: Okay? Anything else anyone else want to share. Okay?

22
00:04:16.500 --> 00:04:27.649
Lauren Dana: And Rikito, great asset, involve problem solving and the topic relevant to the subject. Excellent. Yeah. So I think it really helps when the topic is relevant to you or to your studies.

23
00:04:30.200 --> 00:04:31.479
Lauren Dana: Great. So we're gonna

24
00:04:32.580 --> 00:04:41.589
Lauren Dana: forward here. So I'm going to get started talking a little bit about the Bsph student self-determination theory or Sdt and assessment choice.

25
00:04:43.210 --> 00:05:01.870
Lauren Dana: But as this series and workshop focuses on learner-centered teaching. I did just want to quickly share some learner centered data from Bsph students that may offer useful insights for those of you joining from other institutions. I highly encourage you to explore your own student data to inform your course design.

26
00:05:03.280 --> 00:05:13.920
Lauren Dana: As you can see, our Bsph student population spans a wide age range. This diversity means that your students will likely come to the classroom with varying backgrounds, goals, and interests.

27
00:05:14.400 --> 00:05:26.149
Lauren Dana: designing assignments that are motivating and relevant to such a broad demographic can be really challenging. But it also presents an opportunity to tap into your students unique strengths and experiences.

28
00:05:26.340 --> 00:05:42.890
Lauren Dana: It's also important to note that our students are at different places within their career. So again, creating assignments that are universally applicable and motivating across the spectrum can be difficult, especially when you're trying to meet the needs of early career and seasoned professionals.

29
00:05:44.160 --> 00:05:57.029
Lauren Dana: Additionally, our data shows that over half of our students are enrolled part-time. While this does vary by program, it highlights a common reality among all adult learners. They are busy with many competing priorities.

30
00:05:57.200 --> 00:06:06.710
Lauren Dana: The data reminds us that we're working with a diverse and multifaceted student body and thoughtful assessment. Design is essential to support their success.

31
00:06:08.920 --> 00:06:24.550
Lauren Dana: While our student body is diverse, they do share common needs that when supported, can foster intrinsic motivation. One powerful framework that speaks to this is self-determination theory. Sdt, which is widely used in both psychology and education.

32
00:06:24.950 --> 00:06:35.080
Lauren Dana: Stt identifies 3 core psychological needs, that when met enhance motivation and engagement. So we have autonomy, relatedness and competence

33
00:06:35.310 --> 00:06:41.349
Lauren Dana: incorporating this framework into your teaching practice can significantly boost motivation and academic success.

34
00:06:41.680 --> 00:06:46.669
Lauren Dana: Let's briefly explore just each one of these needs. So we're going to start with autonomy.

35
00:06:46.970 --> 00:06:51.849
Lauren Dana: The supporting autonomy means giving students meaningful choices in their learning path

36
00:06:52.290 --> 00:06:57.029
Lauren Dana: could involve letting students set their own learning goals or select topics that interest them.

37
00:06:57.460 --> 00:07:06.730
Lauren Dana: But for the purpose of this workshop we're really focused specifically on providing choice in assessments, an effective and practical way to support autonomy in the classroom.

38
00:07:07.290 --> 00:07:19.599
Lauren Dana: Now, I'm going to talk a little bit about relatedness, and this refers to students ability to find personal meaning and connection in their coursework. And this was actually a common topic in the chat about assignments that were meaningful.

39
00:07:19.860 --> 00:07:28.820
Lauren Dana: Many of our students, like us, are already working professionals, so designing assignments that relate to their real world experiences can be incredibly motivating.

40
00:07:29.000 --> 00:07:36.000
Lauren Dana: and again offering assessment. Choice can help students connect their personal and professional lives to their academic work.

41
00:07:36.950 --> 00:07:38.829
Lauren Dana: And finally, we have competence.

42
00:07:39.050 --> 00:07:48.180
Lauren Dana: Students want to feel capable and empowered in their learning journey. They want to know that they're not just passive recipients of information, but active contributors.

43
00:07:48.470 --> 00:07:57.110
Lauren Dana: assignments, or assessments that allow students to build on existing strengths or explore new applicable skills, can help cultivate this sense of competence

44
00:07:57.630 --> 00:08:03.640
Lauren Dana: by designing assignments that incorporate choice. You can hopefully tap into all 3 of these motivational needs

45
00:08:03.810 --> 00:08:07.719
Lauren Dana: creating a more engaging, effective, and learner-centered classroom.

46
00:08:09.820 --> 00:08:21.620
Lauren Dana: So far we've been talking about assessment choice as a strategy to meet the needs of your learners. But I'd like to take a moment to define what we actually mean by assessment, choice and highlight. Some of its specific benefits

47
00:08:22.380 --> 00:08:37.270
Lauren Dana: 1st want to emphasize assessment choice is not about lowering standards. Instead, it's about maintaining the same learning objectives and assessment criteria while allowing variation in how students demonstrate their knowledge and skills.

48
00:08:37.650 --> 00:08:42.740
Lauren Dana: So, in other words, assessment choice still holds all students to the same high standards.

49
00:08:42.919 --> 00:08:48.869
Lauren Dana: But it acknowledges that students bring different contexts, experience and needs to the table.

50
00:08:49.000 --> 00:08:57.419
Lauren Dana: When we, as instructors, offer flexible options, we can create opportunities for personalized, meaningful learning that supports engagement.

51
00:08:57.540 --> 00:09:01.749
Lauren Dana: but doesn't compromise academic rigor which is important to all of us.

52
00:09:02.110 --> 00:09:06.999
Lauren Dana: This approach honors both the diversity of our learners and the integrity of our learning. Goals

53
00:09:08.610 --> 00:09:14.569
Lauren Dana: want to briefly review elements that should remain consistent across all assessment choices.

54
00:09:15.500 --> 00:09:26.700
Lauren Dana: The 1st is alignment with course, objectives. No matter the format. All assessment options should require students to demonstrate the key knowledge and skills outlined in your course, goals and objectives

55
00:09:28.960 --> 00:09:36.289
Lauren Dana: comparable level of effort. Each option should require a similar amount of time, effort, and cognitive demand to ensure fairness

56
00:09:37.590 --> 00:09:50.749
Lauren Dana: and finally, consistent assessment criteria. While the format may differ, the core expectations should remain the same ideally, you can use a shared rubric across all options with minor adjustments for format specific elements.

57
00:09:51.310 --> 00:09:58.039
Lauren Dana: just as a reminder. These are the 3 things that need to remain consistent and the same even within assessment choice.

58
00:09:58.480 --> 00:10:13.319
Lauren Dana: However, there's quite a variety of elements that can vary. My colleague Carrie, is going to explore this in more depth, and how to practically apply it to your course. But I just want to quickly share some of the categories of these elements that can vary just to get you started thinking

59
00:10:13.930 --> 00:10:21.120
Lauren Dana: we have timing and deadlines, waiting options, assessment, format collaboration levels.

60
00:10:21.350 --> 00:10:29.210
Lauren Dana: topic or content, flexibility and even feedback methods. So again, these are all elements that could vary within assessment choice.

61
00:10:31.490 --> 00:10:41.250
Lauren Dana: While we discuss the benefits of Std in your course, I do just want to quickly review again some of the very specific benefits for students when you incorporate assignment choice

62
00:10:41.360 --> 00:10:51.270
Lauren Dana: before I list my my benefits. If we could just take a moment in the chat, what are some benefits that you think students may receive when you provide assessment choice.

63
00:10:51.690 --> 00:10:53.440
Lauren Dana: I'll get there in just a moment.

64
00:11:01.040 --> 00:11:09.859
Lauren Dana: Great. You can work towards their strengths right? So students are able to to feel good and skilled, and work towards some of the things that they feel confident with excellent

65
00:11:17.470 --> 00:11:23.409
Lauren Dana: excellence. And yeah, it works with their current state. So their emotion. Right? So what their passions are, what they're feeling

66
00:11:24.180 --> 00:11:33.880
Lauren Dana: that's excellent can relate it to the real world context, which increases motivation. Right? If you're not just doing something in this hypothetical or theoretical world. You're actually using real world context.

67
00:11:35.020 --> 00:11:37.589
Lauren Dana: Emily, great increased agency and ownership.

68
00:11:39.220 --> 00:11:41.699
Lauren Dana: Nice acknowledges, partnerships or teamwork.

69
00:11:41.870 --> 00:11:48.269
Lauren Dana: Yeah, you could definitely build partners or teamwork depending on what the assignment entails. If you're allowed to have that choice. Excellent.

70
00:11:50.730 --> 00:12:09.540
Lauren Dana: great! These are all wonderful. I barely even need to go over my benefits, but I'll just quickly go through and summarize. So again, we have that increased autonomy that we talked about. And so students can take ownership and demonstrate their learning, and this has the benefit of building independence and boosting confidence.

71
00:12:10.020 --> 00:12:18.930
Lauren Dana: personal relevance, which you all already mentioned. Again, this just allows students to bring their own interests, passions, and cultural backgrounds into their work, making it meaningful.

72
00:12:19.280 --> 00:12:31.610
Lauren Dana: real world connections. Again, I think that was Beth in the chat by choosing formats or topics that relate to their own lives and careers. Students deepen their understanding and see the real world value of what they're learning.

73
00:12:32.730 --> 00:12:49.230
Lauren Dana: There's enhanced intrinsic motivation which a lot of you already identified. Right? So if students feel in control and connected to the material, they're more likely to engage thoughtfully and consistently and hopefully, we don't have that drop off at the end of the term, which you can see with motivation.

74
00:12:49.630 --> 00:12:52.969
Lauren Dana: and then finally, more in time invested in learning.

75
00:12:53.290 --> 00:13:00.710
Lauren Dana: Personalized assessments often lead to increased effort. Students are more willing to go the extra mile when the work actually resonates with them.

76
00:13:01.640 --> 00:13:11.860
Lauren Dana: So now we've reviewed some of the theory, some of the benefits. I'm going to hand it over to Carrie, who's going to actually talk about some practical applications, and how you can apply choice in your course.

77
00:13:13.080 --> 00:13:26.440
Keri McAvoy: Great thanks, Lauren, and thank you for that background in Sdp. And assessment choice. But, like Lauren said, now I'm going to take us into a dive of how we can provide assessment into our specific courses.

78
00:13:28.432 --> 00:13:41.209
Keri McAvoy: So just like Lauren mentioned earlier providing choice within assessments is important because it empowers students to demonstrate their understanding in ways that align with their strengths, interests, and learning styles.

79
00:13:41.420 --> 00:13:43.990
Keri McAvoy: This approach, fosters greater engagement.

80
00:13:44.190 --> 00:13:53.859
Keri McAvoy: supports diverse learners, and promotes a sense of ownership over the learning process which ultimately leads to more meaningful and authentic demonstrations of knowledge and skills.

81
00:13:54.300 --> 00:14:09.119
Keri McAvoy: Some different assessments that you're probably already using in your courses are things like quizzes, group projects, essays, discussion, forums and presentations. So how can we take these assessments that we're already using and provide choice within them.

82
00:14:09.320 --> 00:14:17.520
Keri McAvoy: You could start with topic selection, allowing students to select a topic of interest that's within the scope of the course, content.

83
00:14:18.460 --> 00:14:24.749
Keri McAvoy: giving question options, offering a list of questions or prompts, and students choose a set number to answer

84
00:14:25.250 --> 00:14:39.490
Keri McAvoy: flexible due dates is another option. It could look like giving a time window to complete the assignment. Instead of a fixed deadline, or letting students choose from a range of submission windows or offering something like a Grace period bank for extensions

85
00:14:40.380 --> 00:14:49.230
Keri McAvoy: you could give waiting options, which means allowing students to choose how much specific assessment excuse me how much a specific assessment counts towards their final grade.

86
00:14:50.010 --> 00:14:54.450
Keri McAvoy: Allow students to choose between collaborative versus individual work.

87
00:14:55.240 --> 00:15:04.059
Keri McAvoy: provide a choice of resources like readings, videos, or case studies. And let students choose what they use for an assignment

88
00:15:04.330 --> 00:15:11.340
Keri McAvoy: and providing multiple opportunities, allowing students to resubmit after having received feedback on a submitted assignment.

89
00:15:13.510 --> 00:15:25.150
Keri McAvoy: Let's review this example of a formative assessment where students are asked to respond to a prompt, related to course content, and then reply to a few other peers responses in a discussion forum.

90
00:15:25.530 --> 00:15:32.989
Keri McAvoy: The example, prompt states in your own words, explain one concept from this week's module that you found challenging.

91
00:15:33.090 --> 00:15:35.979
Keri McAvoy: How would you apply it in a real world context?

92
00:15:36.590 --> 00:15:52.600
Keri McAvoy: So how could faculty for this course provide choice to this formative assessment, using the strategies that we discussed. And I believe Emily is going to actually place. Yeah, thank you, Emily. The list of strategies I just reviewed on the last slide. So you can refer back to that.

93
00:15:52.720 --> 00:15:59.289
Keri McAvoy: So go ahead and take about a minute to think about it, and then type your answer in the chat, or raise your hand if you prefer that

94
00:16:39.290 --> 00:16:52.970
Keri McAvoy: awesome. Thank you, Celine, allowing students to respond or participate with text or video. So giving a different opportunity for how they respond is a great way to provide choice and assessment.

95
00:16:58.950 --> 00:17:08.899
Keri McAvoy: Yeah, good. So they get to discuss. It's a topic selection essentially right. They get to discuss what within the module, what topic they want to focus on.

96
00:17:10.240 --> 00:17:21.109
Keri McAvoy: And mia also likes topic selection as well as a multi-step discussion that begins individual and then breaks out for a deeper dig with a partner. Great, thank you. Guys.

97
00:17:24.589 --> 00:17:26.129
Keri McAvoy: yeah, let's go ahead, Lauren.

98
00:17:28.400 --> 00:17:57.340
Keri McAvoy: So now that we've reviewed different ways to provide choice within assessment, let's look even bigger. Picture at providing choice of assessments providing choice of assessments is important, because it acknowledges that students learn and express their understanding in diverse ways by offering multiple assessment formats. Educators can create more inclusive learning environments that accommodate different abilities, preferences, and cultural backgrounds.

99
00:17:57.530 --> 00:18:05.229
Keri McAvoy: This approach raises engagement and ownership, allows students to showcase their strengths and allows for growth in other skill sets.

100
00:18:05.960 --> 00:18:27.619
Keri McAvoy: Some ways you can do this are to let students select the medium or format through which they demonstrate learning so kind of like what Celine just said on the last slide. You could have them write a traditional essay, but maybe they would prefer to record a video presentation or design an infographic or slide, deck, or even create a podcast or audio reflection. Instead.

101
00:18:28.870 --> 00:18:51.840
Keri McAvoy: you could also provide choice and assessment type. So letting students select how they're assessed. So maybe instead of a multiple choice quiz, they would actually prefer a short answer, or a creative project, or similar to what I expressed earlier, allowing them to choose group versus individual work, or even graded peer review versus instructor evaluation. Before submitting the final assignment.

102
00:18:53.040 --> 00:18:55.889
Keri McAvoy: you can provide choice in skill emphasis.

103
00:18:56.250 --> 00:19:06.889
Keri McAvoy: so offer assignments that target different skills, like analytical writing, creative thinking, visual communication, oral communication, or even technical proficiency.

104
00:19:07.530 --> 00:19:28.140
Keri McAvoy: You could let students design their assessments so you could have them propose their own project or research question with guidance and approval, and even allow them to design a rubric that they could work collaboratively on with you, or they could choose from a preset criteria to include in that rubric.

105
00:19:28.180 --> 00:19:39.140
Keri McAvoy: So now I'm going to pass it over to Dave Love, who's a faculty at the school of public health. Who's going to share more about how he provides choice of assessment in his course? Dave.

106
00:19:41.590 --> 00:19:50.049
Dave Love: Thanks, Jerry. Those are really interesting ideas, and they just got me thinking whole different way. Thank you. So I want to

107
00:19:50.190 --> 00:19:54.589
Dave Love: give you a quick overview of the course I teach in the 3rd term.

108
00:19:54.940 --> 00:19:59.400
Dave Love: It's called seafood and public health, global trade nutrition and the environment.

109
00:20:00.153 --> 00:20:06.350
Dave Love: We we, you get a lot of environmental health students and international health students.

110
00:20:06.889 --> 00:20:13.919
Dave Love: It's an online course. And so that's that's important. When I get into sharing some of the assessment types.

111
00:20:15.500 --> 00:20:23.589
Dave Love: so what we heard from students was they liked the course, but they wanted more flexibility in the types and number of assignments.

112
00:20:23.730 --> 00:20:27.419
Dave Love: and we heard that from them through their end of year feedback!

113
00:20:29.010 --> 00:20:33.379
Dave Love: And that's what we that's what we heard. And then we also felt like

114
00:20:34.002 --> 00:20:40.330
Dave Love: we had some concerns about how students were were dealing with assignments around the use of AI

115
00:20:40.814 --> 00:20:43.059
Dave Love: Some of the assignments that we

116
00:20:43.230 --> 00:20:50.710
Dave Love: we gave were just really writing heavy, and they were hard to review and grade.

117
00:20:50.940 --> 00:21:05.869
Dave Love: And it just wasn't as much. I wasn't having a lot of fun in the grading, as I don't know if grading can ever be fun, but it was just a sort of droll, so I wanted to think of some new ideas that made it more fun for me to be a teacher

118
00:21:06.350 --> 00:21:07.509
Dave Love: next slide.

119
00:21:09.670 --> 00:21:18.480
Dave Love: So I'll tell you a little bit about the course itself, and sort of the the scales in the course, so that you can understand just different assessments

120
00:21:18.850 --> 00:21:31.610
Dave Love: options. So we have because it's majority online, we have a lot of asynchronous content. But we give it to them in different ways. We have recorded lectures from faculty and outside experts.

121
00:21:31.860 --> 00:21:43.180
Dave Love: but then layered on top of that, I do interviews with experts. So folks who have written key papers that we make have the students read. They're often our interviews with those folks. They're about 10 min long.

122
00:21:43.420 --> 00:22:00.420
Dave Love: We plug in different documentaries that are really well done, that cover the topic and lots of short videos so that students can see what they're what these different simply production methods are because they may not be familiar with with them.

123
00:22:01.408 --> 00:22:11.379
Dave Love: We have some asynchronous interactions, mostly as discussion forums, hosts, and some of this is just to for students who might miss a live talk

124
00:22:11.510 --> 00:22:13.610
Dave Love: they can plug in there.

125
00:22:14.510 --> 00:22:17.730
Dave Love: We also have a journal club and an asynchronous discussion.

126
00:22:18.470 --> 00:22:20.080
Dave Love: If students miss the general pill.

127
00:22:20.320 --> 00:22:24.382
Dave Love: and then we have bi-weekly live talks. So

128
00:22:25.770 --> 00:22:40.240
Dave Love: and also bi-weekly journal. So what I mean by bi-weekly is, every 2 weeks we do a live talk, and then on the odd week we slip a journal club in to give them a time slot so they can interact with each other in in real time.

129
00:22:41.910 --> 00:22:44.990
Dave Love: although virtually so.

130
00:22:45.110 --> 00:23:02.779
Dave Love: we do a written assignment which is a big part of it. This is historically been a big part of their grade. We we break up the written assignment into 4 weekly assignments that are stretched out throughout the course, so they'll do one assignment every 2 weeks.

131
00:23:04.130 --> 00:23:11.720
Dave Love: And we started with just strictly, you know. Give me a thousand or 1,500 word essay with references.

132
00:23:11.890 --> 00:23:16.559
Dave Love: and and make it look like you know a Mini term paper.

133
00:23:17.030 --> 00:23:32.389
Dave Love: And we did that for a couple of years. And and it worked okay. But then we see people kind of drift into this, maybe AI space where we're seeing answers that aren't well thought out. And you know, students that actually come to me and said, I really do want to do something besides an essay.

134
00:23:33.690 --> 00:23:34.315
Dave Love: So

135
00:23:35.450 --> 00:23:42.110
Dave Love: Last year a student came and said, Can I do an infographic instead of an essay?

136
00:23:42.809 --> 00:23:51.379
Dave Love: And I think I brought it to Lauren. And I said, This is interesting. How do I grade this? So we talked a little bit about

137
00:23:51.620 --> 00:23:54.149
Dave Love: how to submit something like that how to grade it.

138
00:23:54.490 --> 00:24:04.509
Dave Love: It was sort of a proof of concept. Last year, and then this year we went crazy and gave students lots of options. Every assignment had multiple modes they could submit.

139
00:24:06.100 --> 00:24:09.490
Dave Love: So, for example, here's some options that we did for our course.

140
00:24:09.970 --> 00:24:13.700
Dave Love: students could write an essa, a traditional essay with references.

141
00:24:14.110 --> 00:24:19.630
Dave Love: or they could write a policy memo and some tweets to help push out their message in social media

142
00:24:20.571 --> 00:24:25.209
Dave Love: Brought certain assignments. They could resign, write a resilience action plan.

143
00:24:25.380 --> 00:24:27.719
Dave Love: They could give a persuasive argument.

144
00:24:28.233 --> 00:24:34.920
Dave Love: They could do an infographic now to grade that infographic. It can't just be pictures. I also ask them to give references.

145
00:24:35.387 --> 00:24:37.470
Dave Love: Like a full reference list of

146
00:24:37.600 --> 00:24:42.899
Dave Love: the sources that they use to to think about these images and and the statistics.

147
00:24:43.490 --> 00:24:51.750
Dave Love: If anybody wants to do more Powerpoint presentations, they can do a Powerpoint. But it's not just any kind of Powerpoint. I actually I asked them to think.

148
00:24:51.920 --> 00:24:59.460
Dave Love: if they are within, within the Ministry of Health of a country, to make a Powerpoint that they could share with their with their supervisors.

149
00:24:59.820 --> 00:25:04.209
Dave Love: So they have something when they're thinking about how to the motivation.

150
00:25:04.330 --> 00:25:12.560
Dave Love: One assignment students could write, develop a menu that was a menu for food for a local elementary school.

151
00:25:13.100 --> 00:25:19.770
Dave Love: and then another one. They could develop a survey, a survey tool to a hypothetical study population.

152
00:25:20.629 --> 00:25:22.189
Dave Love: Okay. Next slide.

153
00:25:22.340 --> 00:25:27.620
Dave Love: Think I'll give him some example of what some of these things look like. Yeah, okay. So

154
00:25:27.870 --> 00:25:33.060
Dave Love: on. The left is is an infographic. The student submitted for an assignment.

155
00:25:33.220 --> 00:25:36.899
Dave Love: and this was alongside other students who also, who submitted essays

156
00:25:37.050 --> 00:25:38.884
Dave Love: and then at the right is

157
00:25:39.880 --> 00:25:47.860
Dave Love: proposed menu for an elementary school in which the task was to incorporate local aquatic foods.

158
00:25:48.340 --> 00:25:50.310
Dave Love: and and give us

159
00:25:50.430 --> 00:25:58.270
Dave Love: what types of foods would you incorporate, and the reasons why nutritional benefits or the local availability?

160
00:25:58.400 --> 00:26:06.079
Dave Love: And then I also asked for a paragraph header before the the menu, so they could kind of unpack the reasons why they made those choices.

161
00:26:06.180 --> 00:26:12.129
Dave Love: And that's really helpful in terms of grading, because it's very hard to grade these nontraditional things.

162
00:26:12.604 --> 00:26:18.799
Dave Love: You really have to think about the rubric and then make the rubric work for multiple types of

163
00:26:19.200 --> 00:26:22.150
Dave Love: of assignments next slide.

164
00:26:22.870 --> 00:26:26.320
Dave Love: We also added something interesting. This year

165
00:26:26.440 --> 00:26:30.120
Dave Love: we were, we wanted to do an oral exam.

166
00:26:30.230 --> 00:26:33.579
Dave Love: because we want to. AI try to AI proof our course.

167
00:26:33.760 --> 00:26:40.180
Dave Love: And so, instead of what had traditionally been a Powerpoint presentation in front of class.

168
00:26:40.430 --> 00:26:49.439
Dave Love: which really wasn't very satisfying for the students to do, because 5 min really wasn't enough. But we needed to get everybody through in one. You know, one long class period.

169
00:26:50.107 --> 00:27:04.699
Dave Love: Sometimes students on this this final assignment were like recording it and emailing it to us instead of showing up for the in-class presentation. This was just wasn't working. So we switched gears, and we did this 15 min oral exam.

170
00:27:05.280 --> 00:27:09.869
Dave Love: We had 3 questions and we had 2 graders.

171
00:27:10.010 --> 00:27:15.070
Dave Love: They didn't get the questions until they entered the exam room zoom room

172
00:27:16.640 --> 00:27:20.160
Dave Love: And the students did great. They really

173
00:27:20.500 --> 00:27:23.949
Dave Love: so showed us what they had learned over the over the terms

174
00:27:24.230 --> 00:27:37.619
Dave Love: to help get them ready for that type of assignment. We gave them a lot of information front loaded from day one on what this oral exam is going to look like. Why we need to do it, how to prepare

175
00:27:39.210 --> 00:27:46.189
Dave Love: we scheduled time before the week before, so that students could have a practice session.

176
00:27:47.001 --> 00:27:50.860
Dave Love: To work on their oral exam skills. Ask any questions of us,

177
00:27:52.660 --> 00:28:07.009
Dave Love: and and at the end of the oral exam we had, we tacked on an extra 5 min that we didn't tell them about, but we sort of let them deep, you know. Debrief the course for us, and said, Tell us what you thought of the course. What what can we do better?

178
00:28:07.740 --> 00:28:16.245
Dave Love: We got the most amazing feedback. So I'm really excited about next year and offering that feedback because you don't always get, you know, the feedback you get from the written

179
00:28:17.564 --> 00:28:21.460
Dave Love: written feedback. It's you know you get it, but you don't.

180
00:28:21.750 --> 00:28:24.779
Dave Love: You don't. It's sometimes better just to hear it than to read it.

181
00:28:24.980 --> 00:28:30.810
Dave Love: because they can be a little bit more nuanced with their critique plus. They're already done with the course, so they can tell you what they really think.

182
00:28:32.620 --> 00:28:35.110
Dave Love: So I really love the oral exam.

183
00:28:35.580 --> 00:28:44.269
Dave Love: One thing to note is that non-english language speakers had a harder time with the oral exam.

184
00:28:44.560 --> 00:28:53.160
Dave Love: and it it was harder for them to express themselves in a because of

185
00:28:53.910 --> 00:28:57.170
Dave Love: some of these ideas are quite complex and nuanced.

186
00:28:57.500 --> 00:29:07.769
Dave Love: and getting a question where they don't have time to sit and think, we have to respond right away, was challenging. So that's something we're going to have to think about for next year. How do we

187
00:29:07.970 --> 00:29:12.429
Dave Love: of think about this in the context of non-english language speakers?

188
00:29:13.610 --> 00:29:17.410
Dave Love: So that is there if there is a next slide I mentioned?

189
00:29:20.080 --> 00:29:20.750
Dave Love: Alright.

190
00:29:21.450 --> 00:29:30.189
Dave Love: okay. So we got really great feedback, and this was not at the end of oral exam. This is the the survey at the end of the course

191
00:29:30.570 --> 00:29:36.100
Dave Love: overall. Students really liked it, and they love these new assessments. So

192
00:29:36.530 --> 00:29:43.850
Dave Love: it was kind of a big. We went out on a big limb to dry out new assessments had no clue if they're going to work.

193
00:29:44.550 --> 00:29:46.600
Dave Love: had no clue how to grade them.

194
00:29:47.180 --> 00:29:49.300
Dave Love: but it turns out they're not. It's not that hard.

195
00:29:50.029 --> 00:29:53.430
Dave Love: So I would say, if you're thinking about trying new assessments

196
00:29:53.610 --> 00:29:59.549
Dave Love: go for it. The students are really hungry for new ways to show what they show what they know.

197
00:30:00.733 --> 00:30:06.749
Dave Love: Here's some quotes and I'll just let you. You read read these quotes if you'd like

198
00:30:14.560 --> 00:30:32.010
Lauren Dana: Great. Thank you, Dave, and I know Dave is being humble, but his reviews, evaluations were excellent. I know all of you that teach. It's not easy to get a hundred percent excellent on assessments and course. So the students, it really paid off all the work you did in the course and offering that assessment choice

199
00:30:33.710 --> 00:30:35.940
Lauren Dana: great. So now I can pass it over to. Oh, sorry, Dave, go ahead.

200
00:30:35.940 --> 00:30:41.669
Dave Love: What one other thing is. Also listen to your ta. Some of these assessment ideas came from the Ta.

201
00:30:42.340 --> 00:30:52.720
Dave Love: Who was a current doctoral student and knows what it's like to go through the courses as a learner. So you know, listen to your ta. If you have, do you want to bounce ideas off anybody. You know.

202
00:30:53.780 --> 00:30:59.810
Dave Love: Lauren was great to bounce ideas, I'll help. And and our ta was asked. So those makes it really easy for me.

203
00:31:01.330 --> 00:31:12.030
Lauren Dana: Great. Thank you. That's a great great point, too. It's gonna be super helpful. Great, thank you, Dave. So I'm gonna pass it quickly over to Carrie, and we're gonna try to get you to put this to work in a very short activity.

204
00:31:12.810 --> 00:31:33.169
Keri McAvoy: Yeah, thank you, Lauren. And I do want to just note that. Mia, I did see your question in there. We're just gonna get through this activity and kind of the wrap up of the 1st half of the picnic Pd session, and then the second half of our picnic pd session is when we're gonna have our QA. For anyone who has questions for Dave or Lauren and myself.

205
00:31:34.460 --> 00:31:39.184
Keri McAvoy: So I did just want to acknowledge that I see that so really quick

206
00:31:40.040 --> 00:32:04.589
Keri McAvoy: So I want you to take a second. I know we went through a lot of information, but I want you to think from courses, past or present, select an assignment from one of your courses and using some of the strategies that we've discussed today, whether that's choice within or choice of assessment, identify one, maybe 2 ways, that you could add those choices within that assessment of your choice

207
00:32:05.030 --> 00:32:32.529
Keri McAvoy: and share your ideas in the chat or out loud with the group. I think Emily's going to add a list in the chat. Let me know if you need me to do that, Emily. That's going to go through. Thank you. Some of those things we went through today. But don't. Our lists are not the things we went over today are that's not restrictive. If you listen to Dave, or thought of other ways that you're already providing choice within or of assessment, please.

208
00:32:32.896 --> 00:32:41.313
Keri McAvoy: Include that in in your response. So Lauren's gonna play some music. We're gonna wait a few minutes and feel free to

209
00:32:41.700 --> 00:32:45.659
Keri McAvoy: type your response in the chat, or raise your hand, and we'll call on you.

210
00:33:51.470 --> 00:34:02.659
Keri McAvoy: Okay, great thanks, Lauren. I know we're we're rushing here a little bit. I'm asking you to do a lot in a short amount of time, but we are over our kind of 1st 30 min.

211
00:34:02.950 --> 00:34:21.390
Keri McAvoy: 30 min of this session. So if anyone would like to raise their hand and share what they think or type in the chat, how could you provide choice within, or choice of an assessment that you've given in in one of your courses, past or present.

212
00:34:38.609 --> 00:34:53.390
Keri McAvoy: Okay, mia, says I'm working with a faculty now that she's also co-taught to create options for gathering needs, assessment data and curriculum development. The students are tasked with developing an educational plan and gathering stakeholder feedback.

213
00:34:53.389 --> 00:35:14.270
Keri McAvoy: And it's sometimes a challenge. We are playing with ideas for offering options, to do outreach to real people or to create profiles and use AI to craft context and skill gaps. I think that's great. Mia, I think anytime you incorporate people, real people from the outside world, it always gets student buy in.

214
00:35:16.110 --> 00:35:27.650
Keri McAvoy: Okay, amazing. So I'm gonna pass it back to Lauren, who's just gonna go through some key takeaways and then we're gonna dive into our QA session.

215
00:35:28.400 --> 00:35:38.209
Lauren Dana: Great. Thank you, Carrie. So before we head into Q. And A, I just want to remind you of a few things so choice in assessments increases autonomy, relatedness and competence.

216
00:35:38.990 --> 00:35:50.850
Lauren Dana: assessment, choice maintains the same learning objectives. So, for example, Dave's course met all the same learning, objectives, and assessment criteria that allowed variation, and how students demonstrate their knowledge and skills.

217
00:35:51.360 --> 00:35:57.650
Lauren Dana: Very, very astute. We went over how you can provide choice within assessments or a choice of assessments.

218
00:35:58.010 --> 00:36:11.219
Lauren Dana: and as Dave's evaluations showed, students will value absolutely any areas of your course when you add choice, it doesn't mean you need to do the entire course, but even one or 2 assignments where you have choice will be valued.

219
00:36:12.190 --> 00:36:19.880
Lauren Dana: Okay, so we have our references. And I'm now just gonna quickly turn it over to Emily. Who's going to give a couple more details before we get to Q. And A.

220
00:36:20.470 --> 00:36:36.319
BSPH CTL Teaching Toolkit: Thank you all again. So much for joining us. We really value your feedback on the workshop today. So if you can, we would love to hear from you. I'll put the link in the chat in just a minute, or you can use the QR. Code

221
00:36:36.440 --> 00:36:41.059
BSPH CTL Teaching Toolkit: also. Go ahead

222
00:36:43.250 --> 00:37:06.399
BSPH CTL Teaching Toolkit: Sorry I'll get the chat in there or the link in there in just a second. I'm sorry, you guys, we have other professional development options with Ctl, including Ta. Training, which has a rolling admission. We'd love for you to join and be part of that with us. We also have lots of workshops offered by Ctl. By Lauren and Carrie and the rest of our team.

223
00:37:06.400 --> 00:37:16.310
BSPH CTL Teaching Toolkit: either synchronously or on demand. We have a Ctl blog as well with up-to-date resources. Thank you, Carrie.

224
00:37:16.310 --> 00:37:26.890
BSPH CTL Teaching Toolkit: for the links in the chat. So Carrie put the evaluation form in there, as well as the links to the essentials course, the training for faculty and tas as well as Ctl. Events in the blog.

225
00:37:27.100 --> 00:37:42.729
BSPH CTL Teaching Toolkit: And now I'll pass back to Lauren and Carrie. We're here for some office, and Dave. Sorry, Dave didn't forget you. We're here for some office hours, I know. Actually, Dave's. Dave's the main show. People are excited to ask you about how it's worked in your class. So thank you.

226
00:37:43.010 --> 00:38:11.189
Keri McAvoy: Yeah. Mia had to jump off, but I did want to go through the questions she asked a few minutes ago, just so she can come back to them in the recording but these are for you, Dave, he says. I have 2 questions about the assignment with format options. One. Did you have students share their assignments? Final version with their peers, too? The different approaches are likely very interesting. And 2, did you have rubrics for the grading. If yes, were they different for each format option.

227
00:38:11.540 --> 00:38:16.919
Dave Love: Yeah. So 1st question, did we did students do peer review?

228
00:38:17.030 --> 00:38:27.869
Dave Love: We had in early years the course like the 1st second year we did do peer review. We found this to be very challenging for 2 reasons.

229
00:38:28.040 --> 00:38:30.579
Dave Love: What happens when a student is late to submit

230
00:38:31.160 --> 00:38:54.920
Dave Love: and they have a peer review partner who's frustrated because, hey, I want to review. But there's nothing for me to review. And then the flip side is what happens. If the peer reviewer just doesn't show up. You know your buddy doesn't review. How do we grade this? What do we do with this information? Where you know other students are getting the benefit of having somebody review to review where in other cases

231
00:38:55.030 --> 00:38:56.820
Dave Love: there's a mismatch in timing.

232
00:38:57.080 --> 00:38:59.379
Dave Love: And so that was really frustrating for us.

233
00:38:59.920 --> 00:39:02.780
Dave Love: So we dropped the Peer Review component.

234
00:39:04.640 --> 00:39:13.629
Dave Love: We've been toying around with other ideas about how to have peer interaction. And we're thinking about maybe next year, creating like interest groups

235
00:39:13.780 --> 00:39:33.669
Dave Love: where students with similar interests can be group can self organize. For example, our assignments are based on geography. You pick a country, and there's 4 countries to pick to choose from. So maybe we'll have continent groups and everybody who's who joined the countries in Asia can work together and can. And you know, kind of interact.

236
00:39:33.820 --> 00:39:36.199
Dave Love: So that's maybe one way we're going to try to do this.

237
00:39:36.797 --> 00:39:40.070
Dave Love: But not not do peer grading for

238
00:39:40.210 --> 00:39:42.739
Dave Love: anything that we have to like check?

239
00:39:43.910 --> 00:39:47.329
Dave Love: And then the other question was, what was it? What was that?

240
00:39:47.330 --> 00:39:53.170
Keri McAvoy: It was, did you have rubrics? And if yes, were they different for each format option.

241
00:39:53.370 --> 00:40:04.120
Dave Love: Yeah, the rubrics were the same for all format options, and I think that was really important for grading, particularly because I think there was a question earlier about how many are in the course we have about 20,

242
00:40:04.250 --> 00:40:16.489
Dave Love: and then we have a team of I'm the lead instructor. We got 2 co-instructors and a ta. So all 4 of us want to have our hand in grading. So if you use the same rubric, it makes it a lot easier for grading.

243
00:40:18.660 --> 00:40:26.319
Keri McAvoy: Awesome. Thank you. And Celine had a question before she left, which you just answered. So I am seeing

244
00:40:29.000 --> 00:40:38.000
Keri McAvoy: Lou asked. How did you make sure that the assessment options are at the same or similar level of difficulty? Essentially.

245
00:40:38.320 --> 00:40:40.691
Dave Love: Yeah, that's a really really good question.

246
00:40:41.820 --> 00:40:43.678
Dave Love: it's hard to do that.

247
00:40:44.590 --> 00:40:48.960
Dave Love: you know, I'm just trying to imagine how much time they're going to invest in the assignment.

248
00:40:50.140 --> 00:40:58.619
Dave Love: some students are better at writing essays, and and they might choose that option because they like that. Other students think more visually.

249
00:40:58.850 --> 00:41:01.989
Dave Love: You know, if you think about an infographic

250
00:41:02.200 --> 00:41:04.690
Dave Love: and a thousand word essay, you know

251
00:41:05.620 --> 00:41:08.709
Dave Love: all the key points that are in the essay

252
00:41:08.830 --> 00:41:11.399
Dave Love: could appear or should appear in the infographic

253
00:41:13.330 --> 00:41:17.710
Dave Love: What's different about the infographic is how those ideas relate to each other.

254
00:41:19.770 --> 00:41:24.109
Dave Love: It's tough. It's really tough to give to give assignments that have the the same amount of time.

255
00:41:24.430 --> 00:41:25.640
Dave Love: Yeah.

256
00:41:26.550 --> 00:41:32.019
Dave Love: I don't know. I mean, I think that comes with just iteratively doing trying things out, you know.

257
00:41:32.160 --> 00:41:42.729
Dave Love: and and getting better at refining how you pitch the the assignments and and the criteria that you're grading on so that you make it rigorous.

258
00:41:43.276 --> 00:41:45.069
Dave Love: For the people who aren't doing essays.

259
00:41:46.950 --> 00:41:47.899
Keri McAvoy: Thank you, dear.

260
00:41:48.613 --> 00:41:55.199
Keri McAvoy: So we don't have any more questions in the chat. Did anyone else want to raise their hand or type in the chat. Emily, go ahead.

261
00:41:56.734 --> 00:42:08.780
BSPH CTL Teaching Toolkit: I love all the assignment options, and I think your class is a little unique. It's it's not something we see super commonly at the school. Do the students get a chance to see each other's

262
00:42:09.040 --> 00:42:17.230
BSPH CTL Teaching Toolkit: work like each other's infographic? And the menu and I just feel like they're cool. So it would be cool to see your peers assignments.

263
00:42:17.230 --> 00:42:27.140
Dave Love: Yeah. So I did ask some of the students who created, like the the best products this year if they'd be willing to share with students next year.

264
00:42:27.641 --> 00:42:32.419
Dave Love: So I've got those I've got like held onto them. Got them saved in a little folder.

265
00:42:33.860 --> 00:42:36.819
Dave Love: The trick is, how do you share things without?

266
00:42:37.230 --> 00:42:43.239
Dave Love: You know, if I share, like the the work products that I might have shared the countries

267
00:42:44.040 --> 00:42:47.400
Dave Love: I might have to redo, what countries the assignments come from.

268
00:42:47.810 --> 00:42:53.220
Dave Love: So that's gonna be tricky. So it's like I could share. But then I might have to change.

269
00:42:53.870 --> 00:42:58.560
Dave Love: That'd be a little bit more work for for our, you know, teaching team to like.

270
00:42:59.230 --> 00:43:01.230
Dave Love: bring it. Bring that in.

271
00:43:01.780 --> 00:43:03.195
Dave Love: So I don't know.

272
00:43:04.490 --> 00:43:08.520
Dave Love: Yeah, that's gonna be a hard one. Yeah.

273
00:43:10.430 --> 00:43:21.309
Dave Love: The other thing that just in terms of how the students felt about the course is how quick you can return. That 1st assignment is going to be a good indicator of how much they

274
00:43:21.510 --> 00:43:29.689
Dave Love: they're gonna enjoy that those assignments? I think. So. We made a really big push to return the 1st assignment within 5 days.

275
00:43:30.760 --> 00:43:36.650
Dave Love: and really be clear about them about when they can expect to get things returned of.

276
00:43:37.110 --> 00:43:43.890
Dave Love: You know they really it's a it's a quick turn for the graders, but it's I think it's important to show them that, like we're invested, too.

277
00:43:44.645 --> 00:43:53.749
Dave Love: So it's not just what you assign. But hey? You could assign the the greatest things ever. But if it takes you 4 weeks to turn around a grade, they're gonna be frustrated.

278
00:43:54.380 --> 00:43:57.600
Dave Love: So there's there's more to it than just the front end.

279
00:43:58.700 --> 00:43:59.390
Dave Love: Yeah.

280
00:43:59.740 --> 00:44:09.910
BSPH CTL Teaching Toolkit: I have another question. If nobody else has questions, I think this is awesome. So I have another question. Can you talk a little bit about the time investment, because it seems like having that

281
00:44:10.290 --> 00:44:15.599
BSPH CTL Teaching Toolkit: you'd have to give a lot of direction. Sorry did you lose me?

282
00:44:15.830 --> 00:44:16.899
Dave Love: Oh, no, I I can hear you.

283
00:44:16.900 --> 00:44:17.665
BSPH CTL Teaching Toolkit: Okay,

284
00:44:18.980 --> 00:44:37.639
BSPH CTL Teaching Toolkit: you'd have to like design, a lot of instructions and things. And then, can you just talk about like, is it a lot more work on the faculty's part to to do all that? Or did you find that then other parts were easier? How did the time investment change for you.

285
00:44:38.740 --> 00:44:41.769
Dave Love: Yeah, writing clear instructions and having the rubric.

286
00:44:41.880 --> 00:44:48.739
Dave Love: And Lauren was really helpful at sharing some rubric ideas. So, having the instructions and the rubric all there

287
00:44:48.850 --> 00:44:57.050
Dave Love: and then. You know, going over it during the live talks, so saving time and live talks to go over, and also

288
00:44:57.340 --> 00:45:00.773
Dave Love: just being open to answering any kind of questions that come in.

289
00:45:02.070 --> 00:45:06.370
Dave Love: yeah, students, you know, they know how to write the essay. But then these other things they're just.

290
00:45:06.770 --> 00:45:11.729
Dave Love: is it? Okay? How do I do this? So yeah, you have to answer questions and and a little bit of hand holding.

291
00:45:12.100 --> 00:45:15.119
Dave Love: And the oral exam. Took a lot of handholding.

292
00:45:15.420 --> 00:45:17.700
Dave Love: They were students that were not

293
00:45:17.970 --> 00:45:22.209
Dave Love: excited about it. But then, you know, they in the end they loved it like

294
00:45:22.470 --> 00:45:26.790
Dave Love: they, I said, when do you get to have spend 15 min on your final?

295
00:45:26.990 --> 00:45:27.790
Dave Love: You know

296
00:45:29.250 --> 00:45:35.990
Dave Love: it's a huge time savings, plus. They scored really well, because they know the material they've been taking the class.

297
00:45:37.450 --> 00:45:41.500
Dave Love: I think one of the big challenges for us is synchronous versus asynchronous

298
00:45:41.995 --> 00:45:48.159
Dave Love: even our even so, only about half the folks could make it to the synchronous.

299
00:45:48.370 --> 00:45:50.690
Dave Love: you know, lectures.

300
00:45:50.920 --> 00:45:56.739
Dave Love: So we've got 20 students, but we know from the beginning only 10 can make it on consistently.

301
00:45:56.920 --> 00:46:08.099
Dave Love: and it's really hard to connect with those, those asynchronous, the distance students who never come to the synchronous stuff. So the 1st time we see them is the final exam. And that's really hard.

302
00:46:09.220 --> 00:46:13.440
Dave Love: Yeah, Kerry. And there's also Beth.

303
00:46:13.440 --> 00:46:22.929
Keri McAvoy: Oh, yeah, I'll let Beth. I'll read Beth's, and then I'll if we can get to mine. I will. Beth said. I like peer review. When one of your los is to

304
00:46:23.050 --> 00:46:38.939
Keri McAvoy: learning objectives is to build mentorship skills. Canvas has a nice peer review feature, and I like to incorporate 2 way. Peer review reflections for both the receiver and reviewer, but not for all assignments. Just the highest one that's awesome. Thank you, Beth, for that

305
00:46:39.547 --> 00:46:46.700
Keri McAvoy: and then thoughts on using an AI bot to sort and make an initial assessment of the submissions.

306
00:46:47.363 --> 00:46:58.875
Keri McAvoy: applying similar criteria and assigned weights to assist the instructor in conducting the final assessment, and grading thoughts.

307
00:47:02.440 --> 00:47:05.790
Dave Love: Yeah, that's a that's a that's an idea on

308
00:47:05.970 --> 00:47:07.845
Dave Love: on this, on Beth's comment.

309
00:47:09.440 --> 00:47:14.459
Dave Love: you know, maybe there's a way to do it so that you don't have to. You don't pair the people up.

310
00:47:14.940 --> 00:47:16.395
Dave Love: You could just

311
00:47:17.450 --> 00:47:21.839
Dave Love: If you're ready to to do a peer review, you just grab somebody's paper and peer review it.

312
00:47:21.840 --> 00:47:22.390
Beth Romanski: Yeah.

313
00:47:22.390 --> 00:47:23.379
Dave Love: Push it back in.

314
00:47:23.380 --> 00:47:23.940
Beth Romanski: Yeah.

315
00:47:23.940 --> 00:47:25.169
Dave Love: Pairing was the problem.

316
00:47:25.170 --> 00:47:36.020
Beth Romanski: Yeah, and what's nice about that? Well, see, I have an advantage. Because I I'm using canvas. And so it the tool itself automatically does that

317
00:47:36.180 --> 00:47:52.869
Beth Romanski: so that you know, you'd have to. Manually. I'm not as familiar with course, plus, but that's an idea, because the pairing could be an issue. If you get a pair, you know, you know it doesn't work out, but the canvas tool you can set it so that it automatically assigns students to peer review.

318
00:47:53.230 --> 00:48:15.479
Beth Romanski: You know it'll just do that itself, or you can manually, as the instructor do it. But I do like I learned this from another webinar, which, by the way, I just want to do a shout out here, I love these little sessions that you host in the summer. Thank you for organizing. But one thing that I've actually thought about more recently, from a session I took was the reciprocal Peer Review.

319
00:48:15.730 --> 00:48:44.500
Beth Romanski: Because oftentimes you're only thinking about it one way, but to have the person who's reviewing to reflect on that process, you know, have some prompts along with the actual review of the assessment. That seems to, you know, sort of increase and reinforce the metacognition in the process of being a peer reviewer. So it creates more accountability that they have to actually not just give like Oh, good, you know, like some peer reviews may not be useful because they haven't really taken the time

320
00:48:44.630 --> 00:48:57.569
Beth Romanski: to do a thorough job. So that's another thing that I learned as a tip. And that's what I meant about the 2 way it's like, what was it like receiving feedback? How are you going to use it? Then also, what was it like giving feedback? And then you know that process.

321
00:49:02.560 --> 00:49:08.180
Keri McAvoy: Yeah, that's amazing. Thank you so much, Beth, and thank you for attending and for the the kudos for these Pd.

322
00:49:08.180 --> 00:49:11.830
Beth Romanski: I love them. You learn so much from each other, and it's great.

323
00:49:12.600 --> 00:49:14.069
Keri McAvoy: They? They really are great.

324
00:49:14.642 --> 00:49:43.317
Keri McAvoy: So I just wanna reiterate and Rikito, I think. Tell me, let me know if I'm expressing this incorrectly, but I think their question was and this could, I think, could be to Dave or myself, and Lauren, who may have just had dropped off. What are our thoughts? This could be open to the audience. Really, what are our thoughts on essentially using AI to kind of make initial

325
00:49:44.310 --> 00:49:47.440
Keri McAvoy: assessment or initial

326
00:49:48.510 --> 00:50:01.190
Keri McAvoy: to assist the instructor in conducting the final assessment and grading and what? And I guess, potentially thinking, because these are so different could kind of putting them into AI help you.

327
00:50:01.470 --> 00:50:05.730
Keri McAvoy: norm the grading a little bit. I I think that's what

328
00:50:06.520 --> 00:50:10.479
Keri McAvoy: is being expressed in the chat. Please tell me if I am wrong.

329
00:50:10.885 --> 00:50:31.160
enriquito lu: Yeah, that was the point where in, if you have a variety of media as well as approaches, whether it's Powerpoint, visual or presentation, etc. for an instructor or trainer the challenges of always going to be time that they can put into

330
00:50:31.992 --> 00:50:53.120
enriquito lu: really, honestly assessing those submissions. So I was just thinking that with all of the chat, with all of the AI bots that are now coming for the different applications that are available is, how can we use AI to initially assess and just create a norm so that

331
00:50:53.754 --> 00:51:04.559
enriquito lu: it assists more the trainer or the instructor make an honest assessment of these admissions, and grade them accordingly.

332
00:51:06.710 --> 00:51:10.639
Keri McAvoy: I'm gonna let Beth go 1st because she raised her hand. Go ahead, Beth.

333
00:51:11.100 --> 00:51:27.649
Beth Romanski: Well, you all the experts. But I I do have a little bit of instruction design background that may be helpful. One thing that I've been working on is a new fellowship curriculum that has what you're all talking about, which is a very flexible capstone project.

334
00:51:28.030 --> 00:51:42.599
Beth Romanski: And so what I was trying to think about is that same question. So we're not going to have a key like a primary instructor. So that also kind of gives some complexities to it. So I've looked at various types of rubrics for

335
00:51:42.680 --> 00:51:59.409
Beth Romanski: capstone projects and what the learning objectives are in terms of just giving scientific presentations, and then both written and oral and visual form. And I've created a rubric. That sort of holistically measures those things without it being a specific.

336
00:51:59.410 --> 00:52:18.339
Beth Romanski: like infographic or a power, you know, like something that's so prescriptive. And I actually used suggestions from AI to help me combine all of the themes in these rubrics that I researched to kind of come up with that sort of overarching rubric that then levels

337
00:52:18.340 --> 00:52:32.390
Beth Romanski: things out just more about like visual elements. But it doesn't specify what type of visual element. So that's what I'm going to try to pilot and see how it goes. So that's just one suggestion, but very interested in learning more about what everybody else is doing.

338
00:52:35.660 --> 00:52:40.029
Keri McAvoy: Great. Thank you for sharing Beth. Emily, I saw your hand was raised. You're good. Okay.

339
00:52:41.900 --> 00:52:44.390
Keri McAvoy: Any other questions.

340
00:52:44.790 --> 00:52:48.139
Keri McAvoy: Where are we on time. We have a few more minutes.

341
00:52:50.170 --> 00:53:16.579
Keri McAvoy: I think I have one super quick question for Dave, because you had one of you had a note on your slide, but I don't think you mentioned it. How you allowed students to choose to drop one of their assignments. Could you talk a little bit more on this. Did you let them choose this? Did they have to choose it at a certain time? Did they have to submit all the assignments and have them graded first, st and then they could drop. The lowest score. Did. Was this the same thing they had to do? Just could you touch on this just a little bit.

342
00:53:16.580 --> 00:53:31.000
Dave Love: Yeah, they we gave. So we they had these written assignments, or a big assignment do every 2 weeks. We? So that's 4 total. We let them do 3 out of the 4 so they could. They could not submit one

343
00:53:31.494 --> 00:53:35.010
Dave Love: it wasn't that they all they all got graded, and they could drop the lowest

344
00:53:35.649 --> 00:53:41.939
Dave Love: and this was more in response to student feedback that we were. The workload was

345
00:53:42.910 --> 00:53:46.099
Dave Love: really intense. I mean, it's a 3 3 credit course.

346
00:53:46.290 --> 00:53:49.470
Dave Love: and we were so we were on a 2 credit course, and we were kind of

347
00:53:49.620 --> 00:53:58.249
Dave Love: on the verge of being a 3. And then, because there's so much work. And we said, Well, we'll just make it a 3. But I think some students come into it and they go. Oh, this is a lot.

348
00:53:58.430 --> 00:54:02.639
Dave Love: So that was an option to give them less, but it really hurts when they drop

349
00:54:03.120 --> 00:54:10.389
Dave Love: any of them, but particularly like the 3rd or the 4, th because I know those are really like more synthesis based like

350
00:54:10.540 --> 00:54:16.240
Dave Love: that come into this end of the course. They're really synthesizing information when they drop that last one. It's like, Oh.

351
00:54:16.680 --> 00:54:19.959
Dave Love: you know, like like, Oh, you know.

352
00:54:20.280 --> 00:54:24.819
Dave Love: put this like dagger in my heart. Don't! Don't do, don't drop the last one.

353
00:54:25.240 --> 00:54:31.380
Dave Love: But they sometimes, you know, workflow and life and crazy stuff happens, and they need to drop.

354
00:54:31.950 --> 00:54:34.289
Dave Love: And you know, it's just the way it is. I think

355
00:54:34.500 --> 00:54:37.688
Dave Love: it's just acknowledging that you know.

356
00:54:38.940 --> 00:54:44.770
Dave Love: that we've got students who have lots of like Lauren shared

357
00:54:45.160 --> 00:54:50.679
Dave Love: wide range age distribution, and also some folks who are not full-time students.

358
00:54:52.330 --> 00:55:01.669
Keri McAvoy: Thank you. And Beth. Asked a question to add on to that, she said. Do you see different outcomes when students drop one? And it's I was actually thinking this as well. So

359
00:55:01.910 --> 00:55:16.959
Keri McAvoy: do you see? You know, maybe students decide to drop the second assignment. Do you see? Maybe in their final assignment that they it's not as strong as others who didn't decide to drop, or any anything in that

360
00:55:17.190 --> 00:55:18.060
Keri McAvoy: realm.

361
00:55:19.720 --> 00:55:26.990
Dave Love: it's hard. I don't have a big end to draw from, but when students drops, when a student drops the 1st assignment, that's a big red flag.

362
00:55:27.140 --> 00:55:31.360
Dave Love: So I had one student drop the 1st assignment because he had a work trip.

363
00:55:31.570 --> 00:55:36.849
Dave Love: and I actually gave him because the first, st the second assignment builds on the first.st

364
00:55:37.160 --> 00:55:46.329
Dave Love: So I did something that was like a little bit out of the ordinary. I gave him the first.st He missed the 1st time, but I gave him somebody else's 1st assignment

365
00:55:47.122 --> 00:55:52.270
Dave Love: from a past year. Because, I said, if you really need this, this is like foundational to.

366
00:55:52.410 --> 00:56:01.010
Dave Love: He's writing about food security in Nigeria, and he needs to know this stuff. So I gave him like a piece of a previous. I was like, look.

367
00:56:01.530 --> 00:56:07.350
Dave Love: just read this, it'll get you up to speed. I'm sorry you missed the 1st one, but you really need this piece.

368
00:56:09.520 --> 00:56:21.010
Dave Love: and he was. It actually worked out really well, because he didn't like drift away. He like reengaged, and he ended up getting stronger as the course went on. But that was just a really awkward thing to do.

369
00:56:21.160 --> 00:56:23.739
Dave Love: and I was in a really awkward position. It was like

370
00:56:24.410 --> 00:56:30.829
Dave Love: you got to do the 1st assignment. So maybe next year I'll say you have to do the 1st assignment. But second, 3, rd and 4th is the choice.

371
00:56:31.300 --> 00:56:32.940
Dave Love: But yeah, it's

372
00:56:33.400 --> 00:56:38.889
Dave Love: not ideal, but just the way their lives are that they need something a little space somewhere

373
00:56:39.340 --> 00:56:41.320
Dave Love: in in my in schedule.

374
00:56:43.470 --> 00:56:46.770
Keri McAvoy: Well, we're just about at time, Emily, go ahead.

375
00:56:46.770 --> 00:57:02.110
BSPH CTL Teaching Toolkit: I was just going to say I'm going to stop the recording, but thank you all so much for joining us. Thank you to Lauren and Carrie for guiding our conversation, and special thanks to Dave. Love, it's wonderful when we get to hear from faculty and how these

376
00:57:02.180 --> 00:57:24.659
BSPH CTL Teaching Toolkit: principles that we're advocating for really show up in your classroom. So thank you so much for taking the time to share your expertise with us. If you have more questions, comments follow up for anyone. You're welcome to reach out to us at Ctl. Lauren and Carrie, specifically on this topic. So I'm going to stop the recording now.