WEBVTT

1
00:00:02.350 --> 00:00:17.559
BSPH CTL Teaching Toolkit: Alright. Well, hello, everyone! We're gonna go ahead and get started, even though II expect a couple more people to join hopefully throughout the session. The session is being recorded, and the recording will be shared with all of the people who registered in zoom

2
00:00:17.620 --> 00:00:32.150
BSPH CTL Teaching Toolkit: and again, just Hello, and welcome to this Ctl. Teaching toolkit workshop on rubrics, benefits and grading strategies. My name is Amy Pinkerton, and I'm a senior instructional designer at the center for teaching and learning or Ctl.

3
00:00:32.520 --> 00:00:40.070
if you have any questions, you can post those in the chat, and then I'll pause throughout today's session to answer those questions.

4
00:00:41.300 --> 00:00:57.219
BSPH CTL Teaching Toolkit: Alright. So we'll go ahead and dive in. So today's session, here are the learning objectives. So we're going to first define some key terms. And then we're going to identify and discuss the benefits of rubric, of using rubrics.

5
00:00:57.290 --> 00:01:16.270
BSPH CTL Teaching Toolkit: And then finally, we'll look at identifying some structures and basic parts of a rubric, and how those can be applied into your own grading strategy. We're also going to be using some tools in today's session, including pull everywhere. So we'll have more instructions when we get to that activity.

6
00:01:18.980 --> 00:01:44.779
BSPH CTL Teaching Toolkit: So before we dive into rubrics, I wanted to describe what effective feedback is because essentially feedback is a tool or sorry rubrics are a tool for delivering feedback so and feedback. The reason why this is so important is because feedback is essential for learning. Students need feedback to know what they're doing. Well, they need feedback to know what they can improve on, and how specifically to improve

7
00:01:44.820 --> 00:01:54.190
BSPH CTL Teaching Toolkit: so when you're thinking about rubrics, think about how your rubrics are delivering feedback. And I also wanna talk about what is effective feedback

8
00:01:54.330 --> 00:02:12.989
BSPH CTL Teaching Toolkit: and some characteristics and effective feedback is specific. It's timely. And it's focused. So specificity that means that you're specific as possible in what you're identifying how the student, what they're doing well and what they need to improve on. So you're giving really specific points about their performance

9
00:02:13.320 --> 00:02:19.850
BSPH CTL Teaching Toolkit: for timeliness. Think about the time between when the student submits their assignments, and when you're delivering the feedback.

10
00:02:19.870 --> 00:02:34.710
BSPH CTL Teaching Toolkit: the the the more immediate the better. So if you can reduce that time as much as possible, the better. So. And I know with some large classes it can be difficult. But feedback usually feedback is more effective

11
00:02:34.780 --> 00:02:54.820
BSPH CTL Teaching Toolkit: whenever there's not a huge lack of time between when students completed something and when they're getting feedback on it. And finally, feedback should be focused towards a specific goal or competency in mind. So when students receive their feedback, it should be really clear that this is how you can improve, and it's for the purpose of achieving this goal

12
00:02:54.870 --> 00:03:05.309
BSPH CTL Teaching Toolkit: or why they, you know what the the feedback should again should be around a specific goal or competency. And usually, that's the learning objectives.

13
00:03:05.570 --> 00:03:20.690
BSPH CTL Teaching Toolkit: So writing feedback is, it could be a whole workshop on itself. So that's all I'm gonna talk about. For, like specifically writing feedback. But we're gonna talk a little bit more about how to incorporate your feedback using rubrics as we go through today's session.

14
00:03:22.260 --> 00:03:28.599
BSPH CTL Teaching Toolkit: Speaking of. So before we go on, I also wanted to define what it is that we mean when we say rubric

15
00:03:28.610 --> 00:03:55.709
BSPH CTL Teaching Toolkit: there are many different types of rubric, but they they tend to have these 2 core purposes, and the first is that a rubric is a tool that promotes the consistent application of learning, expectations, learning objectives or learning standards in a classroom. Or and this is the purpose that we're focusing on today. It measures their attainment against a consistent set of criteria. So when we think of rubrics, or we use that term, this is what we're referring to in today's session.

16
00:03:57.940 --> 00:04:08.309
BSPH CTL Teaching Toolkit: So so again, we're gonna use a tool called poll everywhere throughout today's workshop. And our first poll question is, do you use rubrics in your course? And

17
00:04:08.590 --> 00:04:22.410
BSPH CTL Teaching Toolkit: here are the instructions. So to answer this question. You can join by web going to the URL on the screen. You can join by text by sending if you're using your phone or you if you have a smartphone, you can also scan the QR code.

18
00:04:22.420 --> 00:04:25.610
BSPH CTL Teaching Toolkit: And I'm going to put the yeah URL in the chat.

19
00:04:37.740 --> 00:04:50.179
BSPH CTL Teaching Toolkit: So the yeah URL is now also in the chat. So when you're ready, join the poll and answer the question on the screen. and let me know if you have any questions or issues with joining

20
00:04:54.360 --> 00:04:56.810
BSPH CTL Teaching Toolkit: alright. So far we have some yeses.

21
00:05:06.260 --> 00:05:08.290
BSPH CTL Teaching Toolkit: I'll give it a few more seconds.

22
00:05:12.330 --> 00:05:25.259
BSPH CTL Teaching Toolkit: Alright! So looks like everyone who answered uses rubrics. So that's great. So that means that you're already familiar with some of the concepts that we'll cover today. And that also leads me to my next question, which is.

23
00:05:25.260 --> 00:05:46.259
BSPH CTL Teaching Toolkit: if you use rubrics in your course, what benefits have you experienced while using rubrics? And you can post this in the chat, or if you're comfortable, you can raise your hand and share here on Zoom. So if you're comfortable again, let us know what rubrics have you experien? What benefits have you experienced while using rubrics in your course?

24
00:05:46.460 --> 00:05:51.720
BSPH CTL Teaching Toolkit: Goodness, I'm like a little tongue tied today. Celine.

25
00:05:52.840 --> 00:05:59.129
Celine Greene (CTL): so prior to joining CTO. I was a faculty member, teaching mathematics and computer science.

26
00:05:59.320 --> 00:06:02.150
Celine Greene (CTL): and when there was something that was

27
00:06:02.390 --> 00:06:19.190
Celine Greene (CTL): done in different ways, the rubric kept me on course and free from distractions as to how as to how students might approach something like the reports that use different Internet tools in the computer science course, so it kept me as a faculty member on course, and it also

28
00:06:19.440 --> 00:06:31.900
Celine Greene (CTL): served as a wonderful thing when students I don't know if you know this, but students like to argue their grades sometimes, but it was a really good tool for me as well as for the students, to have a clear course of what we were reaching for.

29
00:06:32.050 --> 00:06:41.520
BSPH CTL Teaching Toolkit: That's great, that's great. And looks like Kathy posted in the chat. More objective, consistent feedback across students. So both of those are really great benefits to highlight

30
00:06:42.680 --> 00:06:47.410
BSPH CTL Teaching Toolkit: alright. And then, if there's no one else, as I said, those were

31
00:06:47.550 --> 00:06:55.139
BSPH CTL Teaching Toolkit: some oh, and I just saw another thing coming on the chat. So Anne said, I think the rubrics have brought more uniformity in our greeting.

32
00:06:55.230 --> 00:07:12.100
BSPH CTL Teaching Toolkit: Our course has 2 faculty and multiple teaching assistants. So the rubric helps helps us grade more reliably. Yes, that's something like 100%. I agree. That's actually something we're gonna talk about today where rubrics can really help with that interrator reliability across multiple graders

33
00:07:12.580 --> 00:07:38.929
BSPH CTL Teaching Toolkit: on Mia added transparency and expectations, clear expectations. Yes, you guys are all hitting things that we'll talk about as we look at more, even more benefits. So as we mentioned, there's many benefits to using rubrics, and I'm gonna highlight some of the benefits that I've seen, and some that really resonate with the faculty and tas that I've worked with here at BS. Ph. And the first one is alignment with your learning objectives.

34
00:07:38.930 --> 00:07:58.069
BSPH CTL Teaching Toolkit: And this I think, Kathy was one who mentioned learning objectives. When you're using rubrics, usually you have to. Well, one of the first things you have to do is identify your criteria, and when you pair your criteria for your rubrics with your learning objectives. Then you can be sure that your assessments are they're mapping to a match up with

35
00:07:58.070 --> 00:08:09.710
BSPH CTL Teaching Toolkit: your learning objectives for the course. And that's really important, because that way. You know that your students are being assessed in a in a valid way, and that they're achieving those learning objectives.

36
00:08:11.260 --> 00:08:18.680
BSPH CTL Teaching Toolkit: And then another benefit is impartial and consistent grading. So

37
00:08:19.090 --> 00:08:45.129
BSPH CTL Teaching Toolkit: when we think about this. And again, this could be consistent grading across grading teams, or even just across one grader with multiple students. So there's so rubrics allow this consistency through reproducible scoring again through with a single grader, across multiple students and with multiple graders. And they add that impartiality because you're grading all of your students in the same way.

38
00:08:47.070 --> 00:09:08.059
BSPH CTL Teaching Toolkit: Rubrics can also help you with your time management. And this is one of my one of the things that really stands out to my, to some of the faculty that I work with where even though rubrics might take some time to develop upfront, they really do save time in the grading process. A well designed rubric can reduce time spent on grading, and they can also reduce

39
00:09:08.060 --> 00:09:17.889
BSPH CTL Teaching Toolkit: possible grade disputes because students can see how they're being assessed. So it can save you time and the grading process, as well as any possible grade dispute time.

40
00:09:19.590 --> 00:09:28.250
BSPH CTL Teaching Toolkit: And then I think mia mentioned clear expectations in the chat rubrics can inspire better student performance

41
00:09:28.270 --> 00:09:38.040
BSPH CTL Teaching Toolkit: in that rubrics that are given to students. They can can help students see how they're being assessed. So they know what's expected of them.

42
00:09:38.210 --> 00:09:50.709
BSPH CTL Teaching Toolkit: and then it also for some students they might look at a rubric, and it might motivate them to meet those standards, to meet those exceeds expectations, levels, and it can help them with their overall assessment process.

43
00:09:52.050 --> 00:10:12.900
BSPH CTL Teaching Toolkit: And then finally, rubrics can be used to show a clear demonstration of change. So, for example, say you have students who are writing a rough draft, and then a final paper. If you use the same rubric for both of those submissions, you can see how the students performance has hopefully improved over time. Or perhaps they've stayed the same. And hopefully they haven't

44
00:10:12.900 --> 00:10:24.809
BSPH CTL Teaching Toolkit: decreased in their level of performance. But you can see that change over time by comparing rubrics or you can see change or differences between students as well

45
00:10:24.830 --> 00:10:27.030
BSPH CTL Teaching Toolkit: as a form of learning analytics.

46
00:10:28.490 --> 00:10:49.409
BSPH CTL Teaching Toolkit: So again, those were just some of the benefits of rubrics. And I wanted to ask, using the same pull everywhere as we did before. Which benefit are you most excited about? So when you think of how rubrics might benefit you, what are you excited about for me? Personally, I'm really excited about the time management and the clear expectations.

47
00:10:59.650 --> 00:11:02.160
BSPH CTL Teaching Toolkit: We'll give that a few couple more seconds.

48
00:11:05.940 --> 00:11:14.950
BSPH CTL Teaching Toolkit: Looks like clear expectations is winning at 50, and then also alignment with learning objectives, and impartial and consistent grading.

49
00:11:23.270 --> 00:11:25.690
BSPH CTL Teaching Toolkit: Alright. 2 more seconds

50
00:11:26.580 --> 00:11:43.550
BSPH CTL Teaching Toolkit: alright. So it looks like impartial and consistent grading has the win, and that is, that's one of the best benefits, especially again, when you're talking about teaching teams where you have to make sure that you are grading consistently across, say, 2 faculty or faculty and teaching assistants. So that is one of the the

51
00:11:43.640 --> 00:11:46.380
BSPH CTL Teaching Toolkit: better benefits of using a rubric.

52
00:11:48.060 --> 00:11:57.419
BSPH CTL Teaching Toolkit: Alright. So before we go any further. Are there any questions about what we've? I know we haven't covered a whole lot. But do you have any questions so far?

53
00:11:58.620 --> 00:12:01.969
BSPH CTL Teaching Toolkit: You can raise your hand or post those in the chat.

54
00:12:05.260 --> 00:12:15.310
BSPH CTL Teaching Toolkit: Alright, I don't see anyone raising their hand, and I don't see anything in the chat. There'd be more. There's gonna be more QA. Opportunities later. So

55
00:12:15.490 --> 00:12:20.420
BSPH CTL Teaching Toolkit: So if you have questions later, feel free to post those. Oh, mia, I just saw your hand raised.

56
00:12:22.670 --> 00:12:35.729
mia lamm: Hey? Sorry. I don't know if you're gonna talk about it later, Amy, but since we're a small group I would love to hear more. If you have any recommendations for when to choose, what type of rubric, when?

57
00:12:35.730 --> 00:12:57.739
BSPH CTL Teaching Toolkit: Yes, so that's actually a great teaser for later where we're gonna go so next we'll look at how to design a rubric. And then after that, we're gonna talk about types of rubrics, and when to use them so that you can pick the best rubric for your grading strategy. So I'll say to your to answer your question. That's coming soon. Thanks, mia

58
00:12:59.020 --> 00:13:01.380
BSPH CTL Teaching Toolkit: alright, any other questions?

59
00:13:03.760 --> 00:13:04.670
BSPH CTL Teaching Toolkit: Okay?

60
00:13:05.700 --> 00:13:33.800
BSPH CTL Teaching Toolkit: So so now that we talked about the benefits of rubric, I wanted to talk about how to actually design a rubric by looking at its basic parts. So starting with the the task description. So the task description. That's just a fancy way of saying these are the instructions that you've given to students for whatever assessment you've assigned. And the reason why these are included on the rubric is to make sure that the rubric aligns with the instructions that were given to students because

61
00:13:33.890 --> 00:13:44.570
BSPH CTL Teaching Toolkit: one of the worst things that you could do is give students instructions and then assess them in a way that doesn't match with the instructions. So it's just a way to check to make sure that again, those are in alignment.

62
00:13:45.090 --> 00:14:10.470
BSPH CTL Teaching Toolkit: Also pro tip after you've created your rubric. If you share the rubric with students with the instructions, so say you have a an instruction page and course, plus then again, that can help motivate students and keep their keep their attention focused on on the assessment. So if you have a rubric. You can share that with students before before you're giving it to them for feedback. You can share it with the instructions.

63
00:14:10.780 --> 00:14:31.509
BSPH CTL Teaching Toolkit:  our next piece of our rubric are the criteria, and these are usually tied to your learning objectives, but they also include any other knowledge or skills that you want students to demonstrate during this assessment, so that could be the learning objectives for the course or for the module, and it could also be things like

64
00:14:31.510 --> 00:14:44.629
BSPH CTL Teaching Toolkit: their grammar and spelling and mechanics or style things like that. So think about what yours, what all you want your students to demonstrate. And that's those behaviors are what go into your criteria

65
00:14:46.460 --> 00:15:02.220
BSPH CTL Teaching Toolkit: next to your levels of performance. And these are just your create descriptions of your criteria across multiple levels. And usually these are 3 to 5 likert scale levels. So you could do like needs, improvement

66
00:15:02.770 --> 00:15:16.749
BSPH CTL Teaching Toolkit: satisfactory or meets expectations or exceeds expectations. You could use numbers like 1, 2, and 3, or letters but in any way to differentiate, these are your levels of performance.

67
00:15:17.510 --> 00:15:20.190
BSPH CTL Teaching Toolkit: and we'll talk more about those in a little bit.

68
00:15:20.300 --> 00:15:36.209
BSPH CTL Teaching Toolkit: and then finally, you should have a place for your score, and also any overall comments that you have for your students. And as you're writing your comments, remember earlier, when we talked about effective feedback, you want to make sure that your comments are specific and focused.

69
00:15:39.430 --> 00:15:55.680
BSPH CTL Teaching Toolkit: So when we're creating these, when we're putting all these basic parts of a rubric together, there are some common errors that you can avoid. And the first is having too many components. So ideally, a rubric should have 3 to 5 criteria and 3 to 5 levels of performance.

70
00:15:55.680 --> 00:16:18.300
BSPH CTL Teaching Toolkit: because once you get any more than that, and once it gets any bigger it can become unwieldy for you as the grader going through each each piece, but it can also be really overwhelming for students. If they receive this long rubric, then they might not know where to start. So too many components can hinder the efficiency of a rubric

71
00:16:18.490 --> 00:16:32.009
BSPH CTL Teaching Toolkit: similar to too many components having too much detail, can also hinder a rubrics. If efficiency. So if you find yourself writing entire paragraphs into your rubric.

72
00:16:32.140 --> 00:16:45.589
BSPH CTL Teaching Toolkit: Then I would suggest condensing that are being more concise and saving those long paragraphs in the comments below. So that the rubric itself doesn't become again. You don't want it to be too too unwieldy.

73
00:16:46.390 --> 00:17:05.630
BSPH CTL Teaching Toolkit: And then, finally, the most frequently seen common error that I see, is inconsistent language, especially across those levels of performance. You should. Your descriptions should use consistent language and then also consistent structure. And this is so common that we're gonna look at an example.

74
00:17:05.680 --> 00:17:15.969
BSPH CTL Teaching Toolkit: So here's our example. And in this example our criteria is, apply Apa style and writing. And I just went with something basic.

75
00:17:16.030 --> 00:17:22.410
BSPH CTL Teaching Toolkit: And we'll look at an inconsistently written levels of performance first.

76
00:17:22.619 --> 00:17:27.790
BSPH CTL Teaching Toolkit: So here we have needs. Improvement contained many styling errors.

77
00:17:28.220 --> 00:17:35.809
BSPH CTL Teaching Toolkit: satisfactory, generally used Apa style guide. and exceeds expectations. Style was well executed.

78
00:17:35.970 --> 00:17:39.680
So here you can see that they all sort of

79
00:17:39.860 --> 00:17:52.940
BSPH CTL Teaching Toolkit: apply to the criteria. They're all connected. But there's no clear delineation of how these build from build across each other. So let's look at these rewritten to be more consistent.

80
00:17:53.440 --> 00:18:04.150
BSPH CTL Teaching Toolkit: So here, in the consistent style in the consistent column. We're using the same language and structure to describe each level. So we'll look at those so, and for the consistent level

81
00:18:04.290 --> 00:18:15.680
BSPH CTL Teaching Toolkit: needs improvement, did not use Apa style, or contained many styling errors. Satisfactory used Apa style, but contained some minor errors

82
00:18:15.740 --> 00:18:23.259
BSPH CTL Teaching Toolkit: and exceeds expectations used Apa style with 0 errors. So again, you can see where we're using the same language.

83
00:18:23.560 --> 00:18:34.849
BSPH CTL Teaching Toolkit: same structure, and then you can clearly as a grader. You can clearly see where students might fall, and if a student were receiving this they would know exactly how they performed.

84
00:18:35.400 --> 00:18:42.410
BSPH CTL Teaching Toolkit: So again, using that consistent language. We'll just give you and your your students a clear understanding of how they performed.

85
00:18:44.580 --> 00:18:57.430
BSPH CTL Teaching Toolkit: Any questions about the basic parts of a rubric, or those common errors, or if you've used rubrics, and you have seen a common error that I did not cover, you can share that experience as well.

86
00:18:58.820 --> 00:19:00.559
BSPH CTL Teaching Toolkit: So any questions

87
00:19:03.790 --> 00:19:04.819
BSPH CTL Teaching Toolkit: sleep

88
00:19:05.060 --> 00:19:11.050
Celine Greene (CTL): alright. II do have a question, something that's always kind of fucked me a little bit, and it

89
00:19:11.160 --> 00:19:19.550
Celine Greene (CTL): flashed in my mind because of your common error. One is a lot of times I find

90
00:19:19.920 --> 00:19:22.990
Celine Greene (CTL): ourselves and and other faculty

91
00:19:23.130 --> 00:19:31.620
Celine Greene (CTL): put something in a rubric because they're used to seeing it in a rubric. But what is that? Why is that criteria in the rubric? I almost feel like

92
00:19:32.090 --> 00:19:37.200
Celine Greene (CTL): like there could be 2 versions of a rubric or or one. And with this

93
00:19:37.550 --> 00:19:47.749
Celine Greene (CTL): like in your example, that we're seeing right here with this fifth column that is rationale like Apa style. What does that have to do with

94
00:19:47.890 --> 00:19:59.049
Celine Greene (CTL): a certain course if it was just an essay or something like that. Maybe if I just did the parenthetical citation, and I wasn't doing it, Apa. But I was doing it. Mla was Apa.

95
00:19:59.120 --> 00:20:09.579
Celine Greene (CTL): The actual measurement and the rationale might be a core competency of that program or something like that. So that that is something that I often

96
00:20:09.900 --> 00:20:36.649
Celine Greene (CTL): wonder about, and I think we should just go make a new rubric. But that's and I and I will say so. It's not. I don't know that it's necessarily an error, but I think that that's one of the things that as faculty constructing rubrics, we kind of get caught up in. That is kind of a little bit of an error as we take the focus off of what we're trying to see students demonstrate, and putting the focus on the box that we think everything should fit into. If that makes sense, II don't know.

97
00:20:36.730 --> 00:20:57.200
BSPH CTL Teaching Toolkit: Yes, yeah, that's a that's an excellent point to make and I think that also ties in with that too many components. Because when you're looking at a a rubric ideal. I think the ideal number was 3 to 5 criteria. But if you have a bunch of those criteria that you think should be there, but there might, but they might not be tied directly. To say the learning objectives.

98
00:20:57.220 --> 00:21:05.560
BSPH CTL Teaching Toolkit: then that would be a good place to start is, what are your learning objectives. What specifically or competencies do you need students to

99
00:21:05.580 --> 00:21:22.010
BSPH CTL Teaching Toolkit: to demonstrate? And then, if there's something that doesn't map to that, then that can that can indicate, hey? Maybe that criteria doesn't need to be there, or or if you have a learning objective that's not represented in the rubric, then that would need to be added. So that's that's a really great point.

100
00:21:22.420 --> 00:21:28.359
Celine Greene (CTL): It's just a different workshop. But anyway, thank you, that'll be part 2

101
00:21:28.540 --> 00:21:31.429
BSPH CTL Teaching Toolkit: alright any other questions or comments.

102
00:21:33.910 --> 00:21:39.370
BSPH CTL Teaching Toolkit: Alright. Well, and again you can post questions in the chat as well.

103
00:21:40.120 --> 00:22:09.159
BSPH CTL Teaching Toolkit: Alright. So now that we've gone over the basic parts of a rubric. I'm going to talk about 3 specific types of commonly used rubrics. And when they should be used, based on your grading strategy needs and some of the needs that will highlight are things like class enrollment size, grading time. And then also whether or not there'll be one grader or a team of graders. So those are some of the things that will play into what grading strategy you might use.

104
00:22:10.570 --> 00:22:17.390
BSPH CTL Teaching Toolkit: And again, there's many different types of rubrics. But II pulled out these 3 most common commonly used types.

105
00:22:17.590 --> 00:22:47.130
BSPH CTL Teaching Toolkit: So the first one is analytical, and this is the the bread and butter of rubrics. It's the one that I think most people think of when they think of rubrics, and it's perhaps the most commonly used type. And this one meets all of the descriptions that we've gone over so far. They have multiple criteria described against multiple levels of performance with the task description and sport and comments. So it's like this is the basic basic rubric

106
00:22:48.100 --> 00:23:16.489
BSPH CTL Teaching Toolkit: and there's a reason why it's the most the most frequently used. And that's because it can provide a lot of prewritten feedback to a lot of students. And even though it takes some time to develop up front. It takes the shortest amount of time to actually use when you're grading. And I found this really great quote, by Educator Heidi Goodridge Andre, where she explains quote, if I were to simply circle boxes on a rubric, analytic rubric

107
00:23:16.560 --> 00:23:37.370
BSPH CTL Teaching Toolkit: and give it back within an assignment, I would still be providing more feedback about the strengths and weaknesses of the work than if I had just assigned a letter grade, and it would not have taken me any longer end quote. So I thought that was a really great summary of the the greatest strength of an analytical rubric. In that, again, it gives a lot of feedback in a short amount of time.

108
00:23:39.300 --> 00:23:54.930
BSPH CTL Teaching Toolkit: So our next type of rubric is our single point rubric. And this is almost identical to your analytic rubric, except that only one level of performance is actually filled in and the other levels of performance are intentionally left blank.

109
00:23:55.260 --> 00:24:09.430
BSPH CTL Teaching Toolkit: And this means that when you're using this type of rubric. This is ideal for scenarios where you have smaller class sizes. But you want to give really detailed feedback to students where if students have

110
00:24:09.550 --> 00:24:22.820
BSPH CTL Teaching Toolkit: exceeded expectation, or if they've if they need to improve then this type of a rubric is really great. If you need to give really again, that really specific feedback to students that wouldn't be pre-written.

111
00:24:23.420 --> 00:24:43.859
BSPH CTL Teaching Toolkit: Again, this is mostly used for for short, small classes that you wanna provide a lot of detailed feedback for it takes the shortest amount of time to develop. But the trade off. It's sort of again, the opposite of the analytic rubric. It takes the longest time to grade, and that's because, you, as the grader would fill in

112
00:24:43.860 --> 00:25:04.390
BSPH CTL Teaching Toolkit: those other levels, depending on where the student fell on the rubric. So if they met that filled in level that meets expectations, then you can just circle it like an analytic rubric. But again, if they exceeded or needed to improve, then the grader would have to fill that in, so it takes more the longest time to grade compared to the other rubrics that we're talking about today.

113
00:25:04.780 --> 00:25:09.130
BSPH CTL Teaching Toolkit: Another weakness is that because you're filling in

114
00:25:09.690 --> 00:25:24.529
BSPH CTL Teaching Toolkit: feedback, that's not prewritten. It also takes some extra training for grading teams. So if you're using this and you have multiple faculty or a team of teaching assistants, then they would need to be trained on how to provide that feedback.

115
00:25:26.650 --> 00:25:51.359
BSPH CTL Teaching Toolkit: And then our third type of rubric is holistic rubric. And this one is quite different from the other 2 in that instead of breaking the assignment down across multiple criteria or parts. You're just looking at the assignment as a whole across 1, one criteria with multiple levels of performance. So again, instead of 3 to 5 criteria, you're just looking at one criteria. The assignment as a whole.

116
00:25:51.790 --> 00:26:18.760
BSPH CTL Teaching Toolkit: and the main advantage of using a holistic rubric is that it's pretty easy on the grader creating a holistic rubric takes less time than the other rubrics to develop, and then grading with one is faster, too, because you're just looking at the students overall performance as a whole. So you're not picking out specific things that the student did or performed. You're just looking like, Oh, yeah, they passed, or they did. Well, they didn't do well.

117
00:26:19.020 --> 00:26:38.329
BSPH CTL Teaching Toolkit: this is usually used. It's great for things like a final project or a final paper where students have already received feedback, say on rough drafts. And now you're just giving them their final grade, or they're great for the whole project. So again, it's its strengths are that it summarizes multiple achievements as a whole.

118
00:26:38.330 --> 00:26:52.280
BSPH CTL Teaching Toolkit: and it takes the shortest amount of time to develop, and it also takes the shortest amount of time to actually use. But its weakness is that it's feedback is not as detailed or targeted again, because you're just giving them an overarching grade versus

119
00:26:52.860 --> 00:26:55.190
BSPH CTL Teaching Toolkit: breaking their performance into parts

120
00:26:55.460 --> 00:27:01.719
BSPH CTL Teaching Toolkit: so it might not be as detailed or targeted and that can lead to questions from students.

121
00:27:03.930 --> 00:27:21.660
BSPH CTL Teaching Toolkit: So so now that we've kind of talked about those 3 types of rubrics, we're gonna talk about some. I'm gonna give you some scenarios, and you can tell me which rubric would you choose for each each of these? So but before we do that, are there any questions about those 3 types of rubrics?

122
00:27:25.210 --> 00:27:38.450
BSPH CTL Teaching Toolkit: See? I don't see any questions. But if you have questions as we're doing, these, feel free to to ask but so so our first scenario is you're teaching a high level class with 12 students. So it's a small class.

123
00:27:38.640 --> 00:27:51.230
BSPH CTL Teaching Toolkit: and your students are completing an assessment that requires a lot of specific feedback. So based on this, which rubric would you choose and feel free? You can post in the chat, or you can raise your hand and

124
00:27:51.600 --> 00:27:54.230
BSPH CTL Teaching Toolkit:  share verbally.

125
00:27:56.400 --> 00:28:02.269
Celine Greene (CTL):  can you go back and show us it's holistic, analytic, and

126
00:28:02.410 --> 00:28:08.640
BSPH CTL Teaching Toolkit: single point. No, thank you. I couldn't remember the name of the last one. Thank you.

127
00:28:09.000 --> 00:28:26.629
BSPH CTL Teaching Toolkit: Yeah. Oh, and mia, posted in the chat single point. They use this for largest enrollment class. But we have a teaching team. So it's challenging to norm across the team. Yes, that's that's exactly right. So yes. So for this one. This is a good scenario for your single point rubric, as mia mentioned in the chat.

128
00:28:26.630 --> 00:28:39.489
BSPH CTL Teaching Toolkit: And in this case it's a smaller class size. So having to write that extra feedback in is manageable. And again, if the assessment requires a lot of specific feedback. That's the

129
00:28:39.530 --> 00:28:47.780
BSPH CTL Teaching Toolkit: the main benefit of this type of rubric that it can provide a lot of feedback. I just realized I didn't show there's the answer single point.

130
00:28:48.340 --> 00:29:03.020
BSPH CTL Teaching Toolkit: Alright, and in addition to this, you could use an analytic rubric. But again, since specifically, they wanted a lot of specific feedback. Single point, was the better answer, even though they both would be correct.

131
00:29:04.580 --> 00:29:20.159
BSPH CTL Teaching Toolkit: Alright. So here's our second scenario. And in this one you're teaching a course with 100 students. So larger, larger team and you and a team of teaching assistants or tas are grading rough traps of a paper, and the rough traps should provide a lot of feedback.

132
00:29:20.170 --> 00:29:25.310
BSPH CTL Teaching Toolkit: So based on this, what do you think would be the best rubric to use? Which would you choose?

133
00:29:36.300 --> 00:29:43.540
BSPH CTL Teaching Toolkit: And I just put the 30, I spelled single point wrong. I just put the 3 types in the chat so you could see what they

134
00:29:44.840 --> 00:29:46.000
BSPH CTL Teaching Toolkit: what they are.

135
00:29:46.150 --> 00:30:04.680
BSPH CTL Teaching Toolkit: Let's see, we have a couple in the chat. We have 2 people who said analytic one is at single point. So here if I were the id for this class. I would recommend analytic only because there's 100 students. But, as Mia said earlier in her chat, you could do this with a single point, rubrick

136
00:30:04.680 --> 00:30:22.720
BSPH CTL Teaching Toolkit: especially since you're looking to provide a lot of feedback. But just know that you would need some additional grade norming training to make sure that everyone is giving effective feedback in the same ways. So so I would recommend analytic but but you could also use single point with some additional training for your teaching team

137
00:30:23.400 --> 00:30:27.380
BSPH CTL Teaching Toolkit: alright. And then our final scenario.

138
00:30:27.460 --> 00:30:48.510
BSPH CTL Teaching Toolkit: you teach a course with 100 students, you and a team of teaching assistants are grading a final, the final draft of a paper the students have already received feedback on their rough drafts, and only need a final score for their overall achievement. So which rubric would you choose? And also by elimination, this one might be a little easier to answer

139
00:30:50.120 --> 00:30:52.769
BSPH CTL Teaching Toolkit: Holistic. That's correct.

140
00:30:53.130 --> 00:31:10.080
BSPH CTL Teaching Toolkit: So yes, so the answer for this one is holistic. And again, that's because students have already received feedback on their draft. So they don't need that really specific targeted feedback. And they're just looking for that overall final score that summarizes their achievements.

141
00:31:11.890 --> 00:31:33.880
BSPH CTL Teaching Toolkit: Very good. So when you think about your course. Think about again. How many students you have to grade? Will you be grading by yourself, or will you be grading with others? And then what are the feedback needs of your students? So will they need a lot of detailed feedback. Will they just need some pre-written feedback, or will they need just an overarching score.

142
00:31:33.960 --> 00:31:43.030
BSPH CTL Teaching Toolkit: And with those things in mind we'll answer our our next pull everywhere question, which is which rubric best suits your grading needs.

143
00:31:46.790 --> 00:31:51.930
BSPH CTL Teaching Toolkit: And it's okay. If you select not sure we'll talk about that next.

144
00:32:15.030 --> 00:32:18.300
BSPH CTL Teaching Toolkit: Alright, I'll give it one more a couple more seconds.

145
00:32:18.700 --> 00:32:32.149
BSPH CTL Teaching Toolkit: So it looks like we're we have, most. We have some analytics and single points. No, no holistics so holistic did not win this this one. And then we have a couple that are not sure. And that's okay.

146
00:32:32.260 --> 00:33:09.939
BSPH CTL Teaching Toolkit: So we can. I will have plenty. It looks like we'll have plenty of time to talk at the end about some specific courses. But But in the meantime I also wanted to highlight that as you're thinking about incorporating rubrics into your course. You can meet with the Ctl instructional designer for grading and assessment consultation, and they can help you if you know what type of rubric you want to? You want to use. They can help you create that or if you're not sure what kind of rubric would be best for you. We can also look at your specific assessments and make some recommendations based on your grading needs.

147
00:33:10.070 --> 00:33:19.089
BSPH CTL Teaching Toolkit: so for that one, the the person who said, not sure I'd recommend reaching out to Ctl. Because we're here. We're we're here to help help with that.

148
00:33:20.270 --> 00:33:44.190
BSPH CTL Teaching Toolkit: I also wanted to mention. So we have a new tool, in course, plus it's a a rubric feature in the course plus grade book. That talks. That is a way that you can actually incorporate your rubrics directly into your course, plus website. So once you've designed your rubric and you're ready to put it into your course plus site. You can use this new rubric feature. Directly with your grade book tool.

149
00:33:44.310 --> 00:33:47.159
BSPH CTL Teaching Toolkit: I'm not gonna go into step by step.

150
00:33:47.300 --> 00:34:09.950
BSPH CTL Teaching Toolkit: Instructions on how to add your rubric, because we have this really great blog post on how to incorporate your rubric, your rubric into gradebook. So I'll share the link to this blog post in the chat, so that, and again, it'll give you some, step by step, instructions on how to do that. Thank you, Celine. Thanks for posting that.

151
00:34:12.270 --> 00:34:26.169
BSPH CTL Teaching Toolkit: I also wanted to highlight some other resources that are just really great for grading and assessment. So we have a teaching toolkit website, and we have an entire page on grading and assessment. So

152
00:34:26.400 --> 00:34:28.359
I highly recommend

153
00:34:28.460 --> 00:34:44.790
BSPH CTL Teaching Toolkit: checking that out. We also again, if you want to contact Ctl. For a consultation, then you can open a help call and we can set up a meeting with Ctl. Oh, Celine, I saw your hand is raised.

154
00:34:44.810 --> 00:34:57.250
Celine Greene (CTL): Yeah, thanks, Amy. I just wanted to point out that the peer assessment tool inside, of course, plus is also a rubric. For again, self evaluation for students and peers, and can

155
00:34:57.740 --> 00:35:05.449
Celine Greene (CTL): mimic the great book, one. That faculty teach faculty teaching the faculty. Ta

156
00:35:05.480 --> 00:35:25.450
Celine Greene (CTL): used to have a rubric of its own in peer assessment. But now that's incorporated in the grade book. But again, the the 2 can mirror each other if you wanted to. Yeah, actually, that's a great, that's a great point. And to add on to that, say, you want students to self assess the their work before, maybe, for like a rough draft before they get

157
00:35:25.710 --> 00:35:34.770
BSPH CTL Teaching Toolkit: your your feedback and a final draft, you can set up a peer assessment where it's just a self-assessment. So that's a really good point. Thank you, Celine.

158
00:35:36.010 --> 00:35:38.790
BSPH CTL Teaching Toolkit: And then I'm going to

159
00:35:38.980 --> 00:35:48.940
BSPH CTL Teaching Toolkit: just open the floor now because we have. We have plenty of time. II thought we would take a little bit longer on the activities. But I just wanted to say

160
00:35:49.150 --> 00:36:00.019
BSPH CTL Teaching Toolkit: in conclusion, rubrics are a great tool for you and your teaching teams to deliver effective feedback and I hope that you reap the benefits of using rubrics in your course.

161
00:36:00.170 --> 00:36:29.740
BSPH CTL Teaching Toolkit: so I'm gonna just open the floor now. So if anyone has questions about their specific assessments or how they might might incorporate rubrics into your course. We can chat about your course specifically, or if you have an assessment in mind, but before otherwise, if you don't have any questions or anything specific, you'd like to talk about. That concludes our workshop. But before you go I'd like you to take a few minutes. Today's session was all about feedback.

162
00:36:29.740 --> 00:36:38.389
BSPH CTL Teaching Toolkit: So please let me know how I did by completing this very short, anonymous workshop evaluation survey. And and I see you've raised your hand.

163
00:36:39.650 --> 00:36:45.909
BSPH CTL Teaching Toolkit: and I'll just leave this on the screen so that you can fill out the survey as as you wish. But, Anne, go ahead.

164
00:36:47.250 --> 00:36:53.880
Anne Lilly: Sure. I just wanted to say, Thank you. This is great. It was very helpful.

165
00:36:53.910 --> 00:37:06.629
we use rubrics in our class. I teach. I Co. Teach a course, in school public health. It's actually ongoing right now. It has 90 students. We use analytic rubrics for our assignments.

166
00:37:06.910 --> 00:37:16.130
Anne Lilly:  and overall. It works well, and I like, I said in chat, I think it does lead to greater uniformity, greater consistency.

167
00:37:16.680 --> 00:37:37.770
Anne Lilly: One thing that we've observed this year in particular is that? You know our, I wanna make sure I'm using the correct terminology here with our levels of performance. For, like a given criteria and our rubric. We have the score jump, I think, by like 2 points per level of performance.

168
00:37:38.020 --> 00:37:45.790
Anne Lilly: And basically, when, if you do that, and you actually follow the score. For that level of performance.

169
00:37:45.820 --> 00:37:57.100
Anne Lilly: When you look at the final grade given, it's often lower than like the overall quality of that assignment is so

170
00:37:57.410 --> 00:38:16.490
Anne Lilly: and sorry. I know this is like very much in the lead. But basically, we have our especially our tas. They want to give like fair scores. They're often grading peers. So fair enough, and I appreciate that mentality. But I guess we're wondering.

171
00:38:16.540 --> 00:38:26.119
Anne Lilly: I feel like we need to recalibrate our rubric, because if you follow it as it is written, and you deduct the number of points you're supposed to for a given level of performance

172
00:38:26.250 --> 00:38:32.810
Anne Lilly: grades end up really like lower than make sense. Given

173
00:38:32.830 --> 00:38:36.199
what the overall performance was actually like.

174
00:38:36.640 --> 00:38:48.550
Anne Lilly: So I guess I'm just wondering like, is that something that happens. And when we created the levels of performance, I think we wanted them to range all the way down to 0.

175
00:38:48.670 --> 00:38:56.100
Anne Lilly: Which it that likely doesn't need to be true. But that's just kind of, I think, what made sense to us at the time.

176
00:38:56.250 --> 00:39:00.110
So I guess, can levels of performance just range down to like

177
00:39:00.440 --> 00:39:25.669
BSPH CTL Teaching Toolkit: 6 points or some random number so that our teaching team could be following the rubric more faithfully, I guess. Yes, yes, so so there's I think there were 2 2 or 3 questions in there. One is should this be recalibrated? And the answer is. Yes, it sounds like it. Secondly, it sounds like

178
00:39:25.680 --> 00:39:39.279
BSPH CTL Teaching Toolkit:  so the that final that needs improvement level or the lowest level. You don't have to start at 0. You can start at whatever number makes sense for you. And I've even seen like other ones, might have

179
00:39:39.370 --> 00:39:49.660
BSPH CTL Teaching Toolkit:  specific points. And then the last column I've seen like 0 2, you know, 10 or 0 to 5. So that

180
00:39:49.910 --> 00:40:09.469
BSPH CTL Teaching Toolkit: you know, you're still technically covering. It's like on a graph when you have the little squiggle line, and then you start the actual graph where technically they could score less. But usually it's going to be like 6 points or 10 points, or however many points would make sense for that scale. But for your specific rubrick I think

181
00:40:10.290 --> 00:40:25.569
BSPH CTL Teaching Toolkit: I don't know if you wanna pull it up or anything but I would probably recommend having meeting with an Id to. Really, we can open it up with you, and make some recommendations based on after we can read over the rubric and help maybe recalibrate it.

182
00:40:25.740 --> 00:40:30.520
Anne Lilly:  can II was just gonna say, because in in

183
00:40:31.630 --> 00:40:34.580
Celine Greene (CTL): in concert with what you were saying, Anne.

184
00:40:34.600 --> 00:40:58.319
Celine Greene (CTL): so one of the things that I again found was like some of the weakest papers. When I again it was like a intro to Comp. Side where where they had to actually do this little research project. But they had to use all these tools, and some of the weakest papers scored so well on the rubrics cause they were meeting and meeting what the rubric was. Now this was a different school. This is a community college, and

185
00:40:58.400 --> 00:41:14.140
Celine Greene (CTL): the Id team members will enjoy this. It was a quality matters course approved. So there was only, but so much I could change of the course without going through this whole rigmarole. And that grading system, including that rubric, was part of that course's design.

186
00:41:14.200 --> 00:41:36.130
Celine Greene (CTL): Now, if I had that extra hidden column that I was Tai talking about with Amy, it would have helped me, I think, as a grader in both, addressing like maybe revising and refining what I was looking for. If I could have had that liberty to change the rubric, because with the whole again. We're I felt like

187
00:41:36.310 --> 00:41:58.359
Celine Greene (CTL): when I was creating rubrics off the cuff. This is 17 years ago, a much younger, greener person that I am now. I felt like I was just doing what was expected, even in creating the rubric so much as the style guidelines. I'm like, what if if I had been in the mindset. And again, I couldn't do it with that particular course.

188
00:41:58.360 --> 00:42:22.280
Celine Greene (CTL): But I think if you, as faculty, say, what is it that I real? Why am I asking them to meet this criteria? Why am I asking them to have this performance level, and that will allow you to hone and draw out better demonstrations of learning like the higher quality. That's not just quote unquote meeting that benchmark. So I just I just know again from personal experience.

189
00:42:22.560 --> 00:42:42.899
Celine Greene (CTL): knowing what I know now versus then. If I had that liberty I would have designed a better quality rubric for myself to bring out better, I think, demonstrations of learning from the students. So, and and unless I misinterpreted you, but that was part of what I was hearing was that they're scoring, but it really doesn't match

190
00:42:43.120 --> 00:42:54.230
Celine Greene (CTL): the quality of the of the product. The score doesn't necessarily equate to what I would see without the rubric. Is that correct cause? I hear you correctly.

191
00:42:54.300 --> 00:42:59.129
We're, I think we're straying in the opposite direction. People were like.

192
00:42:59.200 --> 00:43:08.340
Anne Lilly: would they end up with like an 80 when it's like, actually, this is like an 85, or like an 80, it should be higher in 80. So we were kind of straying in the other direction. But yes.

193
00:43:08.360 --> 00:43:17.780
BSPH CTL Teaching Toolkit: it's not accurately reflecting overall quality. Oh, sorry. Go ahead.

194
00:43:18.710 --> 00:43:36.559
BSPH CTL Teaching Toolkit: Oh, I was. Gonna say it might be helpful because you have an existing rubric, and you've created papers against it. It might be helpful to pull out papers like, you know, this paper should be an 85. You know, this paper should be a 100. This paper should be a 90 and if you can look at those papers

195
00:43:36.560 --> 00:43:56.109
BSPH CTL Teaching Toolkit: and pull out, why, they're that great, then that can also help you recalibrate your your rubric so that it's like, if this is an 85 paper, then you can sort of work backwards and modify your rubrics so that it would. If you were using that rubric with that paper it would come out to 85, and then you can use that as a grade norming

196
00:43:56.110 --> 00:43:59.410
BSPH CTL Teaching Toolkit: paper for your teaching team. Yeah.

197
00:44:00.080 --> 00:44:21.200
Anne Lilly: yeah, and that actually gets at my second question, do you? So again, this is super helpful information. And like sort of conceptualizing what kind of rubric you need and how you might go about designing it. But when you have a teaching team and you are working to get to reliability in using a rubric.

198
00:44:21.240 --> 00:44:30.219
Anne Lilly: Do you guys have best practices from Ctl in terms of like getting to reliability as a group with a rubric.

199
00:44:30.420 --> 00:44:45.119
BSPH CTL Teaching Toolkit: Yes. So, yeah. Yeah. So usually, it involves having the the whole team come in for like a training session, or maybe even multiple training sessions, depending on what types of things that they'll be creating. And it's

200
00:44:45.120 --> 00:45:02.169
BSPH CTL Teaching Toolkit: the the most ideal situation would be to bring in like a past paper or a past presentation. Have them? Have them go through like. How would you grade this and then say, Well, this is how I would grade this and then explain the rationale why and

201
00:45:02.170 --> 00:45:22.439
BSPH CTL Teaching Toolkit: and then train them to grade it like you would grade it so that their grading methods match yours, and then maybe do that one, or you know, one or 2 times, or maybe even a third time, depending on how off their grading was compared to how you would grade something until they they start grading in the same way, or using the same rationality that you that you intend

202
00:45:22.440 --> 00:45:42.170
BSPH CTL Teaching Toolkit: to use for your grading. And I know that at least for one course. Ids have been involved. Instructional designers from Ctl have been involved with that. We had a course. I believe it was me Lou and Kathy who are here today. We had a course where there were, I think, 25 tas who would be grading.

203
00:45:42.260 --> 00:46:07.150
BSPH CTL Teaching Toolkit: So in that case, they had to really make sure that everyone, all the tas were consistently trained. And we took them through, I think at least 2 training sessions. To make sure that they were that they were ready to go for the grading part. And then also just the general how they would be taking that class. Lou and Kathy, do you have anything you wanted? You might want to add about that experience with the problem solving course.

204
00:46:10.470 --> 00:46:16.420
Kathy Gresh: No, I would just say it is a challenge. Especially with such a big team. But

205
00:46:16.440 --> 00:46:17.450
Kathy Gresh: I was

206
00:46:17.480 --> 00:46:47.460
Kathy Gresh: say, it is really worth it in the end. Run II don't know how many times I've seen comments and course, evaluations about different graders grading differently and things like that. And even within our own team, like, we're planning really soon to do sort of a type of an exercise where we're going to be doing a norming exercise among our own team. It's not on. It's on a different type of rubric. But it definitely is a worthwhile activity. And

207
00:46:47.460 --> 00:47:00.689
Kathy Gresh: I think I think people learn a lot. I think that it's also a great learning mechanism for teaching assistance as they participate in it. So for their future. So

208
00:47:01.560 --> 00:47:06.309
BSPH CTL Teaching Toolkit: I just had a thought that might be a great workshop, a future workshop on like, maybe

209
00:47:06.380 --> 00:47:33.269
BSPH CTL Teaching Toolkit: grade norming how to grade norm and then maybe breakout rooms. Yeah. So I'll I'll write that down. Please do sign me up. We do, because we do have a process for doing this because both my Co instructor and I like, we do qualitative data work. So we know about getting people to reliability and like a qualitative research sense. But I think

210
00:47:33.490 --> 00:47:37.489
Anne Lilly: I do think we could do a better job of getting people reliable.

211
00:47:37.500 --> 00:47:38.669
and doing it

212
00:47:38.730 --> 00:47:45.099
Anne Lilly: quicker. But anyway, I any yeah. Another session about that would be appreciated.

213
00:47:45.300 --> 00:47:55.620
Celine Greene (CTL): Alright, I will workshop. Just question

214
00:47:56.470 --> 00:48:03.629
Celine Greene (CTL): are the courses larger or smaller, that you're working with like in terms of student and roster size.

215
00:48:03.920 --> 00:48:10.559
Celine Greene (CTL): Is it like over. you know, over 50 under 50, that kind of thing

216
00:48:11.480 --> 00:48:17.009
Anne Lilly: we have? Last year we had about 100 this year. We have 90.

217
00:48:17.240 --> 00:48:32.330
Celine Greene (CTL): Alright, yeah, I'm thinking, in small. I was just thinking just as a total aside like brainstorming, and it wouldn't work for a larger course, but something that was rubric based. You could have like a grade book category where there are actually 3.

218
00:48:32.400 --> 00:48:59.249
Celine Greene (CTL): Quote unquote rubric assignments is the same exact item, but one's faculty, one one's faculty 2. And the third is the one that you decide is agreeable to share with the student, if that was how it was working. But again, with such a large course that would never work. But that's that's the only reason I asked, because I was just thinking I'm like, Hey, we get the norming when we do the peer assessments. But we don't have that option just yet for faculty assessments. But

219
00:48:59.250 --> 00:49:05.250
Celine Greene (CTL): I don't know. Maybe there's a a course plus future build. And

220
00:49:06.760 --> 00:49:33.819
BSPH CTL Teaching Toolkit: yeah, yeah, I like, I kinda like the idea of having, especially for classes that have multiple graders. If the same graders are, you know, if you have, like 3 readers for one paper, it would be kind of cool to see how those scores come together, so it might be really dependent on on class size. And but I do think that then that goes into the whole, the great norming and the teaching. It goes back into the

221
00:49:33.930 --> 00:49:34.970
Celine Greene (CTL): and

222
00:49:35.430 --> 00:49:48.260
Celine Greene (CTL): equity of student grading like, if it's, you know, am I always going to get this group of students? Well, I'm always going to grade this way, you know, when you talk with the problem solving course where there are 25 tas, and they have the grade norming. But

223
00:49:48.720 --> 00:49:59.600
Celine Greene (CTL): you know, students talk, or whatever I've got Amy as my grader. So I know that it's it's gonna be easy to get this score in this direction, or whatever. I don't know.

224
00:49:59.650 --> 00:50:01.410
BSPH CTL Teaching Toolkit: Yeah.

225
00:50:03.960 --> 00:50:12.380
BSPH CTL Teaching Toolkit: Any other questions, or just comments, or just. If any of you have used rubrics in the past, any lessons learned or recommendations?

226
00:50:15.430 --> 00:50:16.940
BSPH CTL Teaching Toolkit: If not, that's okay.

227
00:50:20.350 --> 00:50:23.110
BSPH CTL Teaching Toolkit: Alright. Well, it sounds like

228
00:50:23.130 --> 00:50:41.139
BSPH CTL Teaching Toolkit: I don't see any hands raised, and it looks like there's nothing new in the chat. So I'm gonna go ahead and conclude the session. But it was very nice speaking with all of you, and oh, and I see you just unmuted sorry

229
00:50:41.370 --> 00:50:43.340
I didn't get to

230
00:50:43.360 --> 00:50:44.560
Anne Lilly: taking care of.

231
00:50:44.680 --> 00:50:45.380
BSPH CTL Teaching Toolkit: Yep.

232
00:50:46.170 --> 00:50:48.900
Celine Greene (CTL): and it's also in the chat if you wanted to click on it.

233
00:50:49.600 --> 00:50:50.620
Anne Lilly: Okay.

234
00:50:51.790 --> 00:50:55.770
BSPH CTL Teaching Toolkit: yeah. And that's very much appreciated. Yup.

235
00:50:56.240 --> 00:51:20.110
BSPH CTL Teaching Toolkit: alright, and then otherwise again. Thank you so much for attending today's session. I. We are ending with a couple of minutes to spare. So again, thanks again. And I hope you found this helpful, and feel free to reach out to Ctl. If you have any more questions about rubrics, or if you'd like to meet up for a consultation. And again we can look at your specific assessments and offer some recommendations.

236
00:51:22.270 --> 00:51:23.420
Celine Greene (CTL): Thanks, Amy.

237
00:51:23.810 --> 00:51:29.249
BSPH CTL Teaching Toolkit: thanks, and thank you, Selene. So much for keeping track of chat and everything. Thanks for your help.