WEBVTT 1 00:00:02.570 --> 00:00:13.079 BSPH CTL Teaching Toolkit: Alright well, hello! And welcome everyone to today's center for teaching and learning teaching toolkit workshop on analyzing student course evaluations. 2 00:00:13.080 --> 00:00:36.300 BSPH CTL Teaching Toolkit: My name is Amy Pinkerton, and I'm a senior instructional designer at Ctl. And I'm joined by my colleague Lauren Dana, who is an instructional designer at Ctl and we just wanted to say, welcome to today's session. It is being recorded, and the recording, along with all of the resources shared today, will be shared with all of the Zoom registrants via email after the workshop 3 00:00:36.470 --> 00:00:49.060 BSPH CTL Teaching Toolkit: and I also wanted to mention that if you have any questions you can post those in the chat. Lauren and I will be monitoring the chats. And there will also be opportunities for QA. Throughout today's session. 4 00:00:50.690 --> 00:00:59.519 BSPH CTL Teaching Toolkit: So specifically, we're going to first, look at feedback and course revision. And specifically, we're going to have a focus on student 5 00:00:59.670 --> 00:01:04.950 BSPH CTL Teaching Toolkit: course evaluation feedback and how that fits within the course revision process. 6 00:01:05.060 --> 00:01:19.899 BSPH CTL Teaching Toolkit: Then we're gonna look at some available types of course, data again, with a focus on the qualitative data that's available through student course evaluations. And then conclude with a hands-on activity and some recommended next steps. 7 00:01:20.200 --> 00:01:35.009 BSPH CTL Teaching Toolkit: We asked, that you bring your student course evaluation data to today's workshop. But if you don't have data to analyze, don't worry. We do have some sample data for you to to analyze for the hands on activity if you don't have your own. 8 00:01:39.780 --> 00:01:46.010 BSPH CTL Teaching Toolkit: Oh, Lauren, can you do me a favor and turn the waiting room off? Thank you thanks. 9 00:01:46.320 --> 00:02:12.289 BSPH CTL Teaching Toolkit: And then, specifically, these are the objectives for today's workshop. So at the end of this workshop, you'll be able to overview the course revision process, identify sources of information or sources. Of course, data to consider in the revision process identify strategies that help alleviate stress surrounding course, evaluation. And then finally analyze student course evaluation data to identify some specific course revisions. 10 00:02:14.740 --> 00:02:21.100 BSPH CTL Teaching Toolkit: So let's take a moment and identify why? Student course, evaluations matter. 11 00:02:21.640 --> 00:02:45.940 BSPH CTL Teaching Toolkit: And the reason why is because student student course evaluations. They're an important factor in determining the perceived quality of teaching and learning for a course, and they allow faculty to critically reflect on their teaching and their course, administration and implementation. So these can help faculty identify some possible updates to a course, identify pain points 12 00:02:45.940 --> 00:02:52.120 BSPH CTL Teaching Toolkit: things and also identify things that are going well and things that should continue in future offerings. 13 00:02:52.120 --> 00:03:10.350 BSPH CTL Teaching Toolkit: So so student course, evaluations, these are things that again, they're not just highlighting the negatives and things that need to be changed, but also identifying what is working well within a course. They can inspire professional development and also inspire your teaching efforts 14 00:03:10.800 --> 00:03:27.079 BSPH CTL Teaching Toolkit: beyond faculty student course, evaluations are also important for programmatic decisions. So they can be used by, for example, school administrators when making data, informed decisions about a course and how a course fits within an an overall program. 15 00:03:29.610 --> 00:03:30.680 BSPH CTL Teaching Toolkit: So 16 00:03:30.840 --> 00:03:45.800 BSPH CTL Teaching Toolkit: sometimes when we think about revision. It can be associated with negative thoughts, feelings like I put a lot of work into this, but my students don't like it. This was working in the past. Why isn't it working anymore? And so forth? 17 00:03:45.800 --> 00:04:10.769 BSPH CTL Teaching Toolkit: And when revision has this sort of the sort of negative connotation surrounding it? It can sour the entire process for you, and that's why I wanna take a moment to sort of refocus how we're looking at revision and re and reexamine it, and think of it more as refinement. Then then something that needs to be improved because it's, you know, wrong or bad, or anything like that. 18 00:04:10.770 --> 00:04:22.529 BSPH CTL Teaching Toolkit: So don't think of revision as a bad thing. It's a good sign if you're revising your course. That means that you're you're updating it based on your experiences and needs and the needs of your students. 19 00:04:22.640 --> 00:04:42.330 BSPH CTL Teaching Toolkit: So courses have this natural life span or life cycle, development, implementation, evaluation, revision and revision is just a part of that cycle, so don't feel bad if you have to revise your course. So again, think of it as refinement, like perfecting a recipe into something that tastes good and something that tastes delicious. 20 00:04:43.710 --> 00:04:48.850 BSPH CTL Teaching Toolkit: So I'm now going to pass things off to Lauren. Who's going to tell us more about the revision process. 21 00:04:49.160 --> 00:05:10.400 Laueren Dana: Thank you, Amy. So many of you. After reflecting upon your recent course offerings, you may want to decide to make course, revisions as Amy was talking about. These revisions may be minor. They may be major, or they may be somewhere in between, but regardless. We really hope that you will follow these steps as you begin to think about course revision. 22 00:05:10.690 --> 00:05:23.999 Laueren Dana: So the first step is, you need to determine the why of the course, why does the course need revisions? Maybe you noticed that the material felt a little stale, or maybe you received negative student feedback in your evaluations? 23 00:05:24.070 --> 00:05:31.740 Laueren Dana: Maybe your students are not picking up on concepts as quickly as they had in the past. Or maybe you're more worried about academic integrity. 24 00:05:31.950 --> 00:05:43.060 Laueren Dana: even though you may add more reasons why you need to revise your course as you dig more deeply into course data, it is important to approach the revision process with your own personal. Why. 25 00:05:43.250 --> 00:05:58.729 Laueren Dana: next you will gather any applicable and available data from your courses. We will review much of the quantitative and qualitative data available to you in this workshop. But just as a reminder, our main focus today will be on student evaluations as a data point. 26 00:05:59.190 --> 00:06:11.699 Laueren Dana: Next, you will spend time analyzing and reflecting on what this data tells you. you will create an action plan for changes. You hope to implement in your course, based on your data analysis and reflection 27 00:06:11.890 --> 00:06:21.759 Laueren Dana: over the next few months you will implement your action plan for the next time. Your course or course is run. However, the course revision process doesn't stop here, as Amy said. 28 00:06:21.790 --> 00:06:31.159 Laueren Dana: as we discussed a few moments ago, the course, reflection and revision process is cyclical, and you will repeat these steps each time after your course runs. 29 00:06:31.400 --> 00:06:49.330 Laueren Dana: However, we do want to remind you that you are human with many competing priorities, and the quality, of course, reflection and the revision process is absolutely going to vary from year to year. You may only be able to make one or 2 small changes some years, while other years you have time to revamp your entire course. 30 00:06:49.620 --> 00:06:58.810 Laueren Dana: at least today, though we promise that you will leave this workshop completing the bolded steps. 2, 3, and 4 for your student evaluations for at least one course. 31 00:07:01.940 --> 00:07:13.909 Laueren Dana: Okay, as promised, we're and going to give a very brief overview of some of the types of available. Course, data that may help you in the course revision process before we focus on student evaluations. 32 00:07:15.580 --> 00:07:19.289 Laueren Dana: Okay, so we're first going to quickly discuss quantitative analytics 33 00:07:20.190 --> 00:07:31.750 Laueren Dana: to access quantitative analytics. You will first go to faculty tools, and on the right column you will see a section labeled Administrative Tools, and the Quantitative data is under student activity reports. 34 00:07:31.980 --> 00:07:41.610 Laueren Dana: You will see that course plus offers data on lecture and content access, live talk attendance. If your course is fully online, discussion, form, access and site access. 35 00:07:41.640 --> 00:08:07.750 Laueren Dana: All of the data points are available to download in either a Pdf or Excel format. This data and the learning analytics from this data are incredibly important in course revisions, and to learn more on how to analyze this data. I do recommend watching our presentation from last June that focuses specifically on quantitative data, learning analytics and course revision. Amy and I will send the link out to that resource in a follow-up email. 36 00:08:09.220 --> 00:08:14.159 Laueren Dana: Okay, we're going to now shift our focus on the qualitative data available on course plus. 37 00:08:16.030 --> 00:08:19.510 Laueren Dana: Okay. So one source is emails as a data source. 38 00:08:19.560 --> 00:08:41.149 Laueren Dana: So the communications that you have with students is a qualitative data source. You can review emails that you send to your students by using the email archive feature in the course plus course email to tool just as a reminder. This only tracks emails that you send to students when using the course plus email tool and selecting the save to email archive which we recommend doing. 39 00:08:41.380 --> 00:09:05.259 Laueren Dana: You may consider gathering information from any emails you receive from students like frequently asked questions or reported issues. So, for example, if a lecture has an FAQ. Associated with it. You can post the QA. On a lecture page. If students report an issue with an assignment like unclear instructions or an external link that doesn't work. You can make the necessary fixes for the next course offering. 40 00:09:05.610 --> 00:09:17.850 Laueren Dana: Okay, so as you're looking through emails, you want to see if there's any trends in email sent to students. any frequently asked questions received from students. And again, any reported issues received from students. 41 00:09:19.620 --> 00:09:28.530 Laueren Dana: Okay? And then, last, but not least, our focus of today. Course, evaluations are a great source of quantitative and qualitative data for your course. 42 00:09:28.840 --> 00:09:35.149 Laueren Dana: Course. Evaluations are vital to the learning process at Bsp. For both students and faculty. 43 00:09:35.210 --> 00:09:42.280 Laueren Dana: Each term students provide anonymous feedback on their courses during a predetermined evaluation period 44 00:09:42.480 --> 00:09:52.059 Laueren Dana: there are standardized evaluation questions that students receive that consists of students rating the course on a numerical scale and open-ended questions. 45 00:09:52.200 --> 00:10:00.240 Laueren Dana: We will send the complete list of questions in our follow-up email. However, for today's purpose, we will place the 4 open-ended questions in the chat. 46 00:10:00.810 --> 00:10:10.119 Laueren Dana: You also have the opportunity to add 3 open-ended questions of your own to course evaluations. These instructions will also be in our follow-up email. 47 00:10:10.470 --> 00:10:24.129 Laueren Dana: Once the evaluation period has closed, you will receive an alert to view the evaluation reports by logging into the course evaluation system. And we will also place that link in the chat. And we had also sent it out in our prior email 48 00:10:24.580 --> 00:10:32.749 Laueren Dana: faculty can compare course evaluations from up to 5 courses, and the courses can be from any of the past 3 academic years. 49 00:10:33.240 --> 00:10:42.039 Laueren Dana: These evaluations should provide faculty with student perspectives as well as guide future course, modifications and redesigns if necessary. 50 00:10:47.700 --> 00:11:13.449 BSPH CTL Teaching Toolkit: Sorry I was muted alright. So. And this is a great time to pause and talk about reflection and specifically critical reflection when looking at your course data, when you're identifying changes for your course, or when you're looking at your considering your data, especially qualitative data. This is also a great time to reflect on your overall experience of the course and the perspectives of your teaching team. If you're working with Co instructors or say teaching assistants. 51 00:11:13.740 --> 00:11:21.890 BSPH CTL Teaching Toolkit: If you were leaving qualitative feedback about your course, what would you say about it? That's a good reflective question to to ask yourself. 52 00:11:22.060 --> 00:11:30.939 BSPH CTL Teaching Toolkit: So, taking a moment to reflect critically about your course will help you contextualize and analyze your available course data 53 00:11:30.960 --> 00:11:56.630 BSPH CTL Teaching Toolkit: and and help you identify what to do next. And while reflection is often overlooked in the course process, it can help elevate your understanding of what's going well, of what might need to be changed and help you determine why those changes might need to be made so Lauren mentioned earlier the why of the revision process. This is the reflection can help you identify. What's the why in the process? 54 00:11:57.000 --> 00:12:06.770 BSPH CTL Teaching Toolkit: And again, this gives you a great starting point, like forming a hypothesis that you can then test against the student data as you're analyzing your your qualitative data. 55 00:12:09.000 --> 00:12:29.090 BSPH CTL Teaching Toolkit: So I know we've covered a lot, and it's only been, you know, the about 13 min since we started, and we've covered a lot. So we wanted to pause here and see if there were any questions. I don't think any questions were posted in the chat, but let us know if you have questions by posting in the chat, or you can raise your hand in zoom. 56 00:12:31.180 --> 00:12:33.870 BSPH CTL Teaching Toolkit: Alright, I'm not seeing anything. 57 00:12:34.620 --> 00:12:45.160 BSPH CTL Teaching Toolkit: Okay. And then again, there'll be more opportunities for QA. Throughout. So and again we'll we'll be monitoring the the zoom chat. So if you do have questions, if you think of anything, you can post it there 58 00:12:46.820 --> 00:13:07.539 BSPH CTL Teaching Toolkit: alright. So so now let's dive into a method for analyzing your your qualitative student data. So again, student course, evaluations, they're vital to the learning process for both students and faculty. But it can be really difficult to analyze qualitative, open ended responses from students. 59 00:13:07.540 --> 00:13:20.429 BSPH CTL Teaching Toolkit: So we've come up with a method for analyzing student responses based on the type of feedback that you receive that will help you take some of that subjectivity out of out of your and analysis. 60 00:13:21.290 --> 00:13:43.219 BSPH CTL Teaching Toolkit: And to do this we're going to go through a number of steps and before so before let's see, it was Friday. I think we sent out an email to all of registrants with a worksheet attached to the email. So I recommend pulling up that worksheet now and if you don't have it, you can. I think we're gonna share it in the chat. 61 00:13:43.220 --> 00:14:04.239 BSPH CTL Teaching Toolkit: Yes, thank you, Emily, for sharing in the chat. So if you don't have the your email open you can download the file from the chat, and we're going to refer to this worksheet throughout the rest of this presentation, because this is going to lead our our demonstration as well as the hands on activity afterwards. So I recommend you have this worksheet open. 62 00:14:04.240 --> 00:14:12.960 BSPH CTL Teaching Toolkit: Hopefully. That has given you enough time to download and open that worksheet. And again, it's linked in the chat. 63 00:14:13.280 --> 00:14:29.509 BSPH CTL Teaching Toolkit: So when you're looking at your comments. The first thing you wanna do is discard any comments that are unrelated to teaching and learning, or are non specific. So so think about comments that, you know, really don't have 64 00:14:29.870 --> 00:14:42.509 BSPH CTL Teaching Toolkit: won't apply, or don't have anything to do with your revisions. Those can be discarded off the bat. So I recommend just striking those out, crossing them out. Ignoring those they can be discarded. 65 00:14:42.890 --> 00:15:12.829 BSPH CTL Teaching Toolkit: And then let's consider your positive comments. And these comments tell you what students think is working well in your course. So these are comments for you to just enjoy and go back to whenever you run into some negative comments, and and then you can also take these and compare them with your less positive comments to see where, if where there are areas where students agree, and then perhaps there's places where students disagree 66 00:15:13.030 --> 00:15:15.420 BSPH CTL Teaching Toolkit: on what is going well in your course 67 00:15:16.360 --> 00:15:20.789 BSPH CTL Teaching Toolkit: next, our next category is actionable suggestions. 68 00:15:20.800 --> 00:15:41.230 BSPH CTL Teaching Toolkit: and these are comments that offer suggestions for your course. Or shed light on pain points in your course that you, as a faculty member have the control of of addressing. So these are things that you could change or you could do something about as the faculty within your the context of your course. 69 00:15:41.270 --> 00:15:56.629 BSPH CTL Teaching Toolkit: For example, the level of effort required to make. Oh, so so whenever you're looking at this this type of comment. You would you should think about what is the level of effort of making this suggested change, and then also any possible trade offs. 70 00:15:57.020 --> 00:16:13.630 BSPH CTL Teaching Toolkit: And then, finally, you're going to categorize any actionable or sorry non actionable suggestions. And these are suggestions that that go beyond the scope of your course. So these are suggestions or pain points. 71 00:16:13.630 --> 00:16:31.459 BSPH CTL Teaching Toolkit: but they're not things that you, as the faculty, can address within the context of your course. So these are things like, you know, why was this course required for the program? Or could this course be offered earlier in a program or later in a program or in other formats things like that where? 72 00:16:31.520 --> 00:16:51.360 BSPH CTL Teaching Toolkit: you know, those are things that perhaps your academic coordinator or program coordinator would need to address or could be something for classroom facilities, management, or even the center for teaching and learning. So what we recommend for this is to just consider forwarding it to relevant departments or individuals based on the type of suggestion that students are making. 73 00:16:51.720 --> 00:17:07.090 BSPH CTL Teaching Toolkit: And so by using this method, you can again, you can categorize your responses. To help you figure out exactly how you should identify what you should do with each comment. So either discard it. Compare it and look for themes or 74 00:17:07.210 --> 00:17:11.430 BSPH CTL Teaching Toolkit: forward it onto the relevant individuals. 81 00:18:34.640 --> 00:18:58.090 Laueren Dana: Great. So now what we're gonna do with the worksheet, Amy. And I thought, instead of just talking about what to do with evaluation. We're actually gonna show you? So what we did is we created a sample student evaluation for 2 of the open ended questions. So the first question that we're gonna work with is, please identify what you consider to be the strengths of the course. 82 00:18:58.090 --> 00:19:20.600 Laueren Dana: So what we're gonna do is, I'm gonna read through the comments, and as I write, read through it, I want you to start to think about what you would do if this was your student evaluation. So going back to some of the action categories that Amy discussed which we'll put in the chat. What would you do? And then Amy's gonna walk through her mind and how she would handle these comments. 83 00:19:20.700 --> 00:19:27.309 Laueren Dana: Okay, so our first comment is, the group assignments were practical and could be applied to real life. 84 00:19:27.370 --> 00:19:30.110 Laueren Dana: I do wish directions had been a little clearer. 85 00:19:30.120 --> 00:19:34.130 Laueren Dana: I love the videos with the interviews. The teaching team was amazing. 86 00:19:34.540 --> 00:19:47.069 Laueren Dana: The weekly email that the Ta sent really helped me stay organized. The video interviews were great. Everything about the course was awesome. I love the course, and I wish it could be offered online as well. 87 00:19:47.410 --> 00:20:05.949 BSPH CTL Teaching Toolkit: Okay, Amy. So what would you do with me? So walk us through your your process alright. And to do that, I'm actually going to go here and edit this as we go through. So some of these right off the bat, I would probably strike through snow. Sorry that was my dog. 88 00:20:06.010 --> 00:20:08.779 BSPH CTL Teaching Toolkit: Sorry you hit the chair. 89 00:20:09.040 --> 00:20:21.010 BSPH CTL Teaching Toolkit: alright. So first off the group assignments for practical and could be applied to real life. I do wish I'm sorry did not mean to strike through that I do. Instructions have been a little clearer. So 90 00:20:21.710 --> 00:20:33.579 BSPH CTL Teaching Toolkit: From this comment the idea which directions have been a little clearer. I'm going to highlight that, because that's something that could be an actionable suggestion that if I were the faculty looking at this. That's something I can address. 91 00:20:33.670 --> 00:20:38.670 BSPH CTL Teaching Toolkit: So yeah, I'm gonna highlight in a different color. So I can see it. 92 00:20:39.090 --> 00:20:40.380 BSPH CTL Teaching Toolkit: And 93 00:20:41.450 --> 00:20:42.599 BSPH CTL Teaching Toolkit: there we go. 94 00:20:43.100 --> 00:20:59.400 BSPH CTL Teaching Toolkit: so that I'm just gonna copy. I'm gonna highlight, because that's again. That's an actionable suggestion that I, as a faculty, could say, oh, that's something I could look at. Now, let's go to the next comment. I love the videos with the interviews. The teaching team was amazing. 95 00:20:59.450 --> 00:21:05.809 BSPH CTL Teaching Toolkit: That's a pot. Those are positive comments, but not no suggestionable actions. There are actionable suggestions there. 96 00:21:06.780 --> 00:21:21.859 BSPH CTL Teaching Toolkit: The weekly email that the Ta sent really helps me stay organized. The video interviews were great. So in this case I might highlight that the weekly email that he sent help me stay organized. So that could be something that I want to keep going. So 97 00:21:22.110 --> 00:21:35.749 BSPH CTL Teaching Toolkit: not really an actual suggestion, but something that is working well. So I might say, Okay, that's something that we should not change. Everything about the course was awesome. That's that can be just discarded, because. 98 00:21:35.880 --> 00:21:56.909 BSPH CTL Teaching Toolkit: it's really great, but it's not helpful, or there's no nothing there to really look at. I love the course and wish it would be offered online as well. So here wish it would be offered online as well. That's something that I would need to talk to the academic or program coordinator about. So I'm gonna highlight that in a different color. 99 00:21:57.550 --> 00:22:06.770 BSPH CTL Teaching Toolkit: and say, Okay, that's something that is a non actionable suggestion on my part as the faculty for this course. 100 00:22:07.780 --> 00:22:11.059 BSPH CTL Teaching Toolkit: And then let's see, this is our our second 101 00:22:11.110 --> 00:22:22.430 BSPH CTL Teaching Toolkit: slide, and this one will do the same thing where we're just gonna go through and organize these. The assignment directions were really confusing, and I wasn't sure what the professor was looking for in the submitted assignment. 102 00:22:22.450 --> 00:22:30.960 BSPH CTL Teaching Toolkit: That's huge. So here I'm going to highlight that again in blue as something that I need to address. The course was boring 103 00:22:31.010 --> 00:22:41.800 BSPH CTL Teaching Toolkit: like. That's not fun to read. But you know, that's something that we would just. I would just strike through because they didn't explain why it was boring. So it's not really actionable. 104 00:22:43.760 --> 00:23:00.240 BSPH CTL Teaching Toolkit: our next point is, I see the point of the group assignments. But my group really struggled. We weren't sure where to submit our assignments, and if we each needed to submit the assignment, I also felt like, not all my group members put in the same amount of effort. 105 00:23:00.340 --> 00:23:07.500 BSPH CTL Teaching Toolkit: Okay, so here there's a lot here. There's a few things to unpack. So group struggled. 106 00:23:08.550 --> 00:23:11.370 BSPH CTL Teaching Toolkit: Weren't sure where to submit. 107 00:23:12.770 --> 00:23:15.159 BSPH CTL Teaching Toolkit: And then also 108 00:23:15.490 --> 00:23:33.069 BSPH CTL Teaching Toolkit: group amount of effort. So here are some 3 or 3 things that I would highlight as potentially actionable on my part where I, as this faculty, might restructure the group work, and then also look at the assignment instructions. See where the where they were. Unclear. 109 00:23:33.600 --> 00:23:43.519 BSPH CTL Teaching Toolkit: I found the teaching team to be rude, and they never responded to me on time. Also way too much work for a 2 credit course. My group also did nothing, yet 110 00:23:43.650 --> 00:23:54.960 BSPH CTL Teaching Toolkit: they got the same grade as me. So some of these things so never responded on time. That's something that I as the faculty, could address 2 credit cores. So here 111 00:23:55.000 --> 00:24:08.799 BSPH CTL Teaching Toolkit: I'm gonna highlight that as I, as the faculty, could look at as an actionable item, but that could also be something that would need to be passed on to your academic or program coordinator. Perhaps there is a an imbalance in the credit scale. 112 00:24:08.820 --> 00:24:14.619 BSPH CTL Teaching Toolkit: And then my group did nothing. Yet they got the same grade as me. So again, some issues with the group work. 113 00:24:15.840 --> 00:24:23.979 BSPH CTL Teaching Toolkit: And then finally, the course had similar learning objectives to another required course in my correct program, I'm not sure if both should be required. 114 00:24:24.230 --> 00:24:34.810 BSPH CTL Teaching Toolkit: So here, that's something that again, that would be beyond me as the faculty for the context of this course, but some feedback I could share with the academic coordinator. 115 00:24:34.890 --> 00:24:49.919 BSPH CTL Teaching Toolkit: So this is how, again, I would go through these 2 particular sets of comments, for this particular so that's how I would categorize these these comments. And then let me go back to 116 00:24:50.910 --> 00:24:54.850 BSPH CTL Teaching Toolkit: the slideshow view current slide. 117 00:24:56.360 --> 00:25:04.170 BSPH CTL Teaching Toolkit: Okay? And then you should be seeing step 2 student course evaluations organized by organize response by theme. 118 00:25:07.460 --> 00:25:31.550 BSPH CTL Teaching Toolkit: Alright, so so now we've reviewed those sample responses. And now we're gonna work on organizing and synthesizing those comments thematically. So if you have discarded comments again, you can just ignore your discarded comments. And now you're gonna look at those comments that we highlighted in blue for the faculty, those actionable suggestions. And see if there are overarching themes throughout. 119 00:25:31.550 --> 00:25:44.020 BSPH CTL Teaching Toolkit: Across these categories here. And I think for us, maybe we could look at those comments that were directed towards the group work. So that would fall under assessments. 120 00:25:44.070 --> 00:25:58.829 BSPH CTL Teaching Toolkit: and for assessments again, using those sample comments we just received. We can look at some of the assessments, and I'll I can go back to look at those assessments. So the group work 121 00:25:58.910 --> 00:26:04.459 BSPH CTL Teaching Toolkit: had some issues where there was some unclear instructions. The group work had 122 00:26:04.550 --> 00:26:11.790 BSPH CTL Teaching Toolkit: some unfairness or a sense of some students thought they were unfair. And then also 123 00:26:12.030 --> 00:26:20.390 BSPH CTL Teaching Toolkit: there are some issues on how they should be submitting their work. So let's I'm going to again escape from here. 124 00:26:20.480 --> 00:26:30.939 BSPH CTL Teaching Toolkit: So under assessment, I would put positive feedback over positive feedback, I believe one of the students said that they were practical and applied to real life. So that's good. They're practical 125 00:26:35.430 --> 00:26:56.480 BSPH CTL Teaching Toolkit: and then applicable to real life. So that's good. So that tells me that the the the content, and the goals of the assessments are probably fine. I'm assuming that at this point my my assessments align with my learning objectives. So now the constructive criticism. Again, there was 126 00:26:56.610 --> 00:27:09.969 BSPH CTL Teaching Toolkit: some unclear instructions. unclear submission guidelines. and then some unfairness between group members 127 00:27:10.370 --> 00:27:12.999 BSPH CTL Teaching Toolkit: or perceived unfairness, I should say. 128 00:27:17.020 --> 00:27:25.740 BSPH CTL Teaching Toolkit: And then are these actionable? Yes, all all of these things are actionable. And that's gonna take us then to step 3 129 00:27:25.800 --> 00:27:35.239 BSPH CTL Teaching Toolkit: and step 3 is where we're going to actually identify what specific actions we can take based on the feedback that we just 130 00:27:35.250 --> 00:27:44.090 BSPH CTL Teaching Toolkit: analyzed and just just identified. So again, we're gonna look at the category. We discussed in Step 2, which were the group assessments 131 00:27:44.310 --> 00:27:55.780 BSPH CTL Teaching Toolkit: and then using feedback from our positive instructive criticism. What are some actionable or actions actionable items that we could change based on our students experiences. 132 00:27:56.110 --> 00:28:02.460 BSPH CTL Teaching Toolkit: So does anyone for this, I could say, does anyone have any 133 00:28:03.890 --> 00:28:05.630 BSPH CTL Teaching Toolkit: any 134 00:28:08.220 --> 00:28:15.600 BSPH CTL Teaching Toolkit: Oh, my goodness, any recommendations, or something like, if you receive this feedback from your students, what are some things that you might do 135 00:28:15.700 --> 00:28:21.510 BSPH CTL Teaching Toolkit: for your actions. So, knowing that again, these are what we're we're trying to address. 136 00:28:22.990 --> 00:28:26.580 BSPH CTL Teaching Toolkit: So we have unclear instructions. 137 00:28:27.590 --> 00:28:36.720 BSPH CTL Teaching Toolkit: unclear submission. And then some perceived unfairness. So for unclear instructions right away. 138 00:28:36.780 --> 00:28:45.679 Laueren Dana: I would say we actually have something in the chat. So review the assignment directions to see how to make them clearer. So that's an excellent. 139 00:28:45.980 --> 00:28:48.639 Laueren Dana: I think. First actionable item for that one. 140 00:28:49.600 --> 00:28:57.659 BSPH CTL Teaching Toolkit: Yes, I think that's excellent. So on here, let me zoom in a little bit more so you can see the screen better. 141 00:29:00.760 --> 00:29:01.789 BSPH CTL Teaching Toolkit: There we go. 142 00:29:02.090 --> 00:29:07.650 BSPH CTL Teaching Toolkit: Yes, so that's excellent. So review the instructions 143 00:29:07.980 --> 00:29:09.549 BSPH CTL Teaching Toolkit: for clarity. 144 00:29:09.900 --> 00:29:23.219 Laueren Dana: and mia also recommended having a ta to review as it's always good to have a second eye on revised instructions, and I would say same if you have an instructional designer. If you would reach out to us, we'd be happy to review. So we could also be a second eye. 145 00:29:23.250 --> 00:29:27.450 Laueren Dana: But I think that's also a great suggestion. Yes, absolutely 146 00:29:28.750 --> 00:29:33.470 BSPH CTL Teaching Toolkit: alright. Those are great, really good, and really good. Recommendations 147 00:29:34.490 --> 00:29:45.699 BSPH CTL Teaching Toolkit: alright. And then for the unclear submission, I would say probably the same, the same things as above, where? You know. Check specifically your submission instructions. 148 00:29:45.910 --> 00:29:51.300 BSPH CTL Teaching Toolkit: for clarity, and then again ask for someone else to review. So those are going to be the same. 149 00:29:53.010 --> 00:29:59.140 BSPH CTL Teaching Toolkit: and then perceived unfairness between group members. Any recommendations for that part. 150 00:30:05.970 --> 00:30:32.189 BSPH CTL Teaching Toolkit: And for this, what I've seen work really well is to have a group, a group con a contract where you might have students identify who's responsible for what? And then agreed to that at the very beginning of the course or at the beginning of the group project, so that everyone has clear roles and everyone has knows what they should be working on, and is held accountable to that within their group. 151 00:30:32.220 --> 00:30:52.150 BSPH CTL Teaching Toolkit: You might also consider a peer evaluation where you ask students to evaluate each other as group members like. How did you? How did your team excuse me? How did your team come together and work? And then any other thoughts or suggestions for for this particular category. 152 00:30:53.170 --> 00:30:57.890 BSPH CTL Teaching Toolkit: If not, that's okay. Oh, I just saw something pop up in the chat. 153 00:30:58.250 --> 00:31:28.419 Laueren Dana: Oh, oh, yeah, go ahead. It's mia. So from a teaching perspective, she might look at this across comments over time, as one group may also be having specific issues. So maybe you just wait. Wait on this one and see if this is a constant, or if it's just between a couple of groups. And then also having clear communication about open feedback to faculty if there is someone not pulling their weight. So I asked who to reach out to if that's the case, like ta faculty email discussion forum, whatever 154 00:31:28.550 --> 00:31:31.290 Laueren Dana: whatever works. Thanks. Mia. 155 00:31:33.120 --> 00:31:39.090 BSPH CTL Teaching Toolkit: yes, very good. All really great suggestions. Thank you. So here, 156 00:31:39.720 --> 00:31:54.829 BSPH CTL Teaching Toolkit: again. And then let's let's take one of these and look at benefits of change, and then drawbacks of change. For the the first 2. Again, those are gonna have have high benefits. Low drawbacks, and also probably low level of effort. 157 00:31:54.910 --> 00:31:57.130 BSPH CTL Teaching Toolkit: So again, Hi. 158 00:31:59.000 --> 00:32:00.230 BSPH CTL Teaching Toolkit: low 159 00:32:01.060 --> 00:32:15.279 BSPH CTL Teaching Toolkit: one in fact, I can't think of any drawbacks to clarifying instructions, for, let's say, specifically, the group, the group contract. Let's focus on that one. The benefit of this change would be 160 00:32:15.730 --> 00:32:18.139 BSPH CTL Teaching Toolkit: peer expectations. 161 00:32:19.570 --> 00:32:23.760 BSPH CTL Teaching Toolkit: locations, accountability 162 00:32:25.320 --> 00:32:30.170 BSPH CTL Teaching Toolkit: ability I can spell and 163 00:32:31.620 --> 00:32:39.220 BSPH CTL Teaching Toolkit: I think those would be the 2, the 2 major benefits drawbacks. It could take some time 164 00:32:39.230 --> 00:32:46.909 BSPH CTL Teaching Toolkit: time to write up a contract. You might have students who disagree. 165 00:32:47.710 --> 00:32:50.050 BSPH CTL Teaching Toolkit: so there might be some 166 00:32:52.230 --> 00:33:04.239 BSPH CTL Teaching Toolkit: monitoring that you, as the faculty might need to need need to do in order to make this contract. And again, they might just you know, there, there's a little bit of effort in the administration. 167 00:33:04.900 --> 00:33:27.570 BSPH CTL Teaching Toolkit: So there's gonna be a little bit more administrative task and and implementing a contract. So this one has a little bit higher level of effort. I wouldn't say it's for the highest, I would say. This is probably a 2.5, or maybe a 3, depending on how how much you put into this contract, or how much you have, the students come up with what? What their own responsibilities will be. So that's how I would 168 00:33:27.650 --> 00:33:42.130 BSPH CTL Teaching Toolkit: categories that particular actionable item across these. So so any questions before we our next step is, we're actually gonna have you to do this for your own data. So now is a good time for 169 00:33:42.330 --> 00:33:43.620 BSPH CTL Teaching Toolkit: questions. 170 00:33:50.260 --> 00:33:55.350 Laueren Dana: We look good on the chat, Amy. So alright thanks. And then I will 171 00:33:55.510 --> 00:33:58.800 BSPH CTL Teaching Toolkit: go back to sharing my screen. 172 00:33:59.520 --> 00:34:02.679 BSPH CTL Teaching Toolkit: Oh, sorry I lost my zoom window for a minute a moment. There. 173 00:34:05.200 --> 00:34:26.759 BSPH CTL Teaching Toolkit: alright! So now is a really great time also to think about. There is an elephant in the room that we haven't addressed yet before we dive into your own student data on the hands, on activity, and that is receiving feedback can hurt. It's not always easy. It can be difficult, especially when you've put a lot of work and effort into your course. 174 00:34:26.760 --> 00:34:38.620 BSPH CTL Teaching Toolkit: So I'm gonna share a short personal experience of some difficult feedback that I received and how and and how that feedback ultimately helped me improve a course that I was working on. 175 00:34:38.699 --> 00:34:59.369 BSPH CTL Teaching Toolkit: So. I mentioned earlier the Ta training course. So the th training course is, of course, for teaching assistance at Bsp, that's run for many years. And in 2020, my my Co. Instructor and I, we noticed from the evaluation data that students wanted 2 things. They wanted more presence from the instructors and clear information about each module. 176 00:35:00.160 --> 00:35:15.609 BSPH CTL Teaching Toolkit: So, based on that feedback, we carefully planned and produced short one to 2 min videos that were module overviews for each module in which I introduced the module. Answered some faqs and just talked about an overview of the module. 177 00:35:16.070 --> 00:35:26.869 BSPH CTL Teaching Toolkit: So now in 2020, well, last year now, because it's now 2024, but in 2023, we were updating the course, and we realized. The feedback from the videos was clear. 178 00:35:26.950 --> 00:35:45.899 BSPH CTL Teaching Toolkit: The students did not like them. In fact, they kind of hated them. They hated these videos. The students wrote that the course evaluation in the course evaluations that they thought the videos were waste of time. They were an extra click. They said the videos were pointless, and they even said that my voice sounded like a cartoon character 179 00:35:45.900 --> 00:35:59.940 BSPH CTL Teaching Toolkit: which hurts little bit like Ouch after licking my wounds, my co instructors and I, we decided to make some revisions, the course and the offering that is currently running the offering that just started January first. 180 00:35:59.940 --> 00:36:18.129 BSPH CTL Teaching Toolkit: And we hope that these changes will help deliver the information that the students need to know in a way that they are is much more preferable in this case. Instead of a video, it's bullet points on an overview page and increased instructor, presence and other ways like online discussions and live talks. 181 00:36:18.560 --> 00:36:34.880 BSPH CTL Teaching Toolkit: So again, feedback can hurt. And sometimes you put a lot of effort into something, only to find out that the students didn't actually want that or didn't like that. And that's okay. The key to addressing this elephant in the room is to give yourself time to to breathe. 182 00:36:34.980 --> 00:36:44.450 BSPH CTL Teaching Toolkit: to recover from the feedback, especially negative feedback. And then, once you've had that moment to to breathe 183 00:36:45.220 --> 00:37:14.590 BSPH CTL Teaching Toolkit: return to your course. Evaluations, any any kind of negative feedback. When you're ready. In this case the feedback was helpful because it was actionable. We could do something about it. But if it was just oh, your voice as a cartoon character that could have. Just that's just discarded cause. That's not something I can change. So so again, we hope that the method that we went over today will help you address this elephant in the room by objectively categorizing and discarding comments when they're not helpful. 184 00:37:15.470 --> 00:37:18.840 BSPH CTL Teaching Toolkit: All right. So any questions before we go further. 185 00:37:21.350 --> 00:37:43.109 Laueren Dana: Okay, so I'm gonna pass things to Lauren. Great. Thank you for sharing that story, Amy. That was, I think, helpful for all of us. Yeah. So now we've gone through the part of step part one of the worksheet which was the sample evaluation that Amy and I created. We're gonna now do hands on practice on how to approach and analyze your student evaluations. 186 00:37:44.530 --> 00:37:51.069 Laueren Dana: Okay, so now we're gonna have you start to complete these steps for one of your own courses, evaluations. 187 00:37:51.260 --> 00:38:11.780 Laueren Dana: We know that you will most not likely complete all the steps that we just did for the sample your own course. Evaluations are probably more complicated, more comments. But we are hoping that by the end of the workshop you can at least get started, and if you need any support you can reach out to us. And then continue this process outside of the workshop. 188 00:38:11.800 --> 00:38:33.220 Laueren Dana: I'm just gonna again have Emily, even though she's done it a couple of times, the link to access your course evaluations if you need it. If by some chance you are struggling to access it, or you just don't have a course that you wanna work on right now at the bottom of the worksheet. We also have created a sample with the ta course. So you can use that just to practice? 189 00:38:33.450 --> 00:38:44.760 Laueren Dana: So the first thing that we're gonna do is step one which is access the student evaluations from one of your courses, or scroll down to the bottom of the worksheet and look at the ta evaluation. 190 00:38:45.250 --> 00:38:57.920 Laueren Dana: So that's step one step 2 is, I'm just going to have you once again. Make sure that you have that workshop worksheet in front of you. We're hoping that this will be helpful, and you can use the charts to to work your way through the steps. 191 00:38:59.840 --> 00:39:12.559 Laueren Dana: And then we're now going to go step by step on how to collate. The student comments and how to create actionable. So next steps so pretty much the same process. Amy and I just went through with the sample evaluations we are going to go through with you. 192 00:39:12.690 --> 00:39:41.139 Laueren Dana: I did, however, want to let you know that we're only gonna give you a set amount of time for each of these steps, and it probably won't feel like a lot but we did. Just said like I said, wanna Get you started, get your feet wet and just as a reminder we will be offering breakout room sessions after the workshop for any of you that hit some personal questions. That you'd like to address after going through the steps, and as you're going through the steps, feel free to use the chat, and we are happy to help you. 193 00:39:42.080 --> 00:40:08.440 BSPH CTL Teaching Toolkit: Okay, so I'm gonna pass it off to Amy. So we can start step one. This is exciting. Thank you. Lauren. Alright. So as Lauren mentioned, we're going to actually, take your own student course data and go through step one. So go through your student data or the sample Ta training data that was provided in the worksheet, and actually go through and strike out or mark out the discarded comments. Mark 194 00:40:08.490 --> 00:40:17.819 BSPH CTL Teaching Toolkit: little, you know. Add a smile, face or something for positive ones. Look for themes and consider what's actionable, and the level of effort or trade-offs 195 00:40:17.820 --> 00:40:38.969 BSPH CTL Teaching Toolkit: or things that might need to be passed on to other people. So I'm gonna set a timer for 5 min. Starting now for you to actually go through and start looking at your own student data. And again, if you don't have data, you can use the ta training data provided in the worksheet. So the timer is running if you have questions, post those in the chat. 196 00:40:57.840 --> 00:41:04.909 BSPH CTL Teaching Toolkit: Oh, and a question did just come in about Ferpa and using Chachi Vt for student course evaluations. 197 00:41:04.910 --> 00:41:29.849 BSPH CTL Teaching Toolkit: So if you're gonna use generative AI for your your qualitative course to help you, I analyze your data. Just make sure that it's completely anonymized. There shouldn't be any kind of student identifiable data. If you're gonna an enter that into the into an AI or something to help you help you analyze that data? So that was a great question. So as long as there's no student identifiable information, it should be for 198 00:41:29.850 --> 00:41:31.030 compliance. 199 00:41:42.230 --> 00:41:48.010 BSPH CTL Teaching Toolkit: Oh, and also yes, for Ferpa there shouldn't be faculty or ta info either. Thank you, Kathy. 200 00:42:27.110 --> 00:42:32.119 BSPH CTL Teaching Toolkit: All right. We still have 3 min left. and I'll give a 1 min warning. 201 00:44:22.430 --> 00:44:24.510 BSPH CTL Teaching Toolkit: Alright. We have 1 min left. 202 00:44:42.090 --> 00:44:51.059 BSPH CTL Teaching Toolkit: and if you want you can give me a thumbs up. If you're ready to move on, or you can. We have 30 s we can sit for 30 more seconds, or 203 00:44:51.190 --> 00:44:52.640 BSPH CTL Teaching Toolkit: you can give me a thumbs up 204 00:45:24.280 --> 00:45:37.939 BSPH CTL Teaching Toolkit: alright, and that is time. Alright. So by now, hopefully, you've you've been able to at least categorize a number enough. Enough of your student qualitative data for step one to move on to step 2 205 00:45:38.390 --> 00:45:52.459 BSPH CTL Teaching Toolkit: and again, here you're going to look for one major theme, and you might have multiple themes. You may have identified more than one theme, and you're in step one. But for today we're just gonna look at one of these themes, and 206 00:45:52.570 --> 00:45:56.469 BSPH CTL Teaching Toolkit: synthesize those comments into positive 207 00:45:56.500 --> 00:46:14.959 BSPH CTL Teaching Toolkit: feedback and constructive criticism and then identify which might be actionable for you as the faculty. So this time we're only gonna give you 2 min. So it's gonna go by faster and again. Look for those major themes. I'm setting a 2 min timer starting now, and if you have questions, post those in the chat 208 00:47:07.550 --> 00:47:09.279 BSPH CTL Teaching Toolkit: alright 1 min left 209 00:48:00.220 --> 00:48:02.549 BSPH CTL Teaching Toolkit: alright, and that is time. 210 00:48:04.110 --> 00:48:23.390 BSPH CTL Teaching Toolkit: And then next again, we're gonna go to step 3. So hopefully, as you were identifying what was positive. And then what could be changed? And what's actionable? You're gonna then gonna take those actionable items and put them here in step 3. I think it's part 2 step 3 in the worksheet 211 00:48:23.390 --> 00:48:37.989 BSPH CTL Teaching Toolkit: and identify some possible actions that you can take. I'm gonna go ahead and start the timer. Now. The slide says 3 min. But for time, since we're already at 1250, I'm like, actually only gonna give you 2 min just for the sake of time. 212 00:48:37.990 --> 00:48:49.470 BSPH CTL Teaching Toolkit: So again, you're gonna identify what changes could be made. And then you're gonna think of any benefits and drawbacks and the level of effort that might be needed to complete that 213 00:48:49.700 --> 00:48:51.179 BSPH CTL Teaching Toolkit: to make that change. 214 00:50:02.280 --> 00:50:04.570 BSPH CTL Teaching Toolkit: we're at the thirty-second mark. 215 00:50:30.140 --> 00:50:40.510 BSPH CTL Teaching Toolkit: And now we're at time. Okay, so hopefully, by now, you have some actionable next steps. And I'm gonna pass things to Lauren. 216 00:50:40.970 --> 00:51:10.559 Laueren Dana: Great. Thank you, Amy. So just our final recommendation for today's workshop is that any instructional designer at the center of teaching and learning is happy to meet with you and help accompany you through this process of getting through your quantitative and qualitative data with that in mind, if you wanna get start a head start today a number of our instructional designers are gonna stay on at the end of today's session. And if you have an online course, your your instructional designer might actually be here. So please feel free to either reach out to us 217 00:51:10.560 --> 00:51:16.970 Laueren Dana: after this session, or just stay on after, and we can help you with some of these course evaluations. 218 00:51:19.320 --> 00:51:22.389 Laueren Dana: And then we did just want to see if anyone had any final 219 00:51:22.770 --> 00:51:25.789 Laueren Dana: questions that we'd be happy to support you with 220 00:51:31.210 --> 00:51:33.870 BSPH CTL Teaching Toolkit: nothing in the chat. So I think we're good. 221 00:51:34.180 --> 00:51:38.589 Laueren Dana: Great. Okay, I'm gonna pass it to Amy to give you some additional resources 222 00:51:39.910 --> 00:51:41.159 BSPH CTL Teaching Toolkit: on the right. 223 00:51:41.360 --> 00:51:54.559 BSPH CTL Teaching Toolkit: Alright, thank you, Lauren. So I wanted to highlight first. The essentials, of course, design, development and teaching at the S. Ph. This is an online self pace, Cla course that we developed for faculty. 224 00:51:54.590 --> 00:52:01.329 BSPH CTL Teaching Toolkit: and in particular, so it's for faculty who want to learn more about course management, redesign and other pedagogical topics. But 225 00:52:01.400 --> 00:52:15.330 BSPH CTL Teaching Toolkit: there's there's a specific module within that course called reflecting on a course that provides more information about course, reflection, review and read and revision. That would be a great accompaniment to today's workshop. 226 00:52:15.420 --> 00:52:29.399 BSPH CTL Teaching Toolkit: So I highly recommend. If you're a faculty at Bsp to take that essentials course. I also wanted to mention. If you're a teaching assistant, or if you work with teaching assistants. They can take the Ta training course, and that pro that 227 00:52:29.620 --> 00:52:45.630 BSPH CTL Teaching Toolkit: prepares students to become tas at Bsp, and then, finally, you can stay up to date on the latest events and recommendations from Ctl. By checking our out our events. Page and subscribing to our blog and links, are, for all of these things are in the chat. 228 00:52:46.410 --> 00:53:16.239 BSPH CTL Teaching Toolkit: There's our references. And I just wanted to say, thank you so much. Thank you so much for your time today for going through that hands on activity, Lauren. And I hope that you found today's workshop valuable as you reflect on and analyze your course. So please let us know how we did by completing this 2 min survey by going to the link in the on the slide, or using your smartphone or smart device to scan the QR code to get to the form that way. 229 00:53:16.790 --> 00:53:40.850 BSPH CTL Teaching Toolkit: and again, just thank you so much the letting us know your feedback and how we did so that we can benefit just like you benefit from your feedback. We also benefit from from receiving feedback. So we appreciate you taking the time to do this workshop evaluation survey? And again, if you wanna stay on after the workshop some ids will be on afterwards. And actually, should we have all the Ids raise their hands? 230 00:53:40.990 --> 00:53:46.180 BSPH CTL Teaching Toolkit: Sure. So he looks like Selene has her hand up. Oh, okay. 231 00:53:46.240 --> 00:53:52.059 Celine Greene: II was going to say, slash, ask 232 00:53:52.100 --> 00:54:09.139 Celine Greene: when we're identifying like common themes and everything. I think it would be helpful. It's always helpful to me to categorize it as a facilitation versus course design, because a lot of the feedback is like when you're saying timely feedback. That's not 233 00:54:09.160 --> 00:54:17.260 Celine Greene: the like core site thing. And that is something that has to be recognized and attended to during the offering. 234 00:54:17.360 --> 00:54:20.930 Celine Greene: not in thinking ahead, and 235 00:54:21.130 --> 00:54:26.130 Celine Greene: in doing so, even if we recognize that as a theme. How might you recommend 236 00:54:26.240 --> 00:54:33.729 Celine Greene: that that is addressed in when it needs to be that just in time, facilitation stuff as opposed to the setup? 237 00:54:33.950 --> 00:54:44.810 Celine Greene: So do you have any recommendations for addressing those? Thematic elements that have more to do with the facilitation, while it's happening as opposed to the design of the court. 238 00:54:45.220 --> 00:55:10.370 BSPH CTL Teaching Toolkit: Yeah, no. That's a great, that's a great distinction. Now, i'm thinking, we should update our worksheet to include that as a that would be a great addition to our worksheet. So that's fantastic feedback. Thank you, Celine. And yes, I think that that is a good distinction to make those kinds of things I would say I would if I were working with the faculty and they received the sort of administrative just in time. Feedback? Suggested items 239 00:55:10.400 --> 00:55:28.499 BSPH CTL Teaching Toolkit: I would recommend doing like some kind of communic if it's feedback, some kind of communication, or a communication plan, or an administrative like course, administration plan before the the term starts, especially if you're working with other people like other like Tas or 240 00:55:28.570 --> 00:55:34.239 BSPH CTL Teaching Toolkit: Co. Instructors, so that you know who is doing what they could use. The who does what 241 00:55:34.300 --> 00:55:43.169 BSPH CTL Teaching Toolkit: worksheet that Cdl has on their toolkit, which all will include a link to that in the email. So that's a really great tool as well. 242 00:55:43.310 --> 00:55:45.290 BSPH CTL Teaching Toolkit: just to make sure, you know, like 243 00:55:45.330 --> 00:56:09.789 BSPH CTL Teaching Toolkit: again, what you should identify, what you should be doing when and then following that plan. Now, if it's during the term, and you're getting that kind of feedback, then I would examine not just what are your expectations for how quickly you're getting back to students. But also, what are the students? Expectations cause? It might be those expectations might not be congruent. So it might be a case where you're, you know, you're doing what you 244 00:56:09.790 --> 00:56:26.629 BSPH CTL Teaching Toolkit: sought out to do. But maybe the students need more and in that case. If it's during a a a term th, then you could perhaps respond to students differently or change change midway. But usually I would address like again that I would address that kind of thing before 245 00:56:26.630 --> 00:56:32.880 BSPH CTL Teaching Toolkit: the next offering starts, so that you have a plan going forward into the next offering. Does that answer your question. 246 00:56:34.350 --> 00:57:00.439 BSPH CTL Teaching Toolkit: Yeah, I think so. Thanks. Yup. Oh, and mia! Mentioned in the chat. There's a midterm survey template in the survey tool and course, plus and yes, particularly for new courses. This can be helpful to take the temperature during the term. Yeah, thank you. Mia, that's a really good point. So those midterm surveys can help you identify if there's something that needs pivoted during an academic term versus waiting until the end to fix it for the next offering. 247 00:57:03.380 --> 00:57:08.440 BSPH CTL Teaching Toolkit: Alright. Any other final thoughts or questions. 248 00:57:10.620 --> 00:57:36.679 BSPH CTL Teaching Toolkit: Alright, and again, a couple of us. We're going to stay on afterwards, so feel free to stay on if you want to continue talking course, evaluation data, but otherwise, thank you so much for attending today's workshop, especially with the 2 h delay earlier. We really appreciate you taking the time to join us today. A link to this recording, along with all the resources shared, will be emailed to all of the Zoom registrants after the workshop. 249 00:57:36.700 --> 00:57:38.639 BSPH CTL Teaching Toolkit: And again, thank you so much. 250 00:57:42.020 --> 00:57:44.289 BSPH CTL Teaching Toolkit: I'm going to stop the recording.