WEBVTT 1 00:00:07.960 --> 00:00:20.709 Lu Chi: Good morning, everyone thank you for joining us today. My name is Lucci. I'm here with my colleagues Brian and Amy, Brian and Emmy, would you like to introduce yourself. 2 00:00:21.630 --> 00:00:40.059 Brian Klaas: Sure. Hi, everybody! My name is Brian Kloss. I work with my colleagues, Amy and Lou, in the setup for teaching and learning, but more appropriately for today's session. I have an instructor appointment here at the school where I teach a number of classes, including a course on using generative AI in the public health space with my colleague, David Dowdy. 3 00:00:41.470 --> 00:00:47.249 BSPH CTL Teaching Toolkit (Amy Pinkerton): And my name is Amy Pinkerton. I'm a senior instructional designer at the center for teaching and learning. 4 00:00:48.340 --> 00:01:01.450 Lu Chi: Thank you. So today, we will be discussing a timely and very important topic generative AI in assessment design, embrace, adapt or recess. So let's dive in 5 00:01:02.630 --> 00:01:30.479 Lu Chi: all right. So by the end of this session you will be able to apply assessment strategies to ethically embrace AI also, you will be able to apply authentic assessment strategies to resist AI when necessary. So those are objectives will help you navigate the complexity of AI in education and make informed decisions about its role in your teaching. 6 00:01:30.930 --> 00:01:49.139 Lu Chi: All right. So let's start with a quick poll to guide you where we are as a group. So the 1st question, have you allowed students use of generative AI in your class? You know, for your assessments? And the second question, have you used generative AI 7 00:01:50.148 --> 00:01:54.619 Lu Chi: to create or design or improve your assessment. 8 00:01:55.600 --> 00:01:58.009 Lu Chi: Alright. Oh, it's already launched. 9 00:01:58.300 --> 00:02:01.030 Lu Chi: We already see some responses. 10 00:02:09.064 --> 00:02:13.310 BSPH CTL Teaching Toolkit (Amy Pinkerton): We're just waiting on a few more participants to answer. 11 00:02:15.810 --> 00:02:20.279 BSPH CTL Teaching Toolkit (Amy Pinkerton): All right, we have more than 50% answered. So I'll go ahead and end the poll 12 00:02:20.830 --> 00:02:22.480 BSPH CTL Teaching Toolkit (Amy Pinkerton): and share the results. 13 00:02:23.230 --> 00:02:24.189 Lu Chi: Thank you, Amy. 14 00:02:25.040 --> 00:02:33.469 Lu Chi: Alright. So the 1st question, have you used generative AI to create a design or improve your assessment? We see that 15 00:02:33.850 --> 00:02:38.109 Lu Chi: half was, half of the participants said, no. 16 00:02:38.580 --> 00:02:43.033 Lu Chi: and about 33% says, Yes, 17 00:02:43.840 --> 00:02:58.769 Lu Chi: okay. And then the second question, have you allowed student use of generative AI in your assessments. Now for same results. You know 50% participants that No. And 33 said, Yes. 18 00:03:00.060 --> 00:03:07.050 Lu Chi: alright. So thank you so much, you know. For participating in this poll. This is very helpful. 19 00:03:07.735 --> 00:03:12.284 Lu Chi: Hopefully, this workshop will help you, you know. 20 00:03:13.210 --> 00:03:26.199 Lu Chi: especially. You have, you know, very little experience. Use the AI to design assessment, or like incorporate AI in students assessment. This workshop will help you 21 00:03:26.510 --> 00:03:34.820 Lu Chi: alright. Next, I will turn to Brian to talk about how to embrace and adapt AI in your assessment. 22 00:03:35.490 --> 00:04:00.459 Brian Klaas: Thanks, Lou. So I'm going to spend the next 15 min or so talking about why, you might want to actually use AI in your assessments, and how you might go about doing that and doing it in a transparent and ethically responsible way. So, as I mentioned earlier, I do teach a course with David Dowdy on using generative AI in the public health space, and so obviously embracing AI for assessments, is really important in a course 23 00:04:00.460 --> 00:04:25.450 Brian Klaas: like that, but, more importantly, are also additionally, you know, I head up the team that builds courseplus here at the Bloomberg School of Public Health, and we've incorporated a number of AI generative AI tools into courseplus. And so, you know, I with that, said, I also want to make it very clear that you know I'm not like, Rah, rah! Gung! Ho! AI is the best generative. AI is the best thing for everything. It's not. It has a lot of flaws. It has a lot of issues. 24 00:04:25.740 --> 00:04:28.688 Brian Klaas: but it can be incorporated into 25 00:04:29.220 --> 00:04:53.609 Brian Klaas: courses and more specifically, assessments. And I want to talk for really, first, st next slide, please a little bit about why and how you should be embracing generative AI. And I'm not going to say that, you know this is something that everyone should do all the time. But in the right context it can be quite liberating and quite useful for students to be able to embrace AI in their work for a number of reasons, some of which I'll talk about as we move along 26 00:04:53.610 --> 00:05:18.590 Brian Klaas: next slide, please. So I think probably the most counterintuitive reason for using generative AI and assessments is that it really does require expertise on the part of the students. Now, you may think like, well, no students use chatgpt, and they just like create their, you know, their papers, and that's it. They don't have to think at all outsources all their critical thinking to generative AI. Well, sure, if you just say, turn into paper. 27 00:05:18.700 --> 00:05:42.420 Brian Klaas: that may be what happens. But remember that every time you use generative AI in particular, you have to apply a certain degree of expertise to it is the information correct is the source legitimate, right generative? AI, even the most advanced models. Chatgpt's 0 3 or Google's Gemini 2.0. They still hallucinate. They make up references I was doing. I've been using 28 00:05:42.430 --> 00:06:06.670 Brian Klaas: the deep research tool in Chat Gpt. Lately for a number of different projects, and I will tell you that while it's very good and very useful, it still generates false links, references that are completely invalid, statistical data that doesn't exist. Now, it doesn't happen very often. In fact, the error rate is probably down to like 0 point 3% of all content generated. But it's still in there. 29 00:06:06.670 --> 00:06:31.579 Brian Klaas: So a student who's going to be using generative AI has to demonstrate a certain degree of mastery of the topic through critical reflection on the generated output. And I think this is something that's often overlooked when we talk about using generative AI in our assessments, students still have to do that work, because it'll be obvious to you as a faculty, and it will probably be obvious to their peers as well. Right? So, looking at and embracing it in a critical 30 00:06:31.580 --> 00:06:37.740 Brian Klaas: is still really important and demonstrating expertise over the content is still really important. Next slide, please. 31 00:06:38.510 --> 00:07:03.459 Brian Klaas: Next thing, abundance. Generative AI is great for abundance. If you're going to look at introducing generative AI, and let's say, the brainstorming step this design step of an assessment don't go easy on the students at all. Don't just ask for one idea, ask for 300 ideas, and then get the students to figure out how to prompt the generative AI to sort of look at all those ideas and come up with the top 3 based on criteria 32 00:07:03.460 --> 00:07:28.040 Brian Klaas: provided by the students. Ask the students to use generative AI to create approaches, novel approaches to what might seem like impossible problems, problems that are intractable or almost impossible to solve. Because, again, generative AI is just spitting out bits and bytes. It can do this so much faster than we as humans could possibly do this, so force the students to do a more 33 00:07:28.040 --> 00:07:48.779 Brian Klaas: work than they might expect to do, because they're using a tool to do it. Come up with 50 ideas, come up with 3 different novel approaches to solving this problem, and then write about this critical reflection, to say why this would work and why this would not work. Don't say brainstorm, an idea, brainstorm 300, and figure out how to come up with 2 or 3 really good ideas. Next slide, please. 34 00:07:51.200 --> 00:08:14.840 Brian Klaas: A generative AI can also be really helpful for experiential learning and assessment. I don't know if any of you have used Claude's artifacts, or now Chatgpt has something similar called artifacts, but you can use these tools. I like Claude in particular. It's kind of my go to generative AI tool for any number of reasons. But one of the really cool things about these tools, both Chatgpt and Claude is, they can make 35 00:08:14.840 --> 00:08:39.739 Brian Klaas: interactive learning activities for you within seconds minutes with some refinement and continued sort of conversation with the generative. AI. So here, you see, I provided very simple, prompt. You're an expert epidemiologist, create interactive experience to teach students that correlation does not equal causation, use a food break outbreak as your example. The classic, you know. Epi! One sort of, you know, egg salad, picnic 36 00:08:39.740 --> 00:09:04.729 Brian Klaas: food, outbreak, example there, and make it really fun and interesting, and within 2 min Claude generated. I took a screenshot here of on the left, a whole sort of interactive exercise, where you collect patient data and then try to figure out, like what was the source of the illness here and again, the goal here is to teach the students that correlation doesn't equal causation here, and you can refine these. 37 00:09:04.730 --> 00:09:29.640 Brian Klaas: And what it generates is essentially an interactive exercise that you can share directly with your students wherever they are in the world. You don't have to have any programming knowledge whatsoever, or you can download these things and put them in your course website or host them somewhere else. And this is one example of any number of different kinds of interactive experiential learning tools you can use. And you can have students to again do the same thing, build their own interactive simulators that they have to then make 38 00:09:29.640 --> 00:09:34.869 Brian Klaas: work properly again to demonstrate understanding and mastery of the material next slide, please. 39 00:09:34.950 --> 00:09:58.749 Brian Klaas: And so the flip side of this, rather than creating things, you, as a faculty member or a ta even can work with the generative AI to create interactive learning experiences. In this case this is a role playing scenario that Ethan Malick. He's a faculty member at the Wharton School of business at Penn came up with. And the link on here. 40 00:09:58.750 --> 00:10:23.740 Brian Klaas: it takes you to the website that has, like the full, prompt, and how he sort of builds interactive simulations for his graduate level students. And don't worry. You're going to get a copy of these slides. You don't have to screenshot this or try to write this down. But again, you can create in-depth really powerful learning experiences. This is not a 1 second thing, right? But you can build these kind of experiential learning experiences that students then go 41 00:10:23.740 --> 00:10:42.919 Brian Klaas: through. You can grade them on the output. You can have the students capture all the conversation and then grade them on that, or again, on some kind of critical reflection about the experience that they have. You don't have to do that negotiation role playing in this case with, you know all 30 students in your course. You could have. I do it and evaluate the student work next slide, please. 42 00:10:43.410 --> 00:11:08.350 Brian Klaas: And one challenge that we all have about is with using generative AI or sorry teaching. Sorry, not teaching in general is providing timely feedback. This is a challenge for all of us. You know I teach courses. One of my courses taught in all terms except summer term, including Summer Institute. I have about 500 students a year, giving timely feedback on their work is really challenging. 43 00:11:08.350 --> 00:11:32.460 Brian Klaas: You can use generative AI to do this as part of a draft process for students. So most of us don't have time to say, Okay, I'm going to give you in-depth feedback on your draft version of your final paper. Well, you can use this as part of the process and have a prompt something like this one. This is actually a really detailed prompt. Again, you're going to get a copy of the slide so you can adapt this to your heart's content. But again, you can provide timely feedback for students 44 00:11:32.460 --> 00:11:51.480 Brian Klaas: in a way that yourself, as the faculty member or ta may simply not have the bandwidth to do, and this can become a part of the process of that sort of final paper or that final project for students. So you can incorporate AI into the development of sort of a cumulative assessment for students in your course 45 00:11:51.580 --> 00:11:52.889 Brian Klaas: next slide, please. 46 00:11:54.186 --> 00:11:59.520 Brian Klaas: And also, you know, I think another sort of reason to embrace AI is because students want it. 47 00:11:59.590 --> 00:12:23.389 Brian Klaas: Students need it. There was a study that came out at the end of last year by cengage, which is a sort of educational business partnership research firm. And in this particular survey 70% of the 2024 year graduates that, said that generative AI should be integrated into courses, and anecdotally, I'll follow this up with. I was talking with David Dowdy, who I teach this class with who's also, you know, the head of 48 00:12:23.390 --> 00:12:31.050 Brian Klaas: academics here at the school, the academic Dean here at the school, and he was meeting with some incoming masters and doctoral students just this last week, and he said the 49 00:12:31.050 --> 00:12:55.900 Brian Klaas: one question that those students had for him in their meeting these one on one or small group meetings was, are, how are you incorporating generative AI into our classes? We want to know. We need to be able to use this skill in the future. So, students, it wasn't just like, Hey, can you use chat? Gpt, to cheat? It was like, How are we going to be building skills that we're going to be able to carry forward in our time. Here at the Bloomberg School of Public Health 50 00:12:55.900 --> 00:13:24.539 Brian Klaas: workforce perspective. Generative AI was Number one on their minds, and the same study, the same sort of survey that's engaged in 62% of the employers that they surveyed believe that graduates should have some kind of foundational knowledge of generative AI, and that they're much more likely to interview and hire people with generative AI experience. So you're not only doing students yourself a favor. But you're also doing your students a big favor by considering how you might build skill building into your classes next slide, please. 51 00:13:26.096 --> 00:13:52.770 Brian Klaas: So that's some reasons why or how you might want to use gender AI and embrace it in your classes. I will talk for the next couple of minutes about trying to be transparent about this. This is really important, and this has to do with academic ethics. This has to do with cheating. But really it's about transparently doing this, so that students understand what's expected of them, and then use the tools in the ways that is optimal for their learning. That's what this is about next slide. 52 00:13:53.450 --> 00:14:18.179 Brian Klaas: So 1st and foremost, you've got to. If you're going to use generative AI in your assessments or in your courses at all, you need to stay upfront and be really transparent about what the learning goals are and why generative AI is being used in that assignment, because a lot of times students will be like, well, you know, this is, you know, work that I don't understand why I have to do this seems like no fun, or it's not interesting, or it's not helping me learn the things I want to learn. 53 00:14:18.180 --> 00:14:42.270 Brian Klaas: So they don't put a lot of effort into it. That's where cheating comes in. That's where academic ethics violations most frequently come in when students don't feel a meaningful connection to the work that they do. So. If you are very clear about the assignments learning goals, and why AI has been designed into those goals. That's really important. And this screenshot here of an example that we use in our generative. I class and actually use this in almost all my classes. 54 00:14:42.270 --> 00:15:05.480 Brian Klaas: How does this assignment fit into the rest of the course? Why am I asking you to do this work? And what are the goals of this assignment? Not just even like? What are the objectives of the assignment? But how does it fit in elsewhere, being really clear about the goals, and why using generative AI will make students more motivated and much less likely to misuse the tools in a way that makes you, as a faculty member, unhappy. Next slide, please. 55 00:15:07.889 --> 00:15:33.400 Brian Klaas: And again, you know, as part of this process, you need to discuss your expectations. Put this in the syllabus, in course, plus, there is a section that's available for any course, plus syllabus use of generative AI for class work, even if you're not using generative AI in your course at all, put it in your syllabus. Put this section in there and just say any use of generative AI, absolutely not right. That's like a big no, no, if you're requiring students to use specific tools. 56 00:15:33.520 --> 00:15:56.499 Brian Klaas: let's say you're going to require them to use midjourney. You're going to require them to use cloud code for their coding work or visual studio code or manus as sort of like an AI agent, or something like that. Then you need to let students know upfront, because not every student is comfortable using these tools. Not every student has access to these tools. But really, I think, more importantly, it's about what your expectations are. Put them in the syllabus. 57 00:15:56.500 --> 00:16:21.440 Brian Klaas: and also make it very clear that this is specific to your course. Only this is particularly important if you're teaching a foundational course here at the school, one of the large courses that are as a prerequisite in most degree programs or for most certificates, because students carry those expectations from one class to another. So you need to make it really clear, clear, as Jan Wernic would say, and tells me over and over again, make it really clear that these rules are for your specific class 58 00:16:21.440 --> 00:16:27.030 Brian Klaas: only, and don't apply to all classes that a student might take next slide, please? 59 00:16:29.498 --> 00:16:37.290 Brian Klaas: And then also, you know, you're going to need to demonstrate how to use these tools. Please do not assume that your students know how to use Chat Gpt. 60 00:16:37.330 --> 00:17:02.260 Brian Klaas: They probably do in some kind of basic foundational way. But there's plenty of students who have not used tools out of fear of using them in any way, shape or form might cause them academic ethics, violations. So especially if you're using a tool like, say, sizepace sizepace is a wonderful, awesome tool that I use a lot. It's great for searching through published literature, real journals, real sources that we would get 61 00:17:02.260 --> 00:17:26.289 Brian Klaas: and use in our own papers. But it's not a tool that is necessarily obvious to a lot of students. So if you want to have students use for a literature review, for example, and show them how to do it. We do this in our class through very simple videos that said, like, okay, we want you to do a deep research project using Google Gemini. This is how we go ahead and you do it right. They need some degree of instruction. 62 00:17:26.290 --> 00:17:37.199 Brian Klaas: I won't assume that they know how to use these tools, because they'll they'll make it up, use it improperly, or in a way that's not really advantageous to the learning process. So you got to show them how to do it. Next slide. Please 63 00:17:39.360 --> 00:18:04.320 Brian Klaas: also talk about data, privacy, your intellectual property, and how to give credit. These are important issues, right? Want to model behaviors for our students. We would have to worry about data, privacy, intellectual property, or giving credit if we were writing a paper and getting it published right. And in the same way we do the same thing with students. Make sure they understand that if they're using the free version of Chat Gpt, any data they put into Chat Gpt is used 64 00:18:04.320 --> 00:18:29.219 Brian Klaas: for training purposes or for business or commercial purposes. Later in the future. If you're using the free version, or even the paid version. Now, if you happen to be using, let's say, copilot. The University's version of copilot which you can sign into. And with your Jed credentials your data is protected. You don't have to worry about that being used for training purposes. Later on, most students realize there's a big difference between tools in terms of like 65 00:18:29.220 --> 00:18:53.399 Brian Klaas: are using it on the free version paid version, how do you turn if you're using the paper, how do you turn? You know the training off so that you're not potentially sharing data that you shouldn't be sharing with a 3rd party company that could be used for training purposes. The same thing with intellectual property. Right? You don't necessarily want to be putting a pre-publication into Chat Gpt, and then having it used for training purposes later on, before it's been published. 66 00:18:53.400 --> 00:19:04.770 Brian Klaas: It's proprietary data in the business that they work in and also putting, you know, the University provides the Welch Library provides some really good sort resources on how to cite generative AI tools. 67 00:19:04.770 --> 00:19:27.790 Brian Klaas: Point your students in those directions. These are important citation tools. They need to know how to do this because they should be citing every single time they use sort of AI, whether it's for brainstorming, creating images or writing texts or rewriting text, whatever it is. And you're going to need to go over that with them, because again, every faculty member has slightly different expectations on how they should be cited. Apa mla, and on and on and on 68 00:19:27.790 --> 00:19:29.100 Brian Klaas: next slide, please. 69 00:19:30.670 --> 00:19:55.619 Brian Klaas: and then make sure you're defining criteria for success. This is just good practice for any kind of assessment. Right? Use rubrics. So I teach now at 5 classes, with the other exception of multiple choice quizzes. I use rubrics for all of my assessments, even the ones in the generative AI course. And this is really helpful, because Number one, it says, this is how you can achieve success in your course or in this assignment, how you can achieve success using generative AI 70 00:19:55.620 --> 00:20:20.520 Brian Klaas: in this assessment, but it also significantly reduces fighting with students about grades. I rarely have to fight with students about grades, and I teach, you know, over 500 students a year, and that's a huge relief to me. And my students also provide example, final products. This can be difficult to do. The 1st time you might be embracing generative AI in a assessment. But it's also something that you should try to do. Certainly, if you've taught the course more than 71 00:20:20.520 --> 00:20:35.390 Brian Klaas: one time, give them examples or ideals to shoot, for they may not be perfect. And I tell my students this all the time. I say, here are examples of great work, or really good work or a level work. That doesn't mean they're perfect and flawless, which is something to remind them of as well. Next slide, please. 72 00:20:37.990 --> 00:21:03.779 Brian Klaas: and then finally, depending on the course you might need to provide options for students who can't or won't be able to use generative AI. Please remember that a number of generative AI tools that are available to us here in the States are not available students in countries like China or Syria or Uzbekistan, where there are sort of export controls in place. So if you're saying, Hey, go, use this tool. A student may not be able to use that tool, Google and all of its tools. For example, students in China cannot use out 73 00:21:03.780 --> 00:21:27.600 Brian Klaas: using an illegal firewall or legal VPN. System which is not cool and not something to be asking students to do, and you may have students depending on your class who will refuse to use generative AI because they simply feel like any use of generative. AI opens them up to academic ethics violations. It's a path. They simply don't want to go down. That may not be very common, but it is something you have to think about. But, more importantly, for those students who may be outside the Us. 74 00:21:27.868 --> 00:21:36.719 Brian Klaas: And whether or not they can actually access those tools, and if they can't, what are you going to do instead? Or how is that assessment going to be provided? An alternative form 75 00:21:37.360 --> 00:21:38.620 Brian Klaas: next slide, please. 76 00:21:40.840 --> 00:22:04.650 Brian Klaas: So here's a really brief example of how we use generative AI in our assessment. This is one assessment in the generative AI. Course I teach with David. Obviously, we're saying, Yes, you have to use generative AI in this class, because it's 1 of the goals. But again, we're trying to be effective. We're trying to be clear and transparent in. Why we're doing this, what the goals are for this particular assessment we give them specific tools that they can use. 77 00:22:04.650 --> 00:22:18.089 Brian Klaas: We're transparent about it. We provide very, very clear grading, criteria, step-by-step instructions for what they need to do in terms of the assignment, and then you don't see it here. But later on this page we have example work from other students as well. Next slide, please. 78 00:22:19.790 --> 00:22:42.719 Brian Klaas: And courseplus itself actually has some generative AI tools in it that you can use right now, if you happen to be teaching a class that has Ctl. Produced lectures, there is this review, quiz. Maker, that can go and take your. It takes the transcription of lectures and generates a sort of formative assessment style review, quiz, mostly multiple choice, true, false match from 2 list questions 79 00:22:42.720 --> 00:23:07.699 Brian Klaas: that you could then sort of look through and be like, Oh, hey! I could actually like use these 3 questions in a review, quiz at the end of a class to make sure that students understand content again. The great thing about generative AI is, it generates a lot of content very quickly, and you can create. Do this one time. You can do this 10 times in your class if you want, and we also have a tool called AI draft coach that's going to be available to all courses in the 4th term. It was sort of a you had to ask for an invitation 80 00:23:07.700 --> 00:23:19.169 Brian Klaas: previously. It's going to be available to everyone in the 4th term. And what this tool does is it basically gives feedback to students on papers, on written long form written documents. It does not rewrite 81 00:23:19.170 --> 00:23:44.149 Brian Klaas: right? So you're not telling. Students. Go to chat, gpt, and have chatgpt rewrite your paper for you. No, this is like this is a specific prompt that works behind the scenes that gives students feedback on how the content that they include the writing quality, the style and tone, and some specific improvements that they might want to use. And there's actually, I think this term, there's about 25 courses using it, and feedback is generally pretty positive about this again. Some students don't like it because it doesn't rewrite everything for 82 00:23:44.150 --> 00:23:50.189 Brian Klaas: them, but faculty like it, because it's an official tool that they can use that. They don't have to worry about cheating with 83 00:23:50.270 --> 00:23:51.630 Brian Klaas: next slide, please. 84 00:23:53.410 --> 00:24:16.550 Brian Klaas: So there's lots and lots of options available to you in terms of embracing generative AI in your classes, because I also do work on course, plus. I did want to take a moment just here to quickly. Just ask, you know, in course, plus from the faculty perspective. What kind of tools would you like to see for helping you build assessments for helping you design assessments for helping you 85 00:24:16.550 --> 00:24:36.890 Brian Klaas: deal with student work? What kind of generative AI tools would you like to see if you have any ideas? You could type them into the tech chat there. And I know there's a couple of questions that came up here. In the chat, and I will, I will respond to them. So, Fran, you want me to list the tools and links in the chat for us. I can. I can list some tools in there. If that would be helpful. 86 00:24:37.130 --> 00:24:38.370 Fran Burman: That'd be great. Thanks. 87 00:24:38.370 --> 00:24:41.040 Brian Klaas: Okay, yes, like, I assume you're talking about like the ones like. 88 00:24:41.040 --> 00:24:42.979 Brian Klaas: Well, you just started you. 89 00:24:43.200 --> 00:24:45.689 Fran Burman: You went through them so fast. 90 00:24:46.420 --> 00:24:49.859 Fran Burman: really quickly, and I have never heard the terms before, and. 91 00:24:50.185 --> 00:24:50.510 Brian Klaas: Okay. 92 00:24:50.510 --> 00:25:02.690 Fran Burman: So you just went through so fast. I have no idea what you were saying. I figured out eventually that Microsoft has copilot and something called Claude, but that was only from one of your slides. So. 93 00:25:02.690 --> 00:25:03.960 Brian Klaas: Okay. Fair enough. 94 00:25:03.960 --> 00:25:07.660 Fran Burman: Had time list them so we can explore them. That'd be great. 95 00:25:09.532 --> 00:25:25.910 Brian Klaas: I can definitely do that for you. And then Caitlin asks any suggestions for how we ourselves can efficiently learn about the basics of these AI tools without investing tons of hours you don't have. I totally hear that? It's a hard question. And really, honestly, Caitlin, the best way to do it is just start 96 00:25:25.990 --> 00:25:50.690 Brian Klaas: seriously, pick a tool whether it's chatgpt or copilot or Claude, and and start figure out something that where you feel like, okay, I need help brainstorming something right? This is something that we as a faculty, need to do all the time. I need to brainstorm an idea for a paper for a intervention for study design. I need to come up with a presentation topic for 97 00:25:50.690 --> 00:26:14.199 Brian Klaas: a conference. I'm working on a Grant proposal, and I need to come up with some ideas for how I might implement this Grant use a generative AI tool for that. They're surprisingly capable and surprisingly good, because by working with the tool directly you will have, you'll gain a lot of experience very, very quickly. The tools are pretty straightforward to use at this point. It's really figuring out 98 00:26:14.400 --> 00:26:34.601 Brian Klaas: where the tool works for you and where it doesn't. This sort of jagged frontier, as it's been described, like sometimes these tools are great for certain things, but not so great for others, and sometimes it depends on your prompts and sometimes not. It really is a very personal kind of thing in terms of figuring out what works best for you and what doesn't. Now that said 99 00:26:34.950 --> 00:26:58.379 Brian Klaas: if you want to learn the basics of all the tools. You are welcome always to take, David and my generative AI. Course. You can do that through tuition remission here at the school. We are offering it in a summer institute over a 2 week version. So that would definitely be one option, because it's a lot of practical hands-on exercises in terms of using generative AI tools 100 00:26:58.380 --> 00:27:20.020 Brian Klaas: along with video demonstrations of how like some of the main tools work for you as well. So that's that's definitely one option. And Lou, I'm sorry, Amy wrote also in the chat meet with an instructional designer. Ctl, they're very, very helpful, and they can definitely talk about how tools will work in your class, or what tools might work in your class. There. 101 00:27:20.020 --> 00:27:21.329 Brian Klaas: Celine, you have a comment. 102 00:27:21.830 --> 00:27:35.079 Celine Greene: Yeah, thanks, Brian. I'm pasting a brand new resource that I just put up on our website today. That was shared out through the educause community, and I'm actually going to be. I haven't yet 103 00:27:35.170 --> 00:27:55.969 Celine Greene: vetted its accessibility or anything. So that's why it's I still have to do it. But in there for the learning on your own. There's this whole section about free professional resources for educators that includes Khan Academy that has getting ready to teach with AI. So those are like another option. And I do want to say I attended 104 00:27:56.350 --> 00:28:06.110 Celine Greene: recent workshop put out from Ctei on AI prompt engineering for instructors. It was a very. It was a 1 h very 105 00:28:06.130 --> 00:28:32.309 Celine Greene: brand new to to AI, but most of the audience, and it was really well done. But again, Ctl. Works with other teaching and learning centers and our instructional design team. Actually, Amy does this great job of putting things on our events calendar. So even if it's not offered by our center for teaching and learning, we do post other opportunities. So I'm sure, as part of the follow up email. 106 00:28:32.310 --> 00:28:47.830 Celine Greene: Brian's course, this resource, and again pointing toward other teaching and learning centers across Hopkins they have some awesome opportunities. Again, Ctei Mike Reese had that AI prompt engineering for instructors. It was really. 107 00:28:47.850 --> 00:28:51.650 Celine Greene: really intro levels, and it was really well done. Thanks. 108 00:28:51.650 --> 00:29:14.889 Brian Klaas: Cool thanks for sharing that Celine and Daniel. Thank you for the suggestion about the sort of quiz Bill exam builder that can take previous exams, and sort of alter them and change them around a little bit, to still be accurate in terms of the questions, and and similarly creating similarly leveled questions. That's really interesting. I may follow up with you on that, Daniel. Because we're always looking for new opportunities to do exactly this kind of work for our faculty 109 00:29:15.348 --> 00:29:25.209 Brian Klaas: so I think that's about it for my section. So I'm gonna turn it over to Lou, who's gonna talk about resisting the use of AI in your assessments. 110 00:29:27.140 --> 00:29:28.459 Brian Klaas: or is it Amy sorry. 111 00:29:28.660 --> 00:29:32.270 BSPH CTL Teaching Toolkit (Amy Pinkerton): I think I think I'll start, and then I'll pass to Lou. But I have perfect 112 00:29:32.270 --> 00:29:57.129 BSPH CTL Teaching Toolkit (Amy Pinkerton): quick thing. But thank you all right. So so, as Brian mentioned, there's a lot of reasons why you want to why you might want to embrace AI. But there are also some valid reasons for resisting AI in your academic assessments and your course design. And I'm going to focus specifically on using student use of AI in their assessments. The 1st thing that comes to mind is you might want. 113 00:29:57.130 --> 00:30:19.710 BSPH CTL Teaching Toolkit (Amy Pinkerton): You might want to encourage original thinking in your students. AI tools can generate responses based on patterns in existing data. But they don't demonstrate original thought. They struggle with things like developing an argument or demonstrating really high level problem solving skills. Also, things like creativity. 114 00:30:19.710 --> 00:30:39.860 BSPH CTL Teaching Toolkit (Amy Pinkerton): So if you, if you want to encourage that original thought, then you might resist AI in an assessment. You also might have some academic integrity concerns. So again, AI generated work may not be the student's own work. So raising concerns about plagiarism and dishonest academic practices. 115 00:30:39.860 --> 00:30:58.870 BSPH CTL Teaching Toolkit (Amy Pinkerton): So by designing assessments that resist AI use, you can promote genuine student efforts and learning. And in this we're going to focus on real world problem solving tasks that require deep engagement. And we're going to talk about authentic assessment, and with that I will pass to Lou. 116 00:30:59.970 --> 00:31:01.202 Lu Chi: Hi, thank you, Amy. 117 00:31:01.630 --> 00:31:10.680 Lu Chi: So now let's look at a specific example. A data analysis project on the impact of public health policies. 118 00:31:10.990 --> 00:31:29.569 Lu Chi: Students work in groups to analyze real world data, apply statistical methods and present their findings. This type of assessment is AI resistant because it requires collaboration, critical thinking and real world application. 119 00:31:30.040 --> 00:31:31.490 Lu Chi: Next slide, please. 120 00:31:32.960 --> 00:31:40.589 Lu Chi: This is an example prompt. You can use to design or redesign your assessment to make a more AI resistant. 121 00:31:41.181 --> 00:31:52.139 Lu Chi: In this prompt, you need to identify the critical content of the assessment tab and then make sure it did. You know? Along with the learning objectives 122 00:31:52.930 --> 00:31:54.460 Lu Chi: next slide, please. 123 00:31:55.970 --> 00:32:09.700 Lu Chi: So to you know, resisting AI in assessment doesn't mean ignoring its existence. Instead, it's about designing assessment that prioritize the skills and the knowledge we want students to develop 124 00:32:09.940 --> 00:32:28.090 Lu Chi: when risking AI in assessments, transparency and intentional design are key. Start by seeing your rationale in the syllabus and emphasizing the value of original thinking and shareholder thinking skills, critical thinking and creativity 125 00:32:28.290 --> 00:32:35.340 Lu Chi: make learning objectives salient and remind students why AI avoidance is important. 126 00:32:35.620 --> 00:32:44.959 Lu Chi: Incorporate process documentation, formative assessments, and the reflection hold students accountable and encourage deep engagement. 127 00:32:45.980 --> 00:32:56.399 Lu Chi: also provide clear examples, and the rubrics, like a brand mentioned before. You know, to students and the focus on the learning process rather than the final. 128 00:32:56.840 --> 00:33:09.299 Lu Chi: the learning assessment that prioritize authentic learning experience. You can foster academic integrity and help students develop durable skills the AI cannot place 129 00:33:09.700 --> 00:33:18.199 Lu Chi: alright. So now let me turn to Amy again. She will discuss the authentic assessment that resistant students AI use. 130 00:33:19.540 --> 00:33:20.949 BSPH CTL Teaching Toolkit (Amy Pinkerton): Alright. Thank you, Lou. 131 00:33:21.360 --> 00:33:37.190 BSPH CTL Teaching Toolkit (Amy Pinkerton): all right. So I'm going to focus again on how to create authentic assessments as a strategy to resist AI and your students work. And so first, st I want to define what we mean when we say authentic assessment. 132 00:33:37.190 --> 00:34:00.250 BSPH CTL Teaching Toolkit (Amy Pinkerton): and this refers to real world assessments, or these are hands on tasks that require students to engage with complex, real world scenarios, audiences, and objectives. This contrasts with traditional academic tasks which often have no broader audience or purpose beyond the course. So when you think of an authentic assessment, think about a real 133 00:34:00.250 --> 00:34:15.850 BSPH CTL Teaching Toolkit (Amy Pinkerton): in real life, competency, or skill or task, that your student might practice in class, but has a clear, applicable, matching task. That would be that they might do in the real world within your field. 134 00:34:16.989 --> 00:34:28.890 BSPH CTL Teaching Toolkit (Amy Pinkerton): and again, the primary goal of authentic assessments is to promote deeper learning and understanding by encouraging students to go beyond just covering content, or gaining surface level familiarity. 135 00:34:29.699 --> 00:34:56.240 BSPH CTL Teaching Toolkit (Amy Pinkerton): and related to that. Authentic assessments are resilient because they require students to perform tasks that demonstrate meaningful application of essential knowledge and skills, and these are especially true when authentic assessments are developed around distinctly human skills and what are distinctly human skills. These are things that generative AI struggles with. 136 00:34:56.239 --> 00:35:16.919 BSPH CTL Teaching Toolkit (Amy Pinkerton): So these are capabilities that require human intervention to accomplish, and usually these involve nuanced judgment, adaptability, personal experience, and you can also tie these to creativity, emotional intelligence, ethical reasoning, and critical thinking. So the things that students. 137 00:35:16.920 --> 00:35:39.110 BSPH CTL Teaching Toolkit (Amy Pinkerton): you know, they might use AI a little bit, for like the lower level things. But these are things that you really need that human input to do well to demonstrate in an assessment. And we have a resource that we're going to share from you from Oregon State University's ecampus, where they've actually mapped distinctly human skills across Bloom's taxonomy. 138 00:35:39.475 --> 00:35:49.709 BSPH CTL Teaching Toolkit (Amy Pinkerton): So I encourage you to take a look at this resource. It'll help you determine where you might want to incorporate some distinctly human skills into your assessments. 139 00:35:52.050 --> 00:36:09.400 BSPH CTL Teaching Toolkit (Amy Pinkerton): And now I'm going to give you some examples of authentic assessment. So some examples include. And again, the key characteristics are real world relevance and complexity and depth. So, for example, multimedia recording targeted to a real world audience. 140 00:36:09.400 --> 00:36:33.279 BSPH CTL Teaching Toolkit (Amy Pinkerton): so instead of having students write a paper, they might record a short public health message or a statement that would be directed towards a targeted audience. Say, a population that's being impacted by a specific public health issue. Now, they might not actually ever send that out into the real world. But within the course they're creating something with that in mind. 141 00:36:33.380 --> 00:36:58.009 BSPH CTL Teaching Toolkit (Amy Pinkerton): Other examples debate aligned with real world contexts. You can kind of combine this with role play where they might take on the perspective of a stakeholder in a debate versus just their own perspective, or maybe both original research. So you might not have your students do an entire research project. But you might focus on one part or one step of the research 142 00:36:58.371 --> 00:37:03.800 BSPH CTL Teaching Toolkit (Amy Pinkerton): process that they would do that mirrors what they might do in real life 143 00:37:03.900 --> 00:37:17.809 BSPH CTL Teaching Toolkit (Amy Pinkerton): and things like mock interviews, and that again plays well with the role play. So those are just some examples of authentic assessments. But again, you want to focus on tasks that have real world relevance that students can practice in class. 144 00:37:19.080 --> 00:37:40.659 BSPH CTL Teaching Toolkit (Amy Pinkerton): Okay? And now we are actually going to do a really quick activity. And here are our instructions. So we're going to have you download a file from the chat. So I think, Lou, you're going to post the file in the chat. So what you're going to do is download this file. So you have your own copy of the word document. 145 00:37:40.870 --> 00:38:04.630 BSPH CTL Teaching Toolkit (Amy Pinkerton): and then so after you've downloaded it. We want you to take a look at our mock assignment, and this mock assignment was developed with the help of Chatgpt. So if you, when you see the slides later. You'll see my little citation if you want to see how I cited Chatgpt for this activity, and what we want you to do is, go through, read the assignment, and then we want you to choose an approach. So we want you to either embrace 146 00:38:04.720 --> 00:38:23.180 BSPH CTL Teaching Toolkit (Amy Pinkerton): generative AI or resist generative AI within this assessment, and then you'll have a couple minutes to make your revisions to the assignment, and then, when the timer ends, we want you to come back and hopefully share some of your revisions. Are there any questions before we get started? 147 00:38:27.470 --> 00:38:44.939 BSPH CTL Teaching Toolkit (Amy Pinkerton): All right? I don't see any questions in the chat. I don't think any hands are raised, so I'm going to go ahead and start the timer music, and once the timer is up we'll reconvene. But in the meantime take a look at the mock assignment and make your revisions. 148 00:38:45.110 --> 00:38:47.909 BSPH CTL Teaching Toolkit (Amy Pinkerton): I'm going to start the timer starting now. 149 00:41:49.120 --> 00:41:52.850 BSPH CTL Teaching Toolkit (Amy Pinkerton): Alright, it's been about 3 min. 150 00:41:53.490 --> 00:42:04.259 BSPH CTL Teaching Toolkit (Amy Pinkerton): If you could raise your hand. If you want additional time, or we can move on. So please raise your hand if you'd like more time on the activity. Okay, all right. Well, I'll give you 2 more minutes 151 00:43:28.740 --> 00:43:41.639 BSPH CTL Teaching Toolkit (Amy Pinkerton): alright, and we are at time. If you're still finishing up. That's okay. But in the meantime I'm gonna pass to Brian to to conclude our activity. 152 00:43:44.010 --> 00:43:45.510 BSPH CTL Teaching Toolkit (Amy Pinkerton): Oh, and, Brian, you're muted. 153 00:43:45.510 --> 00:44:07.270 Brian Klaas: Yeah, it usually helps if I unmute myself. So okay, here we are. So really quick, before anybody shares out necessarily. Just by a quick show of hands. How many of you chose to embrace generative Guy in rewriting? Just click the raise hand tool inside of zoom. If you would do that, please 154 00:44:08.780 --> 00:44:09.760 Brian Klaas: see 155 00:44:10.040 --> 00:44:21.440 Brian Klaas: waiting for people quick to raise hands. We got Fran. Oh, maybe not, mia, was mia. The only person who chose to embrace generative AI. Nobody wants to embrace generative AI. Oh! And Fran did as well. 156 00:44:21.800 --> 00:44:34.229 Brian Klaas: Glad to see that. Glad to see there's a couple of go-getters out there. Would either of you like to share how you chose to embrace generative AI in rewriting and and redoing this assessment? 157 00:44:35.410 --> 00:44:37.610 Brian Klaas: You just unmute yourself, Francis. No. 158 00:44:38.070 --> 00:44:49.619 Mia Lamm: Sure I can try. I needed more time, but I got a little lost in it. I started getting excited about it, but I actually used. I used a copilot. 159 00:44:49.810 --> 00:44:56.420 Mia Lamm: and I use copilot to help me. Redraft, the assignment. 160 00:44:56.670 --> 00:45:04.699 Mia Lamm: and I ended up after a few different approaches, took me a couple few tries to get 161 00:45:05.070 --> 00:45:13.910 Mia Lamm: in the right direction. I changed the assignment to a multimedia presentation. 162 00:45:14.663 --> 00:45:25.419 Mia Lamm: Analyzing the ethical implications of the AI and academics with a timeframe, and then some structure to 163 00:45:25.810 --> 00:45:36.440 Mia Lamm: with some argument of ethical concerns. So, taking 2 different approaches, using some authentic case study and a personal reflection 164 00:45:36.740 --> 00:45:42.850 Mia Lamm: in there. And so, as part of that, you know, also some structured 165 00:45:43.667 --> 00:45:51.470 Mia Lamm: approaches to helping the student and explaining how the AI can 166 00:45:51.600 --> 00:45:56.960 Mia Lamm: support and what they needed to be transparent about and cite 167 00:45:57.830 --> 00:45:58.460 Brian Klaas: Great. 168 00:45:58.460 --> 00:46:03.810 Mia Lamm: I think that was. I think that kind of that's kind of where that's as far as I got. 169 00:46:04.580 --> 00:46:26.529 Brian Klaas: That's wonderful. You're 3 quarters of the way there. It sounds like, I mean, you're talking about transparency and citation and transforming the assessment into something that races generative. AI, just as a sort of a quick question for you, me, were you using copilot inside of word to do this, or were you using, like the co-pilot website to do this. 170 00:46:26.800 --> 00:46:28.140 Mia Lamm: I was using the website. 171 00:46:28.340 --> 00:46:29.170 Brian Klaas: The website, cool. 172 00:46:29.500 --> 00:46:56.949 Brian Klaas: Because there is a copilot there in typical Microsoft fashion for those of you here at this school they do have multiple versions of copilot. There's a website that you can use, which is like Chatgpt, the website. But they also have a tool that a version of copilot that integrates directly inside outlook, word, Powerpoint excel. That in some ways is a little more capable in some ways is really annoying. It's not free. You can get on the you can ask it at Jh. To enable this in your version of 173 00:46:56.950 --> 00:47:00.469 Brian Klaas: Microsoft office for the low, low price of $36 a month. 174 00:47:00.470 --> 00:47:26.949 Brian Klaas: If that's something that you. It is available. I have used it myself, and I'll be perfectly and tell you I get much more value out of paying $2 a month for Chat Gpt. And $20 a month for Claude than I do out of Microsoft Copilot at this point in time. But and also remember when you're talking about when it was talking about converting like multimedia presentations. Just ask for a slide deck. 175 00:47:27.060 --> 00:47:55.619 Brian Klaas: If you do a simple Google search, you'll find 17 different tools that will automatically generate a Powerpoint deck for you, using a topic or a document as the source. You can even do this inside of Microsoft office with copilot embed into Microsoft office. So just turn a Powerpoint slide deck won't enough prevent from just using generative AI to do all the work they actually need to give the presentation need to be able to answer the questions that's authentic assessment, not generating Powerpoint slide 176 00:47:55.820 --> 00:48:09.780 Brian Klaas: alright. So who among you decide to reject generative AI say, and and harden their assessment against using generative AI. If you just raise your hands in the chat, I would greatly appreciate that, so we can see who did that. 177 00:48:11.520 --> 00:48:16.309 Brian Klaas: Anybody anybody reject? Caitlin's? No okay. 178 00:48:16.600 --> 00:48:18.600 Brian Klaas: Anyone else other than Caitlin. 179 00:48:19.280 --> 00:48:21.140 Brian Klaas: And Amy said, No. 180 00:48:22.020 --> 00:48:28.510 Brian Klaas: would either of you like to share how you sort of harden your assessment against the generative AI. 181 00:48:31.980 --> 00:48:38.359 Caitlin Kennedy: I can start. I I sort of was going back and forth, so I'm not sure if it was totally hardened 182 00:48:40.120 --> 00:48:41.355 Caitlin Kennedy: but I 183 00:48:42.920 --> 00:48:52.090 Caitlin Kennedy: changed it from a paper to a video recording and asked students to record a 5 min video of themselves. And I changed 184 00:48:52.690 --> 00:49:07.639 Caitlin Kennedy: a couple of the bullet point instructions to say, provide a personal example of how AI has related to ethical concerns in your own life. And then I changed the Apa format to a annotated bibliography. That shows how each source informed your thinking. 185 00:49:09.530 --> 00:49:10.980 Caitlin Kennedy: That said. 186 00:49:11.260 --> 00:49:17.460 Caitlin Kennedy: you know my next thoughts were, how are we assessing all these? I teach a course with over a hundred students. So 187 00:49:18.000 --> 00:49:28.869 Caitlin Kennedy: you know, was starting to think, Okay, how long would this video be? And if the video is any longer than 3 min or 5 min, multiplying the number of hours it might take to grade that. Watch them kind of thing. So that's where I was. 188 00:49:28.870 --> 00:49:55.069 Brian Klaas: Absolutely. Yeah, that's a real challenge. You know, I have the same challenge in the classes, the different classes that I teach and all faculty do right. How long will this take me to teach? And sometimes by making it more complex or more authentic? In some ways it's going to wind up taking more time to actually do the assessment we're actually looking. We're developing a tool in Ctl, that can sort of help faculty sort of assess this like, how long will it take something to? How long will it take me to grade this 189 00:49:55.070 --> 00:50:18.919 Brian Klaas: sort of an assignment feedback tool that has a number of different features. But one of the features in it is exactly that. If, like, if I'm actually going to do this assessment this particular way, can I do it. If I only have, say, 15 min per student to grade their work, and it uses generative AI to do some analysis and then give some feedback on clarity of instructions, is it? Can you do it in 15 min per student? 190 00:50:18.920 --> 00:50:31.800 Brian Klaas: Or if you're only giving students 2 days and an 8 week term to do this is that adequate time, those kind of feedback tools to help faculty sort of improve the the overall structure and quality of their assessments. And that's something that we're looking at rolling out, probably this summer. 191 00:50:32.310 --> 00:50:35.200 Brian Klaas: But thank you, Caitlin, for sharing that. I greatly appreciate that. 192 00:50:36.570 --> 00:51:04.739 Brian Klaas: And then Pam, Hi, pam! Good to see you, Pam said. Not sure this would be AI approved, but I asked students to evaluate the strengths and weaknesses of the various arguments for against, and ask them to envision. They are faster teaching the class and how they would embrace or resist genitive AI. And why? Okay, that's great. I mean role playing. That's something that Amy brought up in in the sort of list of sort of more authentic assessments that can help resist generative AI. And that's great, because I mean, yes, theoretically, a generative AI tool can do that. But 193 00:51:05.012 --> 00:51:25.440 Brian Klaas: if it's in the context of your specific course that becomes much more difficult for it to be specific to that course as opposed to sort of like a professor who teaches, you know, food, safety would talk about these things. Well, that may not be specifically what you're teaching, I know, like you teach courses on built environment and things like that, and that might help resist AI use in that kind of context. 194 00:51:26.010 --> 00:51:28.010 Brian Klaas: Any comments Amy Lou. 195 00:51:30.440 --> 00:51:33.986 BSPH CTL Teaching Toolkit (Amy Pinkerton): No other than we should probably start wrapping up. 196 00:51:34.380 --> 00:51:58.649 Brian Klaas: Okay, sounds good to me. So thank you all hopefully, you got out of today some ideas for both embracing and resisting the use of generative AI in your assessments. This is a process. And remember that my colleagues here in the center for teaching and learning, who are instructional designers, are amazingly good at working with you. You can bring one assessment to them. You can bring in a whole course full of assessments to them, and they'll gladly work with you on that process. 197 00:51:58.650 --> 00:52:23.429 Brian Klaas: And again, you will next slide, please, just as a couple of last few final resources that you might want to take a look at in addition to the one that just came out today, the one from Educause that Celine put into the chat there is on our own the center for teaching and Learning's Toolkit website, an extensive resource around artificial intelligence and teaching and learning that's created by the team here in the center for teaching and learning that you can review for 198 00:52:23.430 --> 00:52:27.680 Brian Klaas: resources, more ideas, more guidance, and the University itself has 199 00:52:27.680 --> 00:52:52.680 Brian Klaas: and has for about a year now had a website that has guidance on teaching with generative AI dealing with a broad range of topics same thing as the Ctl. Toolkit side, except it's more broadly applicable, and not unless public health specific. And then, finally, there's a link here to the distinctly human skills that can be applied to Bloom's taxonomy. If you're looking to resist the use of generative AI in your assessments. And again, you'll get a copy of these slides. So all of these links 200 00:52:52.680 --> 00:52:59.920 Brian Klaas: will be available to you, and I want to turn it over to Lou to wrap things up, or I might turn over to Amy. Amy forgotten already. Sorry. 201 00:53:00.380 --> 00:53:03.840 Brian Klaas: I've forgotten who I'm turning it over to. I'm turning it over to Amy to wrap things up. 202 00:53:03.840 --> 00:53:27.409 BSPH CTL Teaching Toolkit (Amy Pinkerton): All right. Thank you. So now, before you go, we just ask if you would take a few moments to do a quick, anonymous workshop evaluation survey to let us know how we did. We love hearing from you. We want to hear your feedback. We value your feedback, and we use this to plan other future workshops. So please take a moment to complete this anonymous workshop evaluation survey. 203 00:53:27.410 --> 00:53:39.879 BSPH CTL Teaching Toolkit (Amy Pinkerton): and while I have this open on the screen, if you have any any other final questions? We have a couple minutes left before noon, so if you have questions, we'll open the floor to questions as well. 204 00:53:43.310 --> 00:53:49.710 Pamela Berg: I, I actually have a question that you may not be able to answer if we 205 00:53:50.330 --> 00:53:58.349 Pamela Berg: strongly suspect that a student did use AI for an assignment when they were not asked not to. 206 00:53:59.210 --> 00:54:06.430 Pamela Berg: How would we approach that? Besides rewriting the assignment for next year, or finding other ways to 207 00:54:06.540 --> 00:54:09.670 Pamela Berg: embrace AI for an assignment. 208 00:54:12.100 --> 00:54:40.529 Brian Klaas: So in terms of approaching. Whether or not a student has used generative AI in terms of cheating, this is really difficult to do. That's a simple reality is that it's unless you have a say, long or large amount of direct student writing that's handwritten in a journal by the student. And you could say, This is your voice, and this is the voice that came across and sort of generated thing. It'd be extremely difficult to say that a student proved that student used a generative AI tool, unless, of course, they would come, and. 209 00:54:40.530 --> 00:54:55.080 Brian Klaas: like, you know, had lots of incorrect invalid links that were sort of clearly made up or statistics made up by a large language model. They literally copy and paste an entire paper into a word document that might have some obvious errors. 210 00:54:55.080 --> 00:55:19.679 Brian Klaas: Beyond that, you know all generative AI detection tools that are out there. They don't work. They're biased again against non-native English speakers. They're biased against people who write in a simple and clear declarative style, which is generally what we want. Their false positive rates are very high. Some of these tools, false, positive rates as high as 70%. These are commercial license tools that are encouraged that by some other institutions that faculty use. 211 00:55:19.680 --> 00:55:44.509 Brian Klaas: So that's why we here at Bloomberg basically say, like, look using AI detectors is a terrible idea. But you know, academic integrity concerns Jan Vernick's your best resource. I'm sure you've probably already talked to, Jan. It's very frustrating look. It's a very frustrating position to be in, and I feel like, in terms of designing for the future. The best way to do that is to 212 00:55:44.510 --> 00:56:06.590 Brian Klaas: explicit about how and why, how and where generative AI can be used in your classes. Because that gives students an option to use it. And doesn't make everything an adversarial sort of it doesn't turn everything into adversarial relationship between generative AI. How students are learning how students are creating their assessment or their outputs. And you yourself, as a faculty member. 213 00:56:06.590 --> 00:56:08.899 Brian Klaas: Celine, did you? Wanna you were waving? Did you want to jump in. 214 00:56:08.900 --> 00:56:25.279 Celine Greene: Yeah, as well. I ended up putting it in the chat that we actually have that FAQ of if you have concerns as they're happening. But when you went into the future, looking about the redesigned and just just to summarize 215 00:56:25.930 --> 00:56:29.110 Celine Greene: academic integrity concerns as they were. 216 00:56:29.260 --> 00:56:39.559 Celine Greene: as they relate to AI are no different than any other academic integrity concern. You should be approaching it as an academic integrity concern period. 217 00:56:39.710 --> 00:56:59.929 Celine Greene: AI just happens to be there right. But then, when you're talking about, how do we avoid this, or make just kind of like, how do I make a better quiz, or whatever the things that we've always talked about. Yeah, I mean, Brian alluded to a lot of it. And then the other part of it is, that's why Ctl. And center for teaching and learning 218 00:57:00.040 --> 00:57:10.820 Celine Greene: others exist is to help you in designing your learning activities when you need assistance, including these workshops and also the one-on-one attention. So if you have a 219 00:57:10.950 --> 00:57:36.229 Celine Greene: particular concern, we can visit one on one again. That tried and true thing that worked in 2023 might not work in 2024 might not work in 2025 we may. We have to evaluate, analyze, and evaluate and and redesign. But again, I just wanted to. My my main hand raise was like, Oh, oh, go to the FAQ for for the as it happens. But thanks. 220 00:57:39.640 --> 00:57:40.950 BSPH CTL Teaching Toolkit (Amy Pinkerton): All right. Any other questions. 221 00:57:41.300 --> 00:57:41.615 BSPH CTL Teaching Toolkit (Amy Pinkerton): Nope. 222 00:57:41.930 --> 00:57:42.630 Brian Klaas: Oh, sorry! 223 00:57:42.820 --> 00:57:45.339 BSPH CTL Teaching Toolkit (Amy Pinkerton): Oh, no, I was gonna ask the same thing. 224 00:57:47.770 --> 00:58:01.629 BSPH CTL Teaching Toolkit (Amy Pinkerton): Alright. I don't see any hands raised nothing else in the chat. So again. Thank you so much for joining us today. And you'll receive a follow up email with all of the resources and things that we shared today. Oh, celine, did you have one more thing. 225 00:58:01.630 --> 00:58:17.030 Celine Greene: I just wanted to say so. One of our colleagues, who was on, who is here now recently used AI for something that has nothing to do with teaching and learning, and I just want to encourage people to start to do that because you learn to write better prompts, mia. I don't know if you. 226 00:58:17.030 --> 00:58:18.470 Mia Lamm: Calling me out. 227 00:58:18.470 --> 00:58:22.349 Celine Greene: Yes, I am telling you so fun, and it's. 228 00:58:22.350 --> 00:58:23.350 Mia Lamm: Okay, I did. 229 00:58:23.350 --> 00:58:31.210 Celine Greene: That's how we can like kind of pivot, just grow more comfortable with and not feel like. It's this whole huge new world. It's just like. 230 00:58:31.460 --> 00:58:32.919 Celine Greene: just go ahead, mia. 231 00:58:32.920 --> 00:58:54.569 Mia Lamm: Okay, I'll share really quick. I'm sorry. Yes. So I was creating. I have a very big spring garden plan with a lot of vegetables, a lot of flowers, a lot of fruit trees, blah! Blah! Blah! A big one, and I was trying to come up with when to plant, and when to harvest and what fertilizer to use. And I was trying to document it all. And I went. I started getting all the information I went. Wait a minute. 232 00:58:54.800 --> 00:58:57.149 Mia Lamm: Would AI do this for me? 233 00:58:57.370 --> 00:59:10.209 Mia Lamm: And it did. It took a little massaging. It took a little work. But wow! Did that help me out? It was really fantastic. So I shared it with slings. I'm like, look at this. It's amazing. Saved me hours of work. So yeah. 234 00:59:10.210 --> 00:59:34.719 Brian Klaas: I can follow up with another story for those of you who don't know. I have a home in Puerto Rico. It's where I am right now in Rincon, Puerto Rico power, and the electrical grid is a serious issue here in Puerto Rico. We've only owned our home for about 6 months. We've always already lost a bunch of equipment to brownouts. Power outages happen all the time, but low power situations are really bad for delicate electronic equipment like my apple TV or refrigerator, or a microwave. 235 00:59:34.720 --> 00:59:59.709 Brian Klaas: And so we've lost enough stuff that my husband and I were like. Look, we got to figure out how we can do. How can we deal with low power situations? And so I used the deep research tool in Chat Gpt to do analysis that I'm living in Puerto Rico. Here's the situation. Here's how big our house is, and it took, you know, about 12 min to do this plan that I asked it to do. But it went out and did. Lots 236 00:59:59.710 --> 01:00:24.650 Brian Klaas: of research came up with very specific products that we could look at for not only battery backup, but like whole house power, massaging individual outlet power massaging. And it was great. And I could have spent literally, you know, 10 h searching through Amazon consumer reports, various websites, and it did it all for me. Now I will say that even in here there were a couple of bad links that went to non-existing products on Amazon. 237 01:00:24.650 --> 01:00:43.919 Brian Klaas: But the product name was correct. The link was bad, the product name was correct. But again, it's that kind of time saving that is actually quite accurate or quite helpful, especially now that more and more of the generative AI tools are getting linked up to direct and live web search, which wasn't the case 6 9 months ago. 238 01:00:46.290 --> 01:01:00.509 BSPH CTL Teaching Toolkit (Amy Pinkerton): Right? Right? Those were great examples of like non academic use. Let it let it help you save time. Anyway. So and speaking of time we are at time. So again, thank you so much for attending. We hope you enjoyed today's workshop. 239 01:01:01.540 --> 01:01:02.240 Lu Chi: Thank you.