{"id":2611,"date":"2023-01-20T15:03:39","date_gmt":"2023-01-20T15:03:39","guid":{"rendered":"https:\/\/blogs.manchester.ac.uk\/itl\/?p=2611"},"modified":"2023-11-27T09:13:19","modified_gmt":"2023-11-27T09:13:19","slug":"designing-better-assessmenta-through-innovative-testing-methodology","status":"publish","type":"post","link":"https:\/\/blogs.manchester.ac.uk\/itl\/2023\/01\/20\/designing-better-assessmenta-through-innovative-testing-methodology\/","title":{"rendered":"Designing Better Assessments through Innovative Testing Methodology"},"content":{"rendered":"

[et_pb_section bb_built=”1″ admin_label=”section” fullwidth=”on” next_background_color=”#000000″ global_module=”322″][et_pb_fullwidth_menu global_parent=”322″ _builder_version=”3.0.106″ menu_id=”7″ submenu_direction=”downwards” fullwidth_menu=”off” background_layout=”dark” module_class=”main-navigation” \/][\/et_pb_section][et_pb_section bb_built=”1″ fullwidth=”off” specialty=”on” prev_background_color=”#000000″ next_background_color=”#1d1d1d”][et_pb_column type=”2_3″ specialty_columns=”2″][et_pb_row_inner locked=”off”][et_pb_column_inner type=”4_4″ saved_specialty_column_type=”2_3″][et_pb_post_title _builder_version=”3.17.6″ featured_placement=”above” locked=”on” parent_locked=”off” featured_image=”off” comments=”off” \/][et_pb_text _builder_version=”3.17.6″ background_layout=”light” parent_locked=”off”]<\/p>\n

\"\"<\/span><\/span><\/em>David Schultz is Professor of Synoptic Meteorology within the Department of Earth and Environmental Sciences. He is the author of the award-winning\u202fEloquent Science: A Practical Guide to Becoming a Better Writer, Speaker, and Atmospheric Scientist\u202fand has published over 180 peer-reviewed journal articles on meteorological research, weather forecasting, scientific publishing, and education. David is a Senior Fellow of the Higher Education Academy (SFHEA) and twice winner of the University Teaching Excellence Award.\u00a0<\/p>\n

In this post he talks about what prompted him to propose his ITL Fellowship Project (2021\/22), and the key takeaways.\u00a0\u00a0<\/em><\/p>\n

Your ITL Fellowship project focused primarily on designing better assessment questions. Can you set the scene for us on why you chose this as your focus?<\/b>\u00a0<\/p>\n

The project came about because of the perpetually low scores for assessment and feedback in the NSS (National Student Survey). I always seek feedback from my students beyond the University\u2019s standard UEQs (unit evaluation questions) because I want specific information about my specific assignments, to make them better. Those surveys told me that the students appreciated my feedback that I gave them on their assignments, so I knew I must be doing something right.\u00a0<\/p>\n

Also, I came to this country in 2009\/2010 from the USA where we didn’t have this tradition of sitting final exams several weeks after the course is over to the same degree as here in the UK.\u00a0 In my experience, there was more opportunity for feedback between the instructor and the students in the USA. So, I think that has influenced my approach to assessment in general.\u00a0\u00a0<\/p>\n

When my co-instructor Rhodri (Jerrett) and I were tasked with creating a 100-point multiple-choice question final exam for the course that we taught, we agreed we wanted to do something different within this format.\u00a0\u00a0<\/p>\n

The Director of our Institute of Teaching and Learning loves a BOGOF (Buy One, Get One Free)!\u00a0 Your project exemplifies this perfectly. Can you explain?<\/b>\u00a0<\/p>\n

Well if our assessment format was going to be multiple-choice, we wanted to include more application-type questions. My pitch when bidding for a fellowship was that these kinds of questions serve two key benefits:\u00a0\u00a0<\/p>\n

    \n
  1. Higher-level thinking skills:<\/b> It raises where we are hitting on Bloom\u2019s taxonomy to the \u2018application and evaluation and synthesis and creation of new knowledge\u2019 level rather than the questions testing simply \u2018memorization and understanding\u2019, which lie at the bottom of the pyramid. So that was certainly a goal. If we couldn’t ask essay questions or do other kinds of assessments, then then we could at least get the students thinking along those lines through these multiple-choice questions that we devised.\u00a0\u00a0<\/li>\n
  2. Assessment security:<\/b> The second issue was to ensure that we kept the test as fair as possible. We knew that, because typical multiple-choice questions are pretty straightforward, it\u2019s possible for a student to just look up answers. We knew that during 2020\/21 students were going to be taking the test at home, so it was important to us that that we design it so that a student couldn’t cheat on it. And that even if a student was tempted to just look up an answer, it would be hard to find answers for our questions online. That’s why the title of my project was \u2018How to design cheat-proof multiple-choice questions\u2019.\u00a0<\/li>\n<\/ol>\n

    \"\"<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    \u00a0<\/p>\n

    Fig. 1 Bloom\u2019s taxonomy<\/span><\/span>\u00a0<\/span><\/em><\/p>\n

    \u00a0<\/p>\n

    Why assess by means of a multiple-choice exam? I’m guessing that you would ideally have chosen a different assessment type, but there are pros as well as cons to multiple-choice questions\u2026 Can you give us a sort of defence of the multiple-choice questions?\u00a0<\/b>\u00a0<\/p>\n

    I know that multiple-choice questions are controversial, but I think there probably there are pros as well as cons to them. I don’t have anything in principle against multiple-choice questions as part of a balanced diet:\u00a0<\/p>\n