![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Amit-N-Jinabhai-300x271.jpg\")
Dr\u202f<\/span><\/span>Amit N. Jinabhai<\/span><\/span><\/a>\u202f<\/span><\/span>is a Senior Lecturer in Optometry, working in the Faculty of Biology, Medicine and Health.\u202fHe won\u202f<\/span><\/span>a Teaching Excellence Award (2021)<\/span><\/span><\/a>\u202ffor his ability to\u202fteach\u202fchallenging content\u202fin innovative and engaging ways, to positive effect on both student feedback and performance<\/span>, earning him Honorary Fellowship of the University\u2019s Institute of Teaching and Learning<\/span>.\u202fAmit\u202fis<\/span> also a former\u202fwinner\u202fof the\u202f<\/span><\/span>Association of Optometrists (AOP) \u2018national\u2019 Lecturer of the Year Award<\/span><\/span><\/a>.<\/span> Amit was also awarded <\/span><\/span>\u2018Division Teacher of the Year 2021\u2019<\/span><\/span><\/a> for the Division of Pharmacy and Optometry.<\/span><\/span>\u00a0<\/span><\/p>\n\u00a0<\/p>\n
Introduction<\/i>\u00a0<\/p>\n
Like many academics, one of my biggest challenges in recent years has been designing open-book, on-line summative examinations that were robust against both collusion and plagiarism.\u00a0<\/p>\n
From a broad pedagogical point-of-view, summative assessments (SAs) differ greatly from formative assessments, in that SAs typically:\u00a0<\/p>\n
- \n
- Occur at \u2018a fixed point in time\u2019 only, most likely at the end of an instructional period of learning,\u00a0\u00a0<\/li>\n
- Are designed to assess the unit\u2019s intended learning outcomes (ILOs), which can often serve as a helpful standard, or \u2018benchmark\u2019,\u00a0<\/li>\n
- Are \u2018high-stakes\u2019 assessments, in that they substantially contribute to the final grade for that unit,\u00a0<\/li>\n
- Are helpful for ranking student performance against their peers,\u00a0<\/li>\n
- Assist a programme team in identifying students who are not performing at the appropriate level,\u00a0<\/li>\n
- Assist a programme team in identifying students that should be permitted to progress onto the next stage of study.\u00a0<\/li>\n<\/ul>\n
\u00a0<\/p>\n
My dilemma\u00a0<\/i>\u00a0<\/p>\n
Most units on the Optometry programme, on which I primarily teach, were amenable to on-line, essay-based SAs that were submitted through TurnItIn to check for plagiarism and\/or potential collusion. In stark contrast, my four units are centred on ILOs that are unsuitable for essay-based SAs, as they predominantly involve calculations. This led me to carefully reflect on a key question \u2026 if I want to set questions on an open-book SA, which are based on calculations, how am I going to prevent my students from potentially colluding?<\/i><\/b>\u00a0<\/p>\n
Conceptualisation of a valid approach<\/i>\u00a0<\/p>\n
Initially I researched and considered various methods that had previously been used to prevent collusion and other forms of academic malpractice in open-book SAs. I also discussed my concerns with colleagues outside of the University. Through my research and\u00a0 discussions, I decided on four important approaches that I believed to be practicable and deliverable:\u00a0<\/p>\n
- \n
- Randomisation of the order of the questions,\u00a0<\/li>\n
- Setting a defined \u2018assessment window\u2019 time-period, beyond which the exam cannot be started nor submitted,\u00a0<\/li>\n
- Preventing back-tracking to previously answered questions,\u00a0<\/li>\n
- Setting a defined exam \u2018expiry\u2019 time-period, beyond which the exam gets \u2018cut-off\u2019 and automatically submitted.\u00a0<\/li>\n<\/ul>\n
I chose to use Blackboard as my platform for delivering my SAs for each of my different units, as it enabled implementation of all four approaches simultaneously.\u00a0\u00a0<\/p>\n
Calculated Formula Questions Within Blackboard\u00a0<\/i>\u00a0<\/p>\n
Blackboard supports a \u2018Calculated Formula Question\u2019<\/i> function, facilitating creation of calculation-based questions using \u2018individual variables\u2019 instead of discrete numbers. Figure 1 exemplifies a question that I programmed using \u2018variables\u2019.\u00a0<\/p>\n
Figure 1.<\/strong> An example calculation question in Blackboard. Calculatable variables are presented in blue <\/strong>with square brackets.<\/p>\n
Blackboard also permits users to \u2018programme in\u2019 the full equation that answers the question \u2013 see Figure 2.<\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/dfsgfsdgfdgsg-300x89.png\")
<\/p>\nFigure 2.<\/strong> The equation \u2018programmed\u2019 into Blackboard which answers the question from Figure 1. Calculatable variables are shown in blue<\/strong>.<\/p>\n
Blackboard also specifies important \u2018answer-related\u2019 options that can be selected to align with how learners must provide their final answer; e.g. the number of required decimal places, as well as defining how many different \u2018answer sets\u2019 should be created for each question \u2013 see Figure 3.<\/p>\n
\u00a0<\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Figure-3-300x163.png\")
<\/strong><\/p>\n\u00a0<\/strong><\/p>\n
Figure 3.<\/strong> An example of the \u2018options\u2019 for the seven variables<\/strong> defined in the question\/equation (see Figures 1 and 2). This includes the maximum<\/em> and minimum<\/em> values for each variable, the number of decimal places<\/em> each value is presented to within the question, the number of decimal places required for the final answer,<\/em> and the number of<\/em> different \u2018answer sets\u2019 <\/em>programmed for each question.<\/p>\n
A sample of the individual variable values<\/em> and their associated final answer<\/em>s is shown in Figure 4, highlighting that each calculation question can be programmed into Blackboard to generate different<\/em> sets of variable values, resulting in different<\/em> final answers. To ensure each student sees a different<\/em> question from their peers (thereby minimising collusion<\/em>), programmers must ensure that the number of \u2018answer sets\u2019 for each individual question matches the number of students taking the assessment.<\/p>\n
\u00a0<\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Figure-4-300x110.png\")
<\/p>\nFigure 4.<\/strong> Nine examples (NB: \u2018answer sets\u2019<\/em> are organised in \u2018rows\u2019<\/strong>) of seven different<\/em> variables (see column titles<\/strong>) and their corresponding<\/em> \u2018calculated answers\u2019<\/em><\/strong> via the equation in Figure 2. Although this image only shows nine \u2018answer sets<\/em>\u2019, any<\/em><\/strong> required number can be programmed.<\/p>\n
Once all the desired calculation questions have been \u2018programmed in\u2019, an \u2018assessment\u2019 can be created anywhere within Blackboard, permitting importation of these questions.<\/p>\n
Figure 5 displays some crucial options to further minimise collusion, including a timer function limiting how long students have to complete their SA. Immediately upon this time-limit\u2019s expiry, the assessment is \u2018automatically\u2019 submitted.<\/p>\n
\u00a0<\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Figure-5-300x135.png\")
<\/p>\n\u00a0<\/p>\n
Figure 5. <\/strong>An example of the timer settings, display dates and times that can be programmed into the assessment.<\/p>\n
\u00a0<\/p>\n
Incorporating Disability Advisory Support Service (DASS) requests:<\/u><\/em><\/p>\n
Students receiving DASS-recommended support can be easily assisted using this assessment format. Figure 6 exemplifies how two anonymised students were permitted to complete their SAs within 150 mins (support of 25% extra time) rather than 120 mins.<\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Figure-6-300x118.png\")
<\/p>\n\u00a0<\/p>\n
\u00a0<\/p>\n
Figure 6. <\/strong>An example of the \u2018test availability exceptions\u2019 settings available to programmers.<\/p>\n
Finally, Figure 7 highlights the crucial settings of:<\/p>\n
- \n
- Randomising the order of the questions,<\/li>\n
- Ensuring that each question is presented one at a time only, and<\/li>\n
- Prohibiting backtracking.<\/li>\n<\/ul>\n
I endorse all three options, as they collectively reduce the risk of collusion.<\/p>\n
\u00a0<\/strong><\/p>\n
![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2022\/10\/Figure-7-300x101.png\")
<\/p>\n\u00a0<\/p>\n
\u00a0<\/p>\n
Figure 7. <\/strong>An example of the crucial options which relate to the presentation of the assessment.<\/p>\n
\u00a0<\/p>\n
Conclusion<\/u><\/em><\/p>\n
My innovative approach demonstrates a sophisticated method of creating calculation-based SAs that are robust to collusion. Our external examiners provided extremely positive feedback about this approach for my on-line SAs, which included the following comments:<\/p>\n
\u201cI thought your approach to on-line assessments, and in particular how you managed to safe-guard against collusion, was novel and inspired. I like the idea of providing different questions for each student …\u201d<\/em><\/p>\n
Prof Loffler (Glasgow Caledonian University)<\/em><\/p>\n
\u00a0<\/p>\n
\u201cExcellent paper. I REALLY like the idea of changing the values \u2013 super! … well done and great to see.\u201d <\/em><\/p>\n
Prof Allen (Anglia Ruskin University) <\/em><\/p>\n
\u00a0<\/em><\/p>\n
\u201cThis is a fantastic concept \u2026 It is very innovative and I don’t recall having seen anything like it before. I do agree that this approach will remove any prospect of academic misconduct.\u201d<\/em><\/p>\n
Dr Hamilton-Maxwell (Cardiff University)<\/em><\/p>\n
\u00a0<\/p>\n
Further Reading:<\/strong><\/u><\/em><\/p>\n
Amit has also written for Times Higher Education’s Campus<\/strong>+ on\u00a0 Intended Learning Outcomes<\/strong>, which you can check out below:<\/p>\n
https:\/\/www.timeshighereducation.com\/campus\/intended-learning-outcomes-ilos-sideshow-teaching-tool\u00a0<\/a><\/p>\n
If you are interested in also contributing to Times Higher<\/strong>, please fill in the below survey, and we can organise submission of your content:<\/p>\n
https:\/\/www.qualtrics.manchester.ac.uk\/jfe\/form\/SV_8J3W59DtAZ1kqz4\u00a0<\/a><\/p>\n
[\/et_pb_text][et_pb_post_nav _builder_version=”3.0.106″ in_same_term=”off” show_prev=”on” show_next=”on” prev_text=”Previous” next_text=”Next” locked=”on” parent_locked=”off” \/][et_pb_comments _builder_version=”3.17.6″ show_count=”off” header_font_size_tablet=”51″ header_line_height_tablet=”2″ custom_button=”on” button_text_color=”#ffffff” button_use_icon=”off” locked=”on” parent_locked=”off” button_text_size__hover_enabled=”off” button_text_size__hover=”null” button_one_text_size__hover_enabled=”off” button_one_text_size__hover=”null” button_two_text_size__hover_enabled=”off” button_two_text_size__hover=”null” button_text_color__hover_enabled=”off” button_text_color__hover=”null” button_one_text_color__hover_enabled=”off” button_one_text_color__hover=”null” button_two_text_color__hover_enabled=”off” button_two_text_color__hover=”null” button_border_width__hover_enabled=”off” button_border_width__hover=”null” button_one_border_width__hover_enabled=”off” button_one_border_width__hover=”null” button_two_border_width__hover_enabled=”off” button_two_border_width__hover=”null” button_border_color__hover_enabled=”off” button_border_color__hover=”null” button_one_border_color__hover_enabled=”off” button_one_border_color__hover=”null” button_two_border_color__hover_enabled=”off” button_two_border_color__hover=”null” button_border_radius__hover_enabled=”off” button_border_radius__hover=”null” button_one_border_radius__hover_enabled=”off” button_one_border_radius__hover=”null” button_two_border_radius__hover_enabled=”off” button_two_border_radius__hover=”null” button_letter_spacing__hover_enabled=”off” button_letter_spacing__hover=”null” button_one_letter_spacing__hover_enabled=”off” button_one_letter_spacing__hover=”null” button_two_letter_spacing__hover_enabled=”off” button_two_letter_spacing__hover=”null” button_bg_color__hover_enabled=”off” button_bg_color__hover=”null” button_one_bg_color__hover_enabled=”off” button_one_bg_color__hover=”null” button_two_bg_color__hover_enabled=”off” button_two_bg_color__hover=”null” \/][et_pb_text admin_label=”Related posts” _builder_version=”3.0.106″ background_layout=”light” locked=”on” parent_locked=”off”]<\/p>\n
Related<\/h1>\n
Related posts:<\/h3>
<\/div><\/div><\/div>Black History Month - October 2022<\/h2><\/div><\/div><\/a>
<\/div>Enhancing Feedback on Continuous Assessment<\/h2><\/div><\/div><\/a>
<\/div>Why Showing Up Matters: Unpacking the Power of Attendance in Higher Education<\/h2><\/div><\/div><\/a><\/div>
<\/div><\/div><\/div>\n[\/et_pb_text][\/et_pb_column_inner][\/et_pb_row_inner][\/et_pb_column][et_pb_column type=”1_3″][et_pb_sidebar _builder_version=”3.0.106″ area=”sidebar-1″ orientation=”left” show_border=”off” background_layout=”light” locked=”on” \/][\/et_pb_column][\/et_pb_section][et_pb_section bb_built=”1″ admin_label=”Connect Title” custom_padding=”0px||0px|” _builder_version=”3.0.90″ background_color=”#1d1d1d” module_class=”footer-section” prev_background_color=”#ffffff” global_module=”380″][et_pb_row admin_label=”row” global_parent=”380″ locked=”off” allow_player_pause=”off” parallax=”off” parallax_method=”on” parallax_1=”off” parallax_method_1=”off” parallax_2=”off” parallax_method_2=”off” parallax_3=”off” parallax_method_3=”off” make_fullwidth=”off” use_custom_width=”off” width_unit=”on” use_custom_gutter=”off” make_equal=”off” module_id=”footer-row” _builder_version=”3.0.89″][et_pb_column type=”1_3″][et_pb_text admin_label=”Contact Us Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n
Contact Us<\/h4>\n
[\/et_pb_text][et_pb_text admin_label=”Contact Us” global_parent=”380″ text_text_color=”#aaaaaa” parent_locked=”off” background_layout=”light” text_orientation=”left” use_border_color=”off” border_color=”#ffffff” border_style=”solid” module_class=”contact-us” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n
+44 (0) 161 306 6000<\/p>\n
[\/et_pb_text][\/et_pb_column][et_pb_column type=”1_3″][et_pb_text admin_label=”Find Us Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n
Find Us<\/h4>\n
[\/et_pb_text][et_pb_text admin_label=”Address” global_parent=”380″ background_layout=”dark” text_orientation=”left” use_border_color=”off” border_color=”#ffffff” border_style=”solid” module_id=”address” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n
The University of Manchester
Oxford Rd
Manchester
M13 9PL
UK<\/a><\/p>\n[\/et_pb_text][\/et_pb_column][et_pb_column type=”1_3″][et_pb_text admin_label=”Connect Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n
Connect With Us<\/h4>\n
[\/et_pb_text][et_pb_code admin_label=”Social Media” global_parent=”380″ parent_locked=”off” _builder_version=”3.17.6″]<\/p>\n
- <\/p>\n
-
![\"Facebook](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-facebook-50.png\")
<\/a><\/li>\n<\/p>\n
![\"Twitter](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-twitter-50.png\")
<\/a><\/li>\n<\/p>\n
![\"YouTube](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-youtube-50.png\")
<\/a><\/li>\n<\/p>\n
![\"Instagram](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-instagram-50.svg\")
<\/a><\/li>\n<\/ul>\n
[\/et_pb_code][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"
Dr\u202fAmit N. Jinabhai\u202fis a Senior Lecturer in Optometry, working in the Faculty of Biology, Medicine and Health.\u202fHe won\u202fa Teaching Excellence Award (2021)\u202ffor his ability to\u202fteach\u202fchallenging content\u202fin innovative and engaging ways, to positive effect on both student feedback and performance, earning him Honorary Fellowship of the University\u2019s Institute of Teaching and Learning.\u202fAmit\u202fis also a former\u202fwinner\u202fof the\u202fAssociation […]<\/p>\n","protected":false},"author":5160,"featured_media":2535,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","_exactmetrics_skip_tracking":false,"footnotes":""},"categories":[15,18,21],"tags":[28],"class_list":["post-2534","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-inclusive-teaching","category-quality","category-student-engagement","tag-academic-integrity"],"_links":{"self":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2534","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/users\/5160"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/comments?post=2534"}],"version-history":[{"count":15,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2534\/revisions"}],"predecessor-version":[{"id":2551,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2534\/revisions\/2551"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/media\/2535"}],"wp:attachment":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/media?parent=2534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/categories?post=2534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/tags?post=2534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
-