![\"\"](\"https:\/\/blogs.manchester.ac.uk\/itl\/wp-content\/uploads\/sites\/262\/2023\/09\/Screenshot-2023-09-20-115256-298x300.jpg\")
<\/p>\n[et_pb_section bb_built=”1″ admin_label=”section” fullwidth=”on” next_background_color=”#000000″ global_module=”322″][et_pb_fullwidth_menu global_parent=”322″ _builder_version=”3.0.106″ menu_id=”7″ submenu_direction=”downwards” fullwidth_menu=”off” background_layout=”dark” module_class=”main-navigation” \/][\/et_pb_section][et_pb_section bb_built=”1″ fullwidth=”off” specialty=”on” prev_background_color=”#000000″ next_background_color=”#1d1d1d”][et_pb_column type=”2_3″ specialty_columns=”2″][et_pb_row_inner locked=”off”][et_pb_column_inner type=”4_4″ saved_specialty_column_type=”2_3″][et_pb_post_title _builder_version=”3.17.6″ featured_placement=”above” locked=”on” parent_locked=”off” featured_image=”off” comments=”off” \/][et_pb_text _builder_version=”3.17.6″ background_layout=”light” parent_locked=”off”]<\/p>\n
<\/p>\n
Cesare is a Teaching Fellow soon to be lecturer in the Department of Mathematics and a Senior Fellow of the Higher Education Academy.\u00a0\u00a0Here, he outlines some common features of Large Language Model (LLM)-created writing with Esme Davies, Higher Education Intern in the ITL, and discusses the implications of this technology for how we might assess students\u2019 knowledge and skills in future.\u00a0\u00a0<\/p>\n
\u00a0<\/p>\n
\u00a0<\/p>\n
(GPT stands for Generative Pre-trained Transformer \u2013 a Large Language Model (LLM) type of artificial intelligence (AI) that generates written content from scanning massive online datasets).\u00a0<\/em><\/p>\n There is an important distinction between work that is entirely written by GPT, and work that is co-produced content. Co-produced content is much harder to detect as it contains human elements which vary person to person.\u00a0<\/p>\n However, this last feature does not necessarily always mean content is GPT-written. Weaker students are known to rely on re-defining basic concepts, sometimes to increase their word count.\u00a0 AI will continue to get better so we need to change how we assess students; in the meantime, we can take advantage of the current pitfalls of AI.\u00a0\u00a0<\/p>\n Personally, I am against the use of AI detection tools. Companies train these detectors on specific data sets, but the actual uses of AI are so rich and big and varied that once you go beyond the training data set, detectors are not 100% reliable. For example, there is evidence that texts written by non-native English speakers are particularly vulnerable to being incorrectly flagged as AI-generated, probably because their sentences have a more predictable structure.\u00a0\u00a0<\/p>\n Even if AI detectors were to reach a level of reliability that eliminated most false positives and negatives, we are still moving in a direction where we want<\/i> content to be co-created with AI. This could be for many reasons: write better, save time, for accessibility reasons. The word processor that students use may soon have AI already built in. Employing detectors would disadvantage people who use AI for legitimate reasons.\u00a0\u00a0<\/p>\n The guiding principle of many institutions, including the government, is that we should aim to train people in using AI responsibly and skilfully. It will be interesting to see how assessments might develop to be able to incorporate effective and appropriate use of AI since we know that assessments are one of the main motivators of learning for students.\u00a0<\/p>\n For example, we need to develop more authentic assessments \u2013 possibly some that integrate AI where appropriate, and in other cases explicitly exclude it. One analogy is non-calculator exam papers: when we do not want students to make use of a calculator, we create controlled conditions where this can be ensured, but outside of the exam room we acknowledge the existence of calculators, and we teach them to use them critically and properly.\u00a0<\/p>\n Machine based detection should not be part of the process at all, but the lecturer\u2019s insight and knowledge of their students and what they are capable of is key.\u00a0\u00a0<\/p>\n Students cheat for a variety of reasons. There have always been ways for this to happen, plagiarism through word scramblers, essay mills, asking a friend. ChatGPT only makes cheating easier and faster, but this not an entirely new thing. The best thing to aim for is to try and motivate students not to cheat by changing assessments, and in the long run this will mean moving towards new types of assessment that include the possibility of co-creation with AI.\u00a0<\/p>\n However, this is a long process. Mitigation in the short-term could mean more invigilated exams, but not just that! For example, effective interim measures might include:\u00a0\u00a0\u00a0<\/p>\n None of these suggestions is exempt from caveats, and there is no one-size-fits-all answer.\u00a0<\/p>\n Colleagues should explicitly tell their students their approach to Artificial Intelligence and associated assessment rubric, making it clear to students what is good and bad usage of AI and setting out their expectations.\u00a0\u00a0<\/p>\n Students need to be taught the best uses for AI, such as spellcheck, or summarising, whilst also having awareness of AI\u2019s pitfalls. This will require an honest conversation with students and, depending on the course, this conversation can vary wildly. I have things I would say to my fourth year Algebra course that I would not repeat to my first-year maths students, largely because AI is currently pretty good at first year maths, and terrible at fourth year maths.\u00a0<\/p>\n Colleagues could put their own assignments into ChatGPT and adjust the assignment, or just to have some idea of what ChatGPT written work would look like for that essay question.\u00a0\u00a0<\/p>\n Colleagues need to be trained on AI, how models work, common pitfalls, what models can and cannot do. There are damaging misconceptions such as overconfidence in ability to identify AI-written content. Every peer-reviewed study I have seen on the topic shows that humans consistently fail to identify AI-written content and often misclassify it as human-produced, especially with more advanced models.\u00a0<\/p>\n Support with workload will also be needed. It is undeniable that more authentic assessment needs more time. For instance, monitoring drafts and sending regular feedback takes more time than just marking the whole essay once. In the end authenticity comes down to a human element, taking time to speak to other humans.\u00a0\u00a0<\/p>\n Separate suggestions on providing an Assesment Programme such as Cadmus and requiring students to work only within its bounds could be effective at the moment. However, this approach comes with drawbacks related to privacy and is vulnerable to AI learning to mimic human behaviour.\u00a0<\/p>\n A fundamental element in the following years will be enhancing scholarship and networking, as we need to encourage everyone to share what they have tried on their courses with the wider teaching community, what has worked and what hasn\u2019t.\u00a0\u00a0<\/p>\n In the end we need to see this from the student\u2019s perspective. Right now, every staff member is a person who has grown up with traditional teaching with traditional assessments. Students have experienced blended learning, and now they have AI. We perceive AI as a new thing, but students will soon perceive AI as a tool which was always there, because they will have grown up with it. It is fundamental to involve students in the conversation.\u00a0\u00a0<\/p>\n Cesare G. Ardito – @CesareGArdito , https:\/\/cesaregardito.eu<\/a> .\u00a0<\/p>\n GPT-4 was used for feedback and improvement of an early draft of this article.<\/i>\u00a0<\/p>\n \u00a0<\/p>\n [\/et_pb_text][et_pb_post_nav _builder_version=”3.0.106″ in_same_term=”off” show_prev=”on” show_next=”on” prev_text=”Previous” next_text=”Next” locked=”on” parent_locked=”off” \/][et_pb_comments _builder_version=”3.17.6″ show_count=”off” header_font_size_tablet=”51″ header_line_height_tablet=”2″ custom_button=”on” button_text_color=”#ffffff” button_use_icon=”off” locked=”on” parent_locked=”off” button_text_size__hover_enabled=”off” button_text_size__hover=”null” button_one_text_size__hover_enabled=”off” button_one_text_size__hover=”null” button_two_text_size__hover_enabled=”off” button_two_text_size__hover=”null” button_text_color__hover_enabled=”off” button_text_color__hover=”null” button_one_text_color__hover_enabled=”off” button_one_text_color__hover=”null” button_two_text_color__hover_enabled=”off” button_two_text_color__hover=”null” button_border_width__hover_enabled=”off” button_border_width__hover=”null” button_one_border_width__hover_enabled=”off” button_one_border_width__hover=”null” button_two_border_width__hover_enabled=”off” button_two_border_width__hover=”null” button_border_color__hover_enabled=”off” button_border_color__hover=”null” button_one_border_color__hover_enabled=”off” button_one_border_color__hover=”null” button_two_border_color__hover_enabled=”off” button_two_border_color__hover=”null” button_border_radius__hover_enabled=”off” button_border_radius__hover=”null” button_one_border_radius__hover_enabled=”off” button_one_border_radius__hover=”null” button_two_border_radius__hover_enabled=”off” button_two_border_radius__hover=”null” button_letter_spacing__hover_enabled=”off” button_letter_spacing__hover=”null” button_one_letter_spacing__hover_enabled=”off” button_one_letter_spacing__hover=”null” button_two_letter_spacing__hover_enabled=”off” button_two_letter_spacing__hover=”null” button_bg_color__hover_enabled=”off” button_bg_color__hover=”null” button_one_bg_color__hover_enabled=”off” button_one_bg_color__hover=”null” button_two_bg_color__hover_enabled=”off” button_two_bg_color__hover=”null” \/][et_pb_text admin_label=”Related posts” _builder_version=”3.0.106″ background_layout=”light” locked=”on” parent_locked=”off”]<\/p>\n [\/et_pb_text][\/et_pb_column_inner][\/et_pb_row_inner][\/et_pb_column][et_pb_column type=”1_3″][et_pb_sidebar _builder_version=”3.0.106″ area=”sidebar-1″ orientation=”left” show_border=”off” background_layout=”light” locked=”on” \/][\/et_pb_column][\/et_pb_section][et_pb_section bb_built=”1″ admin_label=”Connect Title” custom_padding=”0px||0px|” _builder_version=”3.0.90″ background_color=”#1d1d1d” module_class=”footer-section” prev_background_color=”#ffffff” global_module=”380″][et_pb_row admin_label=”row” global_parent=”380″ locked=”off” allow_player_pause=”off” parallax=”off” parallax_method=”on” parallax_1=”off” parallax_method_1=”off” parallax_2=”off” parallax_method_2=”off” parallax_3=”off” parallax_method_3=”off” make_fullwidth=”off” use_custom_width=”off” width_unit=”on” use_custom_gutter=”off” make_equal=”off” module_id=”footer-row” _builder_version=”3.0.89″][et_pb_column type=”1_3″][et_pb_text admin_label=”Contact Us Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n [\/et_pb_text][et_pb_text admin_label=”Contact Us” global_parent=”380″ text_text_color=”#aaaaaa” parent_locked=”off” background_layout=”light” text_orientation=”left” use_border_color=”off” border_color=”#ffffff” border_style=”solid” module_class=”contact-us” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n +44 (0) 161 306 6000<\/p>\n [\/et_pb_text][\/et_pb_column][et_pb_column type=”1_3″][et_pb_text admin_label=”Find Us Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n [\/et_pb_text][et_pb_text admin_label=”Address” global_parent=”380″ background_layout=”dark” text_orientation=”left” use_border_color=”off” border_color=”#ffffff” border_style=”solid” module_id=”address” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n The University of Manchester [\/et_pb_text][\/et_pb_column][et_pb_column type=”1_3″][et_pb_text admin_label=”Connect Title” global_parent=”380″ background_layout=”dark” text_text_color=”#ffffff” parent_locked=”off” background_position=”top_left” background_repeat=”repeat” background_size=”initial”]<\/p>\n [\/et_pb_text][et_pb_code admin_label=”Social Media” global_parent=”380″ parent_locked=”off” _builder_version=”3.17.6″]<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/ul>\n [\/et_pb_code][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":" Cesare is a Teaching Fellow soon to be lecturer in the Department of Mathematics and a Senior Fellow of the Higher Education Academy.\u00a0\u00a0Here, he outlines some common features of Large Language Model (LLM)-created writing with Esme Davies, Higher Education Intern in the ITL, and discusses the implications of this technology for how we might assess […]<\/p>\n","protected":false},"author":6724,"featured_media":2756,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","_exactmetrics_skip_tracking":false,"footnotes":""},"categories":[48,43,91],"tags":[28,45],"class_list":["post-2749","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-academic-integrity","category-ai","category-assessment-design","tag-academic-integrity","tag-ai-pedagogy"],"_links":{"self":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2749","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/users\/6724"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/comments?post=2749"}],"version-history":[{"count":7,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2749\/revisions"}],"predecessor-version":[{"id":3317,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/posts\/2749\/revisions\/3317"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/media\/2756"}],"wp:attachment":[{"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/media?parent=2749"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/categories?post=2749"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.manchester.ac.uk\/itl\/wp-json\/wp\/v2\/tags?post=2749"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}There has been a lot of talk about the tell-tale signs of GPT-written work. Could you give us an overview of these features, and their reliability in detecting GPT-written content?\u00a0\u00a0<\/h5>\n
\n
How do AI detection tools fit into this discussion?\u00a0<\/h5>\n
How would you address concerns from colleagues over an increase in academic malpractice with GPT-written content?\u00a0\u00a0<\/h5>\n
\n
What else can colleagues do in the short and longer term to prevent submission of GPT-written work?\u00a0<\/h5>\n
What kind of support do you think staff would need in creating more authentic assessment to reduce GPT-written work in terms of academic malpractice?\u00a0<\/h5>\n
Further reading\/resources:\u00a0<\/h5>\n
\n
\n
Related<\/h1>\n
Related posts:<\/h3>
<\/div><\/div><\/div>From AI Awareness to AI Fluency: A Practical Copilot Toolkit for University Students<\/h2><\/div><\/div><\/a>
<\/div>AI in Teaching and Learning: Learning together<\/h2><\/div><\/div><\/a>
<\/div>How Teaching Students to Study Effectively can Reduce Contract Cheating.<\/h2><\/div><\/div><\/a><\/div>
<\/div><\/div><\/div>\nContact Us<\/h4>\n
Find Us<\/h4>\n
Oxford Rd
Manchester
M13 9PL
UK<\/a><\/p>\nConnect With Us<\/h4>\n
<\/p>\n
![\"Facebook](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-facebook-50.png\"){kind=icon}
<\/a><\/li>\n![\"Twitter](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-twitter-50.png\"){kind=icon}
<\/a><\/li>\n![\"YouTube](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-youtube-50.png\"){kind=icon}
<\/a><\/li>\n![\"Instagram](\"\/\/assets.manchester.ac.uk\/phase1\/templates\/images\/icon-instagram-50.svg\"){kind=icon}
<\/a><\/li>\n