Specificity

Assessments which ask students to discuss or reflect on a very specific case, scenario or critical incident are less vulnerable to AI. The tools can do these tasks, but they are currently producing quite generic work.

• Asking for very specific cases, especially things which have been studied in class or from students' personal experience will interrupt the patterns that language models like ChatGPT have been trained on as that information is unlikely to be well documented. Even as tools evolve and web connection is integrated this is a good way of moving from generalised to specific assessments.
• You could also ask students to produce an annotated bibliography to accompany their assignment and state why they chose certain references.
• Asking students to choose examples (or you choose examples) from the Global South is a good idea as the Large Language Models are largely trained on content from content that already dominates the web. This is also a goal of decolonisation and providing diverse examples for students. This may be more appropriate for some disciplines than others.

Focus on the process as part of the product

Writing or producing an output for assessment is a process and this should be emphasised to students. One method of doing this is setting tasks for completion throughout the course (these can be marked and/or given feedback or not) which students have to submit as part of their finished product, e.g. outlines, drafts, summaries, syntheses of research.

There are more ideas for how to do this in the 'authentic assessment module' section. It is important to note that where students see value in an assessment they are more likely to engage fully and meaningfully and are less likely to look for shortcuts or to drift into questionable approaches.

Note: When making changes to exam question types and skills being assessed, this should be clearly communicated to students and formative/mock practice given.

Test your questions with Generative AI yourself

This is time consuming but the best way to assess the vulnerability of your exam is to run it through an AI tool. This is useful for more quantitative subjects as well. It can allow you to redesign the assessment once you know what sort of answer the tools are producing.

Using a range of ways of presenting information where possible

Most open access (and free) AI tools cannot currently read images very well, though Bing Chat has recently introduced this functionality and it is worth familiarising yourself with its capabilities. Whether in MCQs or in short answers where students have to interpret data, you should use a range of ways of presenting information, such as images, data files etc. that students have to analyse/interpret.

An important caveat here is considerations of accessibility in examinations. If images cannot be read by screen readers, this means that your questions will have to be adjusted for visually impaired students. This has implications for universal design for learning.

Avoid factual recall or highly weighted factual recall questions

Avoid questions which only ask for definitions, explanations or calculations. Generative AI tools are very good at coding and computational activities.

Questions which test higher order skills (creating, evaluating) are less vulnerable to Generative AI.

For questions which do require factual recall to meet learning outcomes, weight these as low as possible or integrate the knowledge recall as part of a higher order skill.

Use multi-step problems

Generative AI tools are trained to predict the next word or string of words and cannot make conceptual leaps. You can interrupt this to some extent by designing exam questions which build on each other or are in parts, for example multi step problems. This also helps to assess higher order skills such as application to a range of scenarios.

You do have to consider the fairness of this however, when designing your exams and how this relates to the learning outcomes. If students cannot answer the first question, they will fail subsequent questions. This will depend on whether we are testing discrete items of knowledge.

In-person exams

This is a solution for exams where factual knowledge is required for a PSRB (professional body) or to meet foundational learning outcomes. Paper or digital exams can be invigilated in a secure environment such as computer lab or exam hall. It is already being used as a solution to prevent collusion in online examinations post-pandemic and is a requirement for some accrediting bodies.

However, this is not necessarily a solution for longer term assessment for learning and educational goals. Exams have been shown to encourage surface learning, create anxiety and are open to claims of being less inclusive than other forms of assessment (Waterfield and West, 2006; Tai et al, 2023). As King’s guidance on Generative AI states, it is important that AI literacy is built into our education and therefore closed book invigilated exams are not a response in the long term to challenges of Large Language Models.

Formative preparation

Ensure that students have opportunities to practice the skills needed for the coursework assignment. This does not have to be formalised written drafts, although where possible, it’s good to have students write something. This can also be used to compare the finished assignment as a possible detection measure.

Formative tasks do not have to add significantly to marking workload, provided that students are doing more in seminars than just discussion work. The extra workload is in planning lessons that allow for more experiential learning and collaborative work (including writing and practising skills for the assessment).

Feedback can be individual but it can also be through:

• peers
• cohort
• informal chats
• online forums
• online collaboration tools such as Padlet

Read about more ideas. 

Allowing use in coursework based on the four levels of acceptance in the university's guidance:

1. Minimal

Includes routine and established use of tools such as auto transcription, spell checkers, grammar check.

2. Limited/selective

Use for clearly delineated tasks as appropriate/allowed/ recommended. May include:

• Ideation/brainstorming
• Concept checking/explanation
• Suggesting structures
• Suggesting titles
• Summarising own or other’s key ideas
• Illustrative image or other media generation

3. Open

No specific restrictions but with requirement to track key stages/tools utilised. Possible uses may include:

• Socratic chat bot/tutor
• Summarisation, simplification, synthesis, translation
• Generate illustrative media content
• Support production of multimedia artefacts

4. Embedded

AI use is a feature of the assessment itself. Here the use of generative AI is a focal aspect of the assessment. This may include:

• Using named tools for a specific outcome
• Output comparison and critique
• Develop or error check code

Programme teams should decide where the use of generative AI tools will and will not be permitted or appropriate for specific assessments and the level of use that is deemed acceptable. This should be clearly communicated to students at the start of a module. It is likely that there may be differences between modules.

Using acknowledgement statements of generative AI use in cover sheets

There are two options from the university's guidance which can be added to a coursework cover sheet for students to show the level of their use. This should be in conjunction with the decision made by the module/programme team around levels of acceptable use. This can be added to a cover sheet/KEATS declaration for plagiarism.

Please note, at KCL we will seek to define acceptable/ fair use rather than trying to specify what is prohibited. An assessment is designed to both develop and evaluate your progress so it is never appropriate to submit chunks of text or other media that are duplicated from another source without clear acknowledgement. Because tools like ChatGPT are generating text on a predication model they are not quotable sources and are not appropriate places to focus research.

King’s College London, unlike some other universities, does not require students to reference generative AI as an authoritative source in the reference list for much the same reason you would not be expected to cite a search engine, a student essay website or be over-dependent on synoptic, secondary source material. However, as we learn more about the capabilities and limitations of these tools and as we work together to evolve our own critical AI literacies, we do expect you to be explicit in acknowledging your use of generative AI tools such as Large Language Models like Microsoft Bing Chat (available via your KCL account in Microsoft Edge), Google Bard or ChatGPT or any other media generated through similar tools.

You should select one of the following two statements, complete it and append it to your references or somewhere prominent with your submission. Please note that so long as acknowledged use falls within the scope of appropriate use as defined in the assessment brief/ guidance then this will not have any direct impact on the grades awarded.

Declarations in (2) below are an important reflective step and should be considered necessary.

1. I declare that no part of this submission has been generated by AI software. These are my own words.

Note. Using software for English grammar and spell checking is consistent with Statement 1.

[or]

2. I declare that parts of this submission has contributions from AI software and that it aligns with acceptable use as specified as part of the assignment brief/ guidance and is consistent with good academic practice. The content can still be considered as my own words. I understand that as long as my use falls within the scope of appropriate use as defined in the assessment brief/ guidance then this declaration will not have any direct impact on the grades awarded.

I acknowledge use of software to [include as appropriate]:

• (i) Generate ideas or structure suggestions, for assistance with understanding core concepts, or other substantial foundational and preparatory activity.

[insert AI tool(s) and links and/or how used]

• (ii) Write, rewrite, rephrase and/or paraphrase part of this essay.

[insert AI tool(s) and links]

• (iii) Generate some other aspect of the submitted assessment.

[insert AI tool(s) and links] / Include brief details]

There should always be an option for students who do not wish to use it to explain that they have not and why.