This guide is recommended reading for all staff with teaching and assessment responsibilities and is the principal point of reference for heads of department, programme leaders and module leaders. At the meso level, the guide dives deeper into the implications of AI for specific programmes or departments. Starting with apparent employability consequences of generative AI, it connects them to programme and assessment design and some of the practical ways we can address issues arising. We support a balanced approach to AI integration, promoting a variety of applications while also emphasising appropriate use and acknowledgement of AI-generated content.

At present, we advise against integrating generative AI into most summative assessments. There may of course be exceptions to this where the subject of study is generative AI or the outputs are an essential component of the assessment design. One primary concern is that generative AI tools, such as ChatGPT, Google Gemini, Microsoft Copilot and Claude AI have not been developed by higher education providers, and we do not have control over or oversight of the biases it might produce or the ways in which data are stored and subsequently used. Microsoft Bing Copilot offers greater protection and assurances when used with a KCL account. As we begin to explore ways in which the foundational data can be determined and/ or limited through, for example, customised bots we will evolve a better and deeper understanding of the ways in which we can utilise these technologies for teaching, learning, assessment and feedback. See here for summaries of the 2023-24 College Teaching Fund research projects, many of which explore these possibilities.

Teaching staff members may provide tips and demonstrate the capabilities of generative AI in formative activities, using Microsoft Copliot where possible and making clear the limitations and/or costs of other tools used. Colleagues should remind students of principles of academic integrity whilst engaging in these activities i.e. using generative AI to support learning, not cut corners or produce a final product. Where possible, participation in activities using generative AI should be optional to respect students who do not wish to use these technologies. It's essential to remind students that achieving a first-class grade is entirely possible without any reliance on generative AI.

If one of the functions of a university programme is to prepare students for the world of work, then it is clearly a duty for us to adapt our programmes to acknowledge changes in that world.

A recent analysis of AI automation exposure shows that law, engineering and business are amongst the most highly at risk industries, with as many as 300 million full-time jobs at risk globally. Given that universities typically struggle to adapt swiftly to change, this likely fast-changing employment landscape connotes profound impacts on many of our programmes; but these are challenges where we might look at the successes of King’s pivot online at the start of the COVID-19 pandemic for inspiration. It adds considerable value for current and future-alert industry intelligence and suggests that we need to be prudent when designing programmes that there is room built in for flexibility with pedagogic and assessment design as well as in terms of what is prioritised in what we teach. Whilst we may instinctively look to detection of generative AI amongst our immediate responses, we should first take opportunities to consider the types of assessments on our programmes and modules where there are vulnerabilities and ways in which we might mitigate impacts of generative AI.

The implications for assessment are significant - all written work and coding are vulnerable. A recent report shows that GPT4 ‘exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers’ (OpenAI 2023).

These tools are only going to increase in effectiveness and prevalence and trying to ignore them, outrun them or prohibit their use are not viable approaches. We should also note that one of the perhaps more obvious ways to challenge this - large scale use of in person invigilated exams - is not considered sustainable nor is it a valid and authentic mechanism for the majority of assessments. Exams remain, of course, a key part of the assessment diet on many programmes but should not be seen as the likely panacea. King’s is committed to maintaining its wider goal to ensure assessment validity, reliability, authenticity, fairness and variety wherever possible.

Assessment design has three key purposes:

1. To move student learning forward - ‘assessment for learning’
2. To evaluate learning/to judge whether and to what extent students have reached or exceeded thresholds of understanding or competence
3. To minimise academic integrity risks and uphold integrity of programmes and its graduates

These purposes are no more true now than they were a year, five years or decades ago. Generative AI may prompt us to consider the effectiveness of our assessments and the authenticity of them, but where students could (especially with knowledge-focussed take-home assessments) produce work using generative AI, the issue is no greater philosophically than students who might have paid another student to do it or employed an essay mill. The key difference is ease of access and a blurring of the lines between what is self-evidently cheating and what may feel more like legitimate use of technological assistance.

Consider this scenario:

A confident and very able student decides that much of the source material they will use for an assignment has been published in their first language which is not English. They read, process and work in their first language, fully responding to the set task. Running short of time, they put the assignment through a translation tool and, bar a few tweaks and edits they are reasonably happy with the output. However, one section doesn’t sound right so they use a combination of ‘Grammarly’ and ‘ChatGPT’ to re-write it and phrase it in a way they are comfortable with. They submit this work.

• To what extent is this a legitimate way to study and produce the assessment?
• Is this likely a breach of academic integrity standards?

There is of course no one correct solution to these questions. Context is fundamental. However, using such vignettes can help staff determine and agree on what is appropriate use and students develop a more critical literacy.

As generative AI tools become increasingly embedded into commonly used suites of tools (see ‘Help me write’ in Google Docs and Copilot for Office 365) so this blurring will be both harder to detect and, in many ways, much harder to track or even be conscious of.

The best way to evaluate current capabilities of Generative AI is to try some of the tools for yourself. It helps to do so in team pairs or clusters to discuss the way prompts are written, the outputs and the implications. In this way, the vulnerabilities to corner cutting, questionable practices or clear breaches of academic integrity become much more evident.

Fact-based, take-away assessments are likely to be the most vulnerable to use of generative AI tools. Whilst an instinctive response may be to look towards in-person invigilated exams or proctored online exams there are many issues with these such as lack of authenticity and validity, a tendency to focus on knowledge recall and the impacts of exam pressure on performance mean they will not be a panacea in either the short or long-terms, though they may provide one option for essential fact-based assessments.

It is worth reviewing knowledge-based assessments and consider ways in which they might be adjusted to:

• Include reflective elements, or higher order thinking skills (creating, synthesising, evaluating)
• Find ways to connect or thread questions so that they follow from one another and interconnect
• Require explicit connections to current events or specific articles or on course events (e.g. A specific lecture or lab-based experiment) - this also encourages essential cognitive engagement.

Given pre-existing worries about essay mills, the vulnerability of essays or other long-form writing is not new, though Generative AI does blur the integrity lines. It is worth considering the nature of questions in essays and ways in which the above advice could inform essay tasks. However, it is also worth considering more fundamental changes to assessments and thinking about the extent to which they are authentic, how much they focus on process (as opposed to the final ‘essay product’) as well as where and how the assessments happen.

Formal processes

Minor changes can be made to assessment questions for essays and examinations with the following caveats (see suggestions on how to tweak summative assessments).

• a. Significant changes during any academic year are not recommended where students will have been preparing for their summative assessments through formative work and the use of past papers. Where substantive changes are made to question style or rubrics from one year to the next, departments are ordinarily required to provide students with sample questions, papers or mock assessments in the new format in good time to allow for adequate preparation.
• b. Where possible, colleagues are encouraged to conduct internal assessment scrutiny processes. In the case of minor adjustments, such as changing a single question, it is not necessary to seek approval from external examiners. We should, however, advise external examiners of changes we make as these may otherwise be picked up in annual reports.
• c. Where minor adjustments are made, module leaders/internal examiners should follow their standard departmental process by seeking any necessary approval, for example via Department Teaching Committee.

Practical ways to modify assessments

Approaches to Assessment in the age of generative AI, designed by King’s Academy in collaboration with King’s academic colleagues, gives guidance for staff in assessment design from three main perspectives:

• Making tweaks in the short term to prevent AI vulnerabilities
• Rethinking assessment strategies, approaches and tasks to include motivating, authentic and less AI-vulnerable assessments
• Using Generative AI in summative assessment tasks, formative activities and feedback

See detailed guidance on considerations and approaches to assessment in the age of AI. 

It is recommended that Heads of Department discuss this guidance with programme leaders and look for opportunities to streamline processes according to departmental practices, and support programme leaders in their coordination roles. Programme Leaders are advised to do the following:

1. Discuss with module leaders the assessment diet on programmes, consider the extent to which they may be vulnerable and ways to adapt or plan for change.
2. Discuss with module leaders and decide what constitutes appropriate use. Depending on the nature of the programme this may be at programme level but more likely it will be at module level and, in some circumstances, may differ across assessments within a module.
3. Communicate decisions at programme level to all students and ensure opportunities for discussion and clarification are built in as early as possible, locating the discussion within the broader consideration of academic integrity.
4. Share guidance on appropriate use, acknowledgement and processes in case of breach of academic integrity policy.
5. Oversee (or delegate responsibility for) management of suspected breaches of King’s academic integrity policy.

As stated earlier, it would be impossible at the institutional level to fix a simple regulatory framework beyond the core principles of presenting text or generated text as one’s own (broadly a part of the existing plagiarism policy) and ‘unfair advantage’. However, we set out below four possible starting points for departments and programme leaders. It will be necessary at programme level at least and possibly at module level to discuss and determine which level is appropriate for any given assessment and the specifics therein:

1. Minimal

Includes routine and established use of tools such as auto-transcription, spell checkers and grammar check.

2. Limited/selective

Use for clearly delineated tasks as appropriate/allowed/recommended.

May include:

• Ideation/brainstorming
• Concept checking/explanation
• Suggesting structures
• Suggesting titles
• Summarising own or other’s key ideas
• Illustrative image or other media generation

3. Open

No specific restrictions but with requirement to track key stages/tools utilised. Possible uses may include:

• Socratic chat bot/tutor
• Summarisation, simplification, synthesis, translation
• Generate illustrative media content
• Support production of multimedia artefacts

4. Embedded

AI use is a feature of the assessment itself. Here the use of generative AI is a focal aspect of the assessment. This may include:

• Using named tools for a specific outcome
• Output comparison and critique
• Develop or error check code

Nb.

In no circumstance is it appropriate to use generated content verbatim without clear indication and acknowledgement or to effectively outsource the writing task, as much as it is inappropriate to get someone else to write something and claim it as your own.

Moreover, it is important to reflect on the outputs of generative AI as there is a tendency for it to produce information that is biased or false. Students should also reflect on the process they used to generate responses through prompting or how they kept track of key stages or tools used, and/or the effectiveness of using these tools within their learning process.

As recommended for all assessment design, staff should consider the implications of changes or of specific requirements or limitations in terms of potential impacts on students with disabilities and any student with a Personal Assessment Arrangement (PAA).

The core principle: “it is university policy that this work should be expressed in the student’s own words and incorporate their own ideas and judgments” applies.

Where possible, as much foundational work should be dedicated to pre-emptive and preventative measures as possible. Where there is suspicion or evidence (please note section dealing with limitations to and guidance on detection) of inappropriate use of generative AI, colleagues should follow the appropriate university-level and agreed faculty and/or departmental procedure. It is fundamental that accusatory language is avoided and that academic misconduct discussions follow existing procedures. It is worth noting here that many existing detectors have been shown to manifest false positives and that these have a bias towards erroneously flagging students who are non-native speakers of English.

It may be deemed appropriate to call an investigatory meeting. The meeting is to provide an opportunity for the student to meet academics to discuss the assignment concerned and any concerns about the authorship of the work. This meeting is aimed at providing the parties with an opportunity to resolve any misunderstandings at an early stage to avoid a case going to a Misconduct Committee unnecessarily.

There is no hard and fast rule on what should be asked in the investigatory meeting as it will depend on the specific case and the nature of the assessment. The panel will ask questions to help them determine whether the authorship of the work has been compromised, and a key element is to ask the student to show understanding of the work and the process of producing it.

This suggested approach for investigating AI cases is not new. It is almost identical to the existing approach in investigating unfair academic practice/academic misconduct of third-party involvement in assignments, such as the use of contract cheating services and ghostwriters, which has been running smoothly generally with very little complaint of unfairness.

The academics will decide on the probability of whether there is a case to answer and will only refer the case to a Misconduct Committee if they think that it is more likely than not that generative AI was used inappropriately in the assessment. Students should be assured that the faculty panel will be making academic judgment holistically based on all information and factors, they will not rely on a single indicator and students should be reminded that the Misconduct Committee will be independent of the faculty, and will consider the case fully, with the student given a full opportunity to make representations and defend any charges.

If the case is referred to a Misconduct Committee, the Faculty would prepare a detailed referral form listing all the grounds of suspicion. Such a document will be made available to the student not less than 10 working days before the hearing and the student may submit written and oral representations to rebut the charge. If the student is concerned about the fairness of the faculty investigatory meeting or disagrees with the judgment of the faculty panel, these can be raised to the independent panel at the Misconduct Committee.