King's College London

qVault.ai - AI-Powered Single Best Answer (SBA) Question Generator & Enhanced Student Progress Tracker (Preclinical Medical Students)

The widely recognized Single Best Answer (SBA) Assessment format is a fundamental element in medical education, testing student’s lateral thinking and providing objective grading. A shortage of accessible resources hinders learning progress; existing materials lack essential services like progress tracking and customization.

This project aims to harness the potential of AI to provide an efficient and accessible solution. By introducing an SBA question bank, we can generate an expansive array of medical questions based on your own teaching resources and personal notes. Furthermore,  AI-powered Enhanced Student Progress Tracking will offer actionable insights and tailored advice through deep understanding of question content.

Mechanism 

1. User Content Upload: Authenticated users can upload teaching resources to our web application, integrated with fine-tuned ChatGPT-3.5. 

1. AI Keyword Identification & Topic Generation: Utilizing fine-tuned pre-set prompts, the AI will analyse content, categorizing keywords into a list of key topics, which users can select for question generation.  

1. AI Question Generation: The AI will generate formatted SBA questions and presented as a deck. 

1. User Learning & Advanced AI Feedback:  

• Users complete the questions on the web application.  
• Post-completion, users receive scores and explanations. Users can provide feedback on individual questions, classifying them as easy, challenging, erroneous, or providing typed comments. 

• Users can also review, filter, and select questions, save and create personalized "decks" in their accounts.  
• Users can view their performance metrics (e.g. Point Biserial, Correlation Data, Facility, Quintiles) in a performance dashboard while seeking AI suggestions for areas of improvement.  

1. Database Integration: The generated question decks will be incorporated into the database. Each question will carry tags (e.g. feedback, difficulty, validity, type) for future analysis and reuse. A unique educational profile containing data points will be generated for each user, including basic information, interactions, and performance metrics.  

AI QB Learning & Improvement 

• Topic selection and question feedback will be recorded by analysing user interactions and ratings. 
• Flagged questions will be replaced and analysed for shortcomings. 
• All feedback and performance metrics will be consolidated into a database. The team will periodically review data for fine-tuning, maximising accuracy and relevance. 
• All previously generated questions will be reusable for other users.  

AI Enhanced Student Progress Tracking 

• AI-powered analysis on the user's educational profile provides a performance dashboard for viewing, with areas of improvement and the option to generate additional questions.   
• User reports can be downloaded for self-monitoring and sharing.   

 In conclusion this project establishes a sustainable AI-automated SBA question bank and precise student tracking system, enhancing learning assessments and educational support.