YouTube chatbot test in VR. The basic Windows Text-to-Speech voice is a bit robotic and facial blend-shape animation is also a little primitive, but overall a good proof of concept.
Patient #1 3D Chatbot
Try the resource for yourself. Download and unzip package, run chat-bot-2a.exe (Currently PC only)
Download for PCPatient #1 Web Chatbot
Try the resource for yourself. Speech currently only works on Chrome or Firefox (PC and mobile). Text input works on all other platforms and browsers.
Chat with the patientProject Description
The benefits of Virtual Patient simulations for Medical Education have long been recognised. With recent advances in AI, chatbots have become more widespread on social networking platforms and almost ubiquitous in all sorts of online customer support. We are currently investigating their use in the Stage 2 MBBS medical curriculum.
A Chatbot is a conversational agent, a computer program designed to hold a conversation with human users. Due to their programmatic logic, for them conversations are very much like a game of tennis: talk, reply, talk, reply. This makes them ideal in designing simulations of Patient-Doctor interactions, testing the questioning style of young doctors is particularly useful in teaching clinical reasoning, where the emphasis is on asking the right questions and processing the available information.
This system uses a server-based chatbot which can be accessed either from a web browser (including mobile devices) or, for an even more immersive and engaging interaction, using 3D NPC avatars developed in Unity. Input and output (talking to the avatar and getting responses back) is possible using both text as well as voice recognition and text-to-speech technologies. This is particulalry immersive in VR where interaction is entirely hands-free and fully simulates a face-to-face conversation.
Software used: ChatScript / Dreamweaver / Mixamo / Unity.
Project Details
- Client: Faculty of Life Sciences & Medicine.
- Status: Ongoing
- Themes: interactivity, Simulation, AI, VR, Virtual Patients
- Team: Jerome Di Pietro