AI Based Doctor-Patient Diarization System Using Speech Recognition and NLP with SOAP Notes Generation

International Scientific Journal of Engineering and Management  ·  2026

Prof.Shruti Hatwar Prof.Shruti Hatwar; Atharv Borhade Atharv Borhade; Yash Dhole Yash Dhole; Vishal Wagh Vishal Wagh; Sanskar Darekar Sanskar Darekar;

HIGHLIGHT

This paper proposes an AI-based Automated Clinical Documentation System that converts doctor-patient conversations into structured medical records.

RELATED

Papers      Patents     Grants     Venues     Experts    Cite    Share    Save

ABSTRACT

Clinical documentation is an essential yet time-consuming task in modern healthcare systems. Doctors often spend a significant amount of time manually recording patient consultations, which increases administrative burden and reduces patient interaction time. This paper proposes an AI-based Automated Clinical Documentation System that converts doctor-patient conversations into structured medical records. The proposed system integrates Whisper Automatic Speech Recognition (ASR) for speech-to-text conversion, pyannote.audio for speaker diarization, spaCy for medical entity extraction, and Large Language Models (LLMs) for SOAP note generation and consultation summarization. The system accurately identifies speakers, transcribes medical conversations, extracts key clinical information such as symptoms, diagnosis, medications, and treatment plans, and automatically generates structured SOAP notes. MongoDB is used for secure storage and retrieval of consultation records. The generated clinical documentation can be accessed through dedicated doctor and patient dashboards, improving healthcare workflow efficiency and reducing manual documentation effort. Experimental results demonstrate the effectiveness of the proposed approach in automating clinical documentation while maintaining accuracy and scalability. The proposed system contributes towards intelligent healthcare management by combining speech processing, natural language processing, and generative AI technologies into a unified platform. Key Words: Doctor Diarization System, Artificial Intelligence, Natural Language Processing (NLP), Speech-to-Text, Whisper, pyannote.audio, Speaker Diarization, SOAP Note Generation, Medical Conversation Analysis, MongoDB, Healthcare Management System, spaCy, Large Language Models (LLMs), Clinical Documentation, Patient Management System, Role-Based Authentication, Digital Healthcare, Audio Processing.

 Paraphrase     Questionfy     Wikify     Simplify     Summarize