VisionSpeak Object Detection and Narration System

VisionSpeak Object Detection and Narration System

International Journal of Scientific Research in Engineering and Management · 2026

HIGHLIGHT

Using a laptop webcam and the pre trained YOLOv8s model, our system achieves 22 FPS on consumer hardware (Intel Core i3) with 81% average precision across seven common indoor objects.

RELATED

ABSTRACT

Abstract - We present VisionSpeak, a web-based object detection system that narrates visual scenes through audio feedback. Using a laptop webcam and the pre trained YOLOv8s model, our system achieves 22 FPS on consumer hardware (Intel Core i3) with 81% average precision across seven common indoor objects. The core contribution lies in the narration control pipeline: temporal stability filtering reduces unnecessary speech by 73% (from 23.7 to 6.4 narrations per minute) while maintaining 95% detection recall. Our system runs entirely offline using pyttsx3 for text-to-speech, ensuring privacy and consistent operation without cloud dependencies. The graphical interface eliminates command-line complexity, allowing non-technical users to adjust confidence thresholds (0.1–0.9) and monitor live detections through an annotated video feed. We evaluated VisionSpeak through 500 manually annotated test frames across five indoor environments and a usability study with 10 participants completing standardized tasks. Results show 87% task success rate and a System Usability Scale score of 82/100. Performance degrades predictably under challenging conditions: dim lighting reduces precision by 15%, while objects smaller than 5% of frame area show 35% lower detection rates. VisionSpeak serves as an educational demonstration tool for computer vision concepts and a prototype platform for studying narration strategies in vision-based systems. The modular architecture supports future extensions including OCR integration, depth estimation, and deployment on wearable devices. Keywords: Object detection, YOLOv8, assistive technology, real-time narration, computer vision, accessibility, offline text-to-speech, human–computer interaction, visual impairment, frontend-backend integration.