Abstract: Falls are a major risk for independently living elderly people, often resulting in hospitalization or loss of independence if help is delayed. Reliable, unobtrusive fall detection is important – but existing systems often rely on a single modality, require manual activation, or constantly surveil the user, limiting user acceptance. We present Smart Companion, an audio-visual interactive lying person detection system embedded in a robot vacuum cleaner, augmented with a voice assistant. It operates as a daily-use appliance with the added functionality of lying person detection and fall hazard notification. The system utilizes multi-view image analysis with a convolutional neural network for lying people detection and initiates verbal interaction to further assess the situation. With this combination, the system reliably triggered emergency services and caused only one false alarm during six months of a prototype field study in the private apartments of elderly users. The dialog and system behavior were developed through a user-centered design process, contributing to high user acceptance.

Keywords: Fall detection, Lying person detection, Fall hazard notification, Human-robot interaction, Convolutional neural network, Voice assistant, Multimodal sensing.

__________________________________________________________________________________________