Harvard researchers develop AI app to aid blind navigation

Researchers at the John A. Paulson School of Engineering and Applied Sciences at Harvard University have developed an artificial intelligence-based application to help blind and visually impaired individuals navigate safely and avoid obstacles, utilizing tools available on smartphones such as the camera, GPS, and motion sensors.

The application, named "Mobilio," leverages smartphone technologies, including the LiDAR sensor when available, to guide users along pedestrian routes, maintain direction, and avoid obstacles. Its development followed a survey of the needs of more than 100 blind and visually impaired people. Initial tests involving 14 participants showed promising results: when used alongside a white cane, the app enabled users to complete outdoor routes about 13% faster compared to using Google Maps with a cane, and reduced collisions with obstacles on indoor routes by around 41%.

Mobilio continuously analyzes the surrounding environment through the phone's camera, while simultaneously using GPS and motion sensors to determine the user's location and direction. It then provides audio instructions that guide movement and adapt to the user's responses.

The app's computer vision model is trained from a pedestrian's perspective, enabling it to recognize elements such as sidewalks, crosswalks, and pedestrian pathways. This allows it to offer continuous navigation assistance, rather than simply detecting nearby obstacles.