How Scalefocus Changes the Lives of the Visually Impaired
Something extraordinary happened two months ago at the Stevie® Awards, which are among the world’s most prestigious business awards.
Scalefocus not only brought home a Gold Stevie® award but also became the first-ever Bulgarian company to win the exclusive Grand Stevie®. To put things into perspective, the Grand Stevie® is only awarded to a handful of Gold Stevie® winners.
The highly acclaimed product that overcame the fierce competition in the Healthcare Technology Solution category is called SoundVision. It is an innovative solution that combines a hardware device with a mobile app and self-learning ML service for object recognition that helps visually impaired people lead a more fulfilling and independent way of life.
The project is another achievement for Scalefocus, which has showcased our expertise in the field of healthcare, and the meeting with Nikolay Shekerov (Portfolio Manager) and Boyan Moshelov (Project Manager) gave us a chance to learn everything we need to know about SoundVision.
“What makes the SoundVision project fascinating is that it started as an internal discussion of innovation ideas during a hackathon,” reveals Nicky. “This was actually the one that won the hackathon. The company then committed to seeking funding for the concept, which was inspired by the way whales use echolocation to navigate underwater. Obviously, the first presentation of the idea just had to involve sperm whales.”
However, it didn’t all start swimmingly well, and it took about a year before Scalefocus found funding and partners. The renowned Norwegian Computing Center (where object-oriented programming was invented, by the way) was the first to join in and was shortly followed by Norway’s largest store that sells products for the visually impaired. The funding came from the Norway Grants innovation program, and the partner in Bulgaria is the Synergia Foundation which also specializes in visually impaired people.
“The Norwegian Computing Center helped us with the ML models we use for object recognition,” Nicky continues. “The connection to the mobile application we developed enables the device to recognize objects, products, currencies, and even read text that the visually impaired user hears.”
So, what is the solution like?
The hardware device is about the size of a small torch. It provides basic functionalities such as distance measurement and color recognition. The brightness sensor is also quite important – an utterly blind person would not know whether the lights are on, which can inflate the electricity bill if they stay on for months. You can also use it as a compass – when visually impaired people follow Google Maps instructions and are told to turn westward, the information is virtually useless. The device helps them find directions so they can make use of the instructions.
“We realized that we needed to be independent of the smart device models that the app was running on because all devices work differently depending on the phone model,” Nicky continues. “This validated the need for a separate hardware device because you would know what sensors you’re working with and how to process the data to provide the user with correct information.”
“Phones use a compass to tell directions, but the thing with various phone brands and models is that their parts are different,” Boyan added. “This is a problem we ran into during development, and it’s valid for Android but not iOS, where the parts are universal. Also, in the work process, we found that the lidar is the most suitable and user-friendly solution for measuring distance. Of course, we made sure the power is too low to interfere with other devices.”
The mobile app is the second component of the solution that provides even more functionalities. All functionalities this ML engine provides for recognizing objects, currencies, etc., are available through the mobile application. Navigation in buildings is another key functionality. The deviations that GPS gives in such an environment require more experimentation and the addition of guidance functionality based on Bluetooth beacons.
“Our colleagues implemented the Bluetooth beacons technology that uses triangulation for guidance,” Boyan clarifies. “The whole office got equipped with Bluetooth beacons, and we thought it would be easy – just write the code, and triangulation will guide you from there. However, it turned out that the building’s architecture was critical. Windows and doors blocked the signal, which was among the challenges we overcame to achieve optimal triangulation accuracy.”
The challenges did not stop at the navigation functionality, as Nikolay Prodanov, iOS Technology Team Lead, explains: “With iOS, it is relatively easy to make the application usable for blind people. The challenge was to make the menus and functionalities speak when the app was locked. We achieved this with an additional voice assistant that we synchronized with the built-in API.
The communication and continuous command exchange between the Bluetooth device and mobile phone were also problematic. So, we chose the reactive approach using Combine Future Publishers to manage the command flow. Another challenge was to control the app via the Bluetooth device’s joystick on a locked cell phone. We had to implement a Background mode with a State machine to replicate the application’s navigation and navigate within it.”
“To recognize the brightness when the application is disconnected, we used the phone’s rear camera and the CameraX API,” says Boyan Lozanovski, Android Software Engineer. “We disabled all automatic camera settings and manually set the sensor sensitivity and exposure time values. Our goal was to maximize the brightness difference in the images captured in the room and outdoors under different lighting conditions. Then we extracted the brightness value into sound by playing around with the sample rate and frequency of the background sound. Apart from sound, we also used a custom view to display the same information in a simple visual format. To smoothen the transitions, those changes are animated dynamically, and the number of analyzed frames per second is calculated in real-time. “
SoundVision also boasts Software Development Kits (SDKs) that allow integration with other external solutions. SDKs make the ecosystem open and enable the connection with other off-the-shelf applications and external ML engines when proceeding to mass production at a later stage. This allows external developers who would like to contribute to the variety of services to do so fairly quickly.
“The idea came from within, from our people. This makes the project different,” sums up Nicky. “We as a company have supported this idea, found funding, and given people a chance to work on something interesting and different that originates from their ideas.” We have really learned a lot, we have discovered a new world.
We can now tell the difference between people who were born blind and people who were born with sight but then lost it. In fact, the majority of our target group is visually impaired, they are not entirely blind, so they only need partial assistance. It’s very important that we were able to work on our own ideas of something that could have a tremendous social impact and significance, something important, a noble cause.”
Researching the competition shows that as a complete solution – a combination of a device with a mobile application, SoundVision is unparalleled in the number and range of functionalities and capabilities.
“All of us at Scalefocus would love to see SoundVision on the market. That is why we need to find a partner to take on mass production and distribution. We have developed an affordable solution that, unlike similar solutions, can be reasonably priced for the end user. The target value is several tens of euros, and the price will drop drastically as production increases. The cost will be much, much lower with thousands of units produced, and it will also allow us to use even better sensors and other components,” Nicky concludes.
Scalefocus’ Norwegian partners have already stated that SoundVision will be singled out as a good example to the European Commission and one of the most significant projects funded by Norway Grants.
Successful implementation does not happen with every project, particularly R&D projects and especially those involving universities and science people who are much less business savvy and commercially driven.
Deriving from a decade of solid business and technology expertise, we are committed to setting goals high and meeting them to deliver world-class end-to-end products and services. The international recognition brought by prestigious awards such as the Grand Stevie® provides further evidence, and we are certain that SoundVision is far from the final installment in the growing list of Scalefocus innovations that improve people’s quality of life.
