Engineers develop haptic feedback sleeve and 3D-printed glasses that allow blind people to ‘see’

Researchers at Technical University of Munich (TUM) have developed a novel setup based on infrared (IR) goggles with the potential to enable visually impaired people to effortlessly navigate obstacles in their daily lives.

Comprised of two infrared cameras on an FDM-printed frame, the team’s goggles transmit data about a user’s surroundings to a padded motorized sleeve, which tells them how far away objects are via haptic feedback. . In the future, the system could replace the traditional walking stick, allowing blind people to detect obstacles further away, perform new two-handed tasks and cross with greater independence.

“Our solution is inexpensive, lightweight, and designed for indoor and outdoor use,” the engineers explain in their article. “The haptic feedback sleeve does not interfere with the user’s sense of hearing, which is widely used by the visually impaired. It also eliminates stigma, allowing people who are blind to add a helpful tool for more independence, security and confidence.

The researchers’ 3D-printed infrared glasses and haptic sleeve system. Photo via Manuel Zahn and Armaghan Ahmad Khan.

Modernize the cane

Whether the result of injury, infection, old age or refractive errors, visual impairment affects at least 2.2 billion people worldwide, 40 million of whom are completely blind. Yet, despite the magnitude of the problem revealed by these World Health Organization (WHO) The figuresalong with continuing medical advancements, most blind people still rely on canes for daily navigation.

While these aids provide users with a reasonable idea of ​​their immediate surroundings, they have obvious detection distance limitations and their effectiveness on elevated surfaces such as stairs remains mixed. Likewise, walking sticks can be difficult to use in snowy weather conditions, which obscure the path ahead and often leave blind people in need, limiting their day-to-day independence.

To circumvent these constraints, various electronic travel aids or “ETAs” have been developed which effectively guide wearers, by providing them with an acoustic or haptic “map” of their environment. However, according to the TUM team, many existing ETAs require extensive training to master and interfere with the other senses of blind people, while often facing a trade-off between resolution and lightness.

The cartographic layout of the
The engineer’s “haptic sleeve” map layout. Image via Manuel Zahn and Armaghan Ahmad Khan.

Allowing blind people to “see”

Unlike many conventional haptic ETAs, which rely on Microsoft Kinect sensors, the researchers chose to base theirs around a pair of Intel® RealSense D415 camera mounted glasses. In the team’s setup, data obtained through these cameras, capable of capturing images at up to 1280 x 720 pixel resolution, is fed into a motor-loaded stretch fabric designed to be worn on the forearm.

In turn, this “haptic sleeve,” featuring 25 sewn-in vibration motors, can then be programmed so that its vibration intensity increases as wearers get closer to any obstacle. To make this possible, the engineers found that they initially had to “downsample” the data collected through their 3D printed glasses into a 5×5 array, which allowed them to vary the voltage of each respective motor in their new device.

Once they mapped out their eye-and-arm-mounted setup, the team continued to put it through both haptic pattern testing and an obstacle course, littered with walkways, doors, and doomsdays. potential obstacles. In the first case, the system was able to achieve near-perfect accuracy of 98.6%, but in the second, only four out of five subjects emerged unscathed, showing that it still requires user training.

Impressively though, the researchers’ devices were able to consistently detect indoor obstacles at distances of up to 3 meters, meaning its capabilities surpass even the longest walking sticks, while it also proved capable of operating outdoors in total darkness, so it might even provide users with better nighttime perception than perfectly sighted people.

While the engineers admit that the system’s effectiveness is somewhat dependent on “the musculature of the forearm region”, they claim that users were still able to reduce their lesson times by 53% in testing. With more R&D, including the addition of a voice control system, the TUM team therefore believe their setup could represent a new, accessible way to give “more independence to the visually impaired”.

Heatmaps captured by the team's 3D printed infrared goggles during testing.
Heatmaps captured by the team’s 3D printed infrared goggles during testing. Image via Manuel Zahn and Armaghan Ahmad Khan.

AM in the fight against ocular deficiencies

Instead of looking to treat the problems caused by blindness, researchers at several other universities are now looking to treat the disease at its source, by developing bioprinted corneal transplants. To Marmara Universityfor example, scientists have succeeded in creating new PVA-chitosan constructs, with the light bending properties of real corneas.

Similarly, a team based in Florida A&M University said it performed the high-throughput printing of human corneas in June 2019. Performed using a CELLINK BIO X 3D bioprinterit was believed at the time that the project would lead to advances in the treatment of corneal conditions and other eye-related conditions.

In more recent applications, 3D printing has also been deployed in practice by clinicians at London’s Moorfields Eye Hospital. Using this technology, a team was able to produce the world’s first 3D-printed eye prosthesis and fit it to local man Steve Verze, in a way that drastically reduced the time he had to wait for a replica realistic eyepiece.

The researchers’ findings are detailed in their paper titled “Obstacle avoidance for blind people using 3D camera and haptic feedback sleevewhich was co-written by Manuel Zahn and Armaghan Ahmad Khan.

To stay up to date with the latest 3D printing news, don’t forget to subscribe to the 3D Printing Industry Bulletin or follow us on Twitter or like our page on Facebook.

For a deeper dive into additive manufacturing, you can now subscribe to our Youtube channel, with discussions, debriefs and photos of 3D printing in action.

Are you looking for a job in the additive manufacturing industry? To visit 3D printing works for a selection of roles in the industry.

The featured image shows one of the researchers wearing their prototype 3D-printed infrared glasses. Photo via Manuel Zahn and Armaghan Ahmad Khan.

Sarah C. Figueiredo