HandSight
Darren Smith & Lee Stearns

Description

HandSight is a prototype glove to aid the blind. It can sense the lightness or darkness of a surface with tactile feedback from a vibration motor for each individual finger. It can also sense distance from physical objects or obstructions and indicate direction and distance with the same vibration feedback. It supports additional modes, and the possibilities are nearly endless. The glove can connect over Bluetooth to switch modes and visualize the sensor readings.
See our instructables page for how our project was built.

Video



Components

  • Arduino Pro Mini (5V)
  • Infrared Reflectance Sensors x4
  • Vibration Motors x4
  • Ultrasonic Range Finders x2
  • Custom 3D-Printed Cases x4
  • 9V Battery and Case with Switch
  • Glove

Modes

Shape Detection


The shape detection mode is used to feel out the edges of the object, after which it is possible to build an internal representation of the object's shape. Our initial inclination was to have the device vibrate whenever the sensor crossed a threshold, an edge detection mode. However, we obtained much better results when we set it to vibrate whenever the sensor reading was between an upper and lower threshold for black. The upper threshold prevents the device from vibrating while in midair, while the lower is the threshold between black and white.

Texture


Both the sensors and the vibration motors support analog modes, so there is no reason why our feedback needs to be binary. The texture mode allows the user to feel variations in the texture of an image. There is a limit to how much a user can differentiate the varying degrees of vibration, but it is good enough for course uses.

Navigation


The navigation mode switches from the four infrared reflectance sensors at the fingertips to the two wrist-mounted ultrasonic range finders. This mode simulates a white cane, allowing the user to sweep their arm back and forth and detect nearby surfaces or obstacles. The vibration motors on the fingertips give feedback based on how close the detected object lies, one motor for each sensor on the index and pinky fingers. The ultrasonic sensors work best on flat surfaces perpendicular to the user. They still work at oblique angles, but less reliably the further from 90 degrees you go. They also occasionally have trouble with cloth or heavily textured surfaces, and will obviously fail in extraordinarily noisy settings.

Massage


Vibrating fingertips? How could we not add a massage mode?

Typing


The typing mode is more of a proof of concept at this point. We only have access to four sensors on our device that can be used for this purpose, which limits us to 16 characters. Adding an additional sensor, or perhaps support for gestures, would allow us to make this into a useful feature. However, our current components do not support this.

Challenges

Getting Clean Readings from IR Reflectance Sensor


The infrared reflectance sensor that we decided to use gives readings over the analog pin. It uses the built in pull-up resistor on the Arduino, and varies from 0 for strong reflectance to 1023 for no reflectance. Since black surfaces tend to absorb light while white surfaces reflect it, we can use these readings to differentiate between the two fairly accurately.

However, the sensor is quite sensitive to the angle and distance at which it is held from the surface. Its maximum range is 2cm, but it works best at about 1-2 mm. There needs to be enough space for the light from the built in infrared LED to spread a bit. The sensor also works best if it is held perpendicularly to the surface.

We solved both of these problems by printing a custom case for the sensor on a MakerBot 3D printer. The case holds the sensor away from the surface, and it is wide enough that when a user presses their finger down the sensor is held flat automatically. The case also houses the vibration motor and contains several small holes for sewing it to the fingertip of a glove.

Bluetooth


Bluetooth allows us to monitor and control the device wirelessly. The Bluetooth module we used connects to the serial pins on the Arduino, and so using it is no different than communicating over a USB cable.

We developed a Windows Phone 8 app to communicate with our device. It is paired with the Bluetooth module, so connecting is as easy as pushing a button. The app visualizes the readings from the six sensors, and has several buttons for calibrating the device or switching modes.

Managing Complexity and Tediousness of Many Parts in Small Area


By far the most time consuming part of this project was building the prototype. After 3D-printing the cases for the sensors, we needed to thread wires through the hold and solder the sensors in place. The infrared reflectance sensor consists of an IR LED and an IR sensor, and when we add in the vibration motor we end up with six wires for each finger. Add in the four wires for each of the ultrasonic range finders, six for the bluetooth, and two for the battery, and our device quickly becomes a tangle of wires and solder. Aside from managing the pathways, it was also crucial to wrap all of the connections in electrical tape and solder the wires carefully to the microcontroller so that there are no short circuits.

It was also necessary to sew each of the sensors to the fingertips of the glove, as well as a pouch for the battery and Velcro to cover the circuit. Future iterations of this device should focus on refining this process so that the device is more durable and so that it is possible to wash it.

Difficulty of Interpreting Vibrations

Although our device makes it possible to feel out the shapes of line drawings and objects, there is still a learning curve. It takes some time before a user can reliably assemble the coarse haptic feedback into an internal visualization with any accuracy. We were able to improve this by switching from edge feedback to black feedback, as described above in the shape detection section. Eliminating the lag between the sensor and the vibration feedback seemed to help as well.

Deliverables

GitHub
Slides
Video
Instructables

Resources

Circuit Diagram for IR Reflectance Sensor - user jmillerid
Infrared Reflectance Sensor (SparkFun)
Zener Cards (for shape identification)
Political Map (for testing grayscale texture)

Inspirations

The Tacit Project (Sonar for the Blind)
Keyglove Wearable Input Device (KickStarter)
Magic Finger (UIST 2012)
Designing Easily Learnable Eyes-free Interaction - Kevin Li
Haptic Steering Wheel
Apple Patent for Haptic Screen
vibration grid for finger (It must exist, but we couldn't find a reference)

Individual Techniques for reading Braille:
http://www.youtube.com/watch?v=A_fNTvWXdFU
http://www.youtube.com/watch?v=xqSAEMZdoOs

Wearable Assistive Devices for the Blind
  • Survey of wearable devices for blind
  • Small tactile displays for finger tips
  • Wearable devices on wrist (with picture)