Arduino. Theremin. Gangam Style.

Lee Stearns, CS Masters
Harish Vaidyanathan, HCI Masters

Introduction

Our inspiration for this assignment was the theremin, a strange musical instrument that is played with two hands without touching any keys or controls. We wanted to simulate the experience using our Arduino and distance sensors. This resulted in a hardware package that could be used for many purposes, so we also experimented with other musical and gaming applications.



Hardware

These are the components we used to build our system:
  • Arduino Uno
  • Breadboard and wires
  • 2 types of distance sensors (infrared and ultrasonic)
  • Laptop computer

Overall_circuit.png
sensors_used.png

Discussion

We tested two types of distance sensors to see which would function better for our purposes. The first set of sensors were infrared sensors from Sharp which have a range of 10 to 80 centimeters. The second were a pair of generic ultrasonic rangefinders found on Amazon, which have a much larger range of 2 centimeters up to 3 meters.
The two types of sensors each have their strengths and weaknesses. On one hand, the ultrasonic sensors have a wider range and field of view. The ultrasonic distance readings are a lot more stable. However, these sensors really need a flat surface for better object detection. This is the due to the fact that ultrasonic sensors emit pulses in a cone and the cone diverges as the distance increases. Further, excess background noise can disrupt the function of the ultrasonic sensors. The refresh rates with ultrasonic sensors is also a bit lower than that of the infrared sensors, due to a combination of hardware issues and the relative speed of sound versus light. Infrared sensors emit a very narrow beam. As a result, any available object can be used as an input device and they are better at detecting hand movements.
Both of the sensors produce noisy readings, which we were able to work around by implementing a low-pass filter which simply averages the last few distance readings. This adds a bit of lag, but it is small enough to be barely noticeable.

Usage

The two sensors interface with our Arduino board a bit differently. The IR sensor functions as a potentiometer, outputting its distance reading as a voltage value over an analog pin. We can then convert this voltage reading to a physical distance in centimeters using the following equation (derived from a graph in Sharp's documentation):

volts = analog * 0.0048828125
cm = 27 / volts^1.1

sharp_ir_voltage_graph.png


The ultrasonic sensor interfaces with the Arduino over two digital pins: one for input and one for output. A command is sent over the first pin telling the sensor to send out a audio ping and wait to hear its echo. The amount of time between the ping and the echo is then read back over the second pin. This distance value can be converted to centimeters by dividing it by 58.

Once we have our distance readings for either device, the functionality is the same. The values are output over the serial port, and can be read by an application on any operating system and programming language that can interface with the serial ports. We developed our demo applications using C#.

Application 1: Music (Theremin)


lee_theremin.png
















In this application, we simulate the functionality of a theremin using the two distance readings from our device. The program generates a periodic sound wave that plays constantly and changes form depending on the settings and on the input from our device over the serial port. The distance values are connected to two variables, which control the frequency (pitch) and amplitude (volume) respectively. Smaller distance readings correspond to higher frequencies or amplitudes, while larger readings correspond to lower frequencies or amplitudes, resulting in silence when the user's hand is not over the sensor. The volume is locked in place when its value jumps suddenly, making it possible for a user to set the volume and then take their hand away.
We provide intuitive visual feedback by displaying the currently playing waveform, a ten second history of that wave form, and the corresponding note on a piano. We also include several changeable settings. The user can adjust the number of octaves the frequencies will span, making it easier to select more precise notes with a smaller number of octaves. There is also a collection of settings to round the note to a desired scale so that it will stay in tune while playing a song. There is a tremelo feature that applies a sine wave at a different frequency over the waveform, simulating vibrato. Finally, there are several different types of additive waveforms, which are combined to simulate different types of instruments.

Application 2: Gaming (Pong)

pong_lee_harish.png

In this application, we created a simple 2 player pong game. One player controls the left paddle and the other player controls the right paddle. The paddle movements are associated with the distance measures from the 2 sensors. When a player removes his paddle away from the sensor, the virtual paddle is locked in place and it turns red to indicate to the user to move his paddle in the detectable range of the sensor. We also apply additional filtering to help improve the stability of the paddle.

Links and References


YouTube Video Demo
GitHub Code

NAudio .Net Sound Library
Arduino - Using a Sharp IR Sensor for Distance Calculation