presenting:

. X-Track .

A wireless smart gadget that connects you to the music visually and physically.

Presentation: https://www.dropbox.com/s/kcjf99ho1zbf6o9/X-Track.pptx
Code: https://github.com/adilyalcin/Xtrack
Instructable: http://www.instructables.com/id/X-Track-Wireless-music-visualization-and-tracker/
Video: Various video shots are included as part of instructable.

Features


Remote wireless visualizations fully synced to the beat:
  • Supporting dynamic control of song playback.
    • You can adjust tempo and the visual will always be on the correct beat, and you can even scratch !!
  • DJ software automatically analyses song bpm (beats per minute), aligns the beats with the song, and send the high-resolution beat phase messages to external control application.
  • The external application on PC broadcasts beat time information to all gadgets in the scene.

Live crowd tracking:
  • By making use of the additional sensors (accelerometer), each gadget sends crowd information back to the main PC for logging and any other purposes.

Small form-factor:
  • The gadget prototype comes in a small package (11x9x3cm). Great for hand-held interaction or wearing ! See photos below (TODO).

Support for multiple remote gadgets:
  • i.e. You can have 200 synced displays in a single room
  • + you can gather sensor data from all (although probably in a smaller sampling rate, more devices will put more pressure on the radio channel!)

Hardware


Summary
On the Device:
  • Arduino FIO ($25)
  • RGB LED 8x8 Matrix ($60)
  • Triple Axis Accelerometer ($25)
  • XBee 1mW Series 1 Radio ($23)
  • 5V Step-Up Break-board ($6)
  • Vibration Motor (Optional) ($5)
  • Polymer Lithium Ion Battery ($17)
  • Case ($10)
  • + Cable wires
    • Total cost = ~$165
    • If ordered as 100 batch: ~130$
    • Minimum = ~$90 (~$70 in 100 batch) (single-color LED matrix, bare accelerometer, less powerful batteries)
On the PC:
  • XBee Explorer
  • XBee 1mW Series 1 Radio
  • FTDI Basic Breakout
    • Total cost: 60$

Arduino FIO (The microcontroller)

external image ArduinoFio.jpg
Info:
Microcontroller
ATmega328P
Operating Voltage
3.3V
Input Voltage
3.35 -12 V
Input Voltage for Charge
3.7 - 7 V
Digital I/O Pins
14 (of which 6 provide PWM output)
Analog Input Pins
8
Clock Speed
8 MHz
Pro's:
  • Includes an Xbee socket already (saves $10)
  • Polymer Lithium Ion Batteries from sparkfun provide 3.3V, no step-up component (3.7 to 5V) needed (saves $6)
  • Includes battery charger using USB port (saves 10-15$)
Con's:
  • You'll need a 3.v to 5.v stepper for the display (costs 5$)
  • No direct programming interface available. You need to use FTDI cables/breakout boards.
  • The longest dimension is slightly bigger than the LED matrix

LED Matrix - Serial Interface - Red/Green/Blue (The Visuals)


external image 00760-04-L_i_ma.jpg

Info:
  • https://www.sparkfun.com/products/760 ($59.95)
  • RGB LED Matrix + RGB Matrix Backpack Controller
  • Communicates through SPI serial input protocol
  • Runs on 5V
  • Current: 120mA (typical) 275mA (max)
  • Dimension: 2.38x2.38"x0.78''
Wiring:
  • Using SPI Interface:
    • VCC(5V)<->5V
    • GND<->GND
    • MOSI<->PIN11
    • CS<->PIN10
    • SCLK<->PIN13

Triple Axis Accelerometer Breakout - ADXL335

external image 09269-1_i_ma.jpg
Info:
Wiring:
  • 5V/GND...
  • X<->Analog Pin 0, Y<->Analog Pin 1, Z<->Analog Pin 2

XBee 1mW Trace Antenna Series 1 (x2) (Wireless Communication)

external image 11215-01_medium.jpg
Info:
  • https://www.sparkfun.com/products/11215 ($22.95x2=$45.90)
  • Point-to-point (PTP) & point-to-multipoint (PTM) radio running the [[http://en.wikipedia.org/wiki/IEEE_802.15.4|IEEE 802.15.4]] protocol
  • This is a simple, cheaper XBee module with enough range, easy to control (than Series 2), allows point-to-point and broadcast communication.
  • I chose trace antenna since the amount of space used was a basic concern, and trace antenna is sufficient.
  • The radio receiver should work inside the plastic box. "The radios do not have any issue radiating through plastic cases or housings, and so the antennas can be completely enclosed in those types of situations."[[@http://www.digi.com/technology/rf-tips/2007/08|info]]
Wiring:
  • Just place it in the XBee socket of FIO!

Vibration Motor

external image VibrationMotor-01-L_i_ma.jpg
Info:
  • https://www.sparkfun.com/products/8449
  • Vibrates on the beat, or controlled manually in real-time!
  • 2-3.6V operating range.
  • Current: 75mA (note that fio provides a max of 40mA per pin)
  • In the current prototype, it sticks inside the case.

FTDI Basic Breakout - 3.3V (For programming the micro-controller using USB connection)

[[image:https://dlnmh9ip6v2uc.cloudfront.net/images/products/9/8/7/3/09873-01_i_ma.jpg]]
Info:
  • https://www.sparkfun.com/products/9873 ($14.95)
  • Works on 3.3V
  • "Uses SMD 6-pin header on the bottom"
  • "This board will auto reset any Arduino board that has the reset pin brought out to a 6-pin connector"
  • "This board has TX and RX LEDs that allow you to actually see serial traffic on the LEDs to verify if the board is working"
  • USB cable required (A<->mini B) . Assuming you have one...
Wiring:
  • Only necessary when updating the program on FIO.
  • TODO (add photo)

XBee Explorer Dongle (connecting XBee to PC)

external image 09819-01_i_ma.jpg
Info:
  • https://www.sparkfun.com/products/9819 ($24.95)
  • Allows you to plug-in XBee hardware to your computer. The computer will see the device through a COM channel.
  • Use X-CTU software to configure your XBee

Logic Level Converter => TODO Replace with NCP1402-5V Step-Up Breakout

[[image:https://dlnmh9ip6v2uc.cloudfront.net/images/products/8/7/4/5/08745-03-L_i_ma.jpg]]
Info:
  • https://www.sparkfun.com/products/8745 (1.95$)
  • "The SparkFun logic level converter is a small device that safely steps down 5V signals to 3.3V and steps up 3.3V to 5V."
  • Necessary since Arduino FIO provides 3.3V output, and the LED Matrix operates under 5V.
Cons:
  • Need to supply both high (5V) and low (3.3V/2.8V/etc) for it to operate :(, so it's not a complete solution.

Polymer Lithium Ion Battery - 2000mAh

external image Batt2AJST-01-L_i_ma.jpg
Info:
Note: Expected power consumption of proposed design:
  • 270mA (max) LED Matrix
  • 50mA XBee unit
  • 30mA (max) arduino
  • Total: 350mA (worst case), 2000mAh should be enought for about 6hrs of unplugged operation!

Project Case

external image 08601-05-L_i_ma.jpg
Info:
Notes:
  • RGB Matrix : 2.38x2.38" (can stick out a little (0.25cm))
  • Arduino FIO :1.1" x 2.6" (Fits next to RGB Matrix)
  • Battery: 0.25x2.1x2.1" (5.8x54x54mm) => Fits under the display
  • The Display can stick out a little
_
| FIO | LED MATRIX |
|_|_BATTERY__|
<-- 3.4'' -> <-- 2.6' --'>


AKAI LPD8 Pad MIDI Controller

external image images?q=tbn:ANd9GcQN3z2kLN7Wmafqct5pGq3eLpLUFdbJ8HcTj7fFvahvT4QvFw4fsQ
Info

The controls are mapped to Preset 4, which by default send MIDI messages on Channel 4.
Button Index Layout:
5 6 7 8
1 2 3 4
Program Change messages: 1-8 => Select the corresponding display type.
Pad messages:
1 => Vibration on beat (toggle on/off)
5 => Force-Vibration (momentary on/off)



Programming


Arduino Programming

Language: Arduino
XBee library: http://code.google.com/p/xbee-arduino
Additional resources:
Notes:
  • Most of the code is for generating different types of patterns and animations for the display (clearing pixels, drawing lines,circles, patterns, etc)

PC Programming (X-Trak Control Software)

Language: Java
XBee library: http://code.google.com/p/xbee-api/
MIDI Library: [[@http://docs.oracle.com/javase/1.4.2/docs/api/javax/sound/midi/package-summary.html|Java Midi package]]
Additional Resources:
Notes:
  • A custom midi message listener is written with help from the MidiInDump example linked above. The message decode function is extended to listen for the specific beat-phase message and send it across using XBee packets.
  • The software also listens to incoming XBee messages through a custom XBee listener, and logs these messages to an external file (by device id and the transmitted x-y-z raw acceleration values).

PC MIDI Communication


DJ Software that extracts beat information from songs : [[http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CFUQFjAA&url=http%3A%2F%2Fwww.native-instruments.com%2Fen%2Fproducts%2Fdj%2Ftraktor-pro%2F&ei=cZG1UMulAdOP0QGczoFA&usg=AFQjCNGW1PTl7qQNPvT9qk9x94kGtGsMzw|Traktor]]

Transferring beat information from Traktor DJ software to X-Trak Control Software
Option 1: Midi clock synch (low-res (1 message per beat), not responsive to scratching and dynamic track control) [[[@http://www.native-instruments.com/knowledge/questions/817/How+to+send+a+MIDI+clock+sync+signal+from+TRAKTOR%3F|info1]]]
Option 2: Using beat-phase output. Higher resolution, responsive to dynamically changing track position
Steps:
  • Add Generic MIDI device to controller list, select LoopBe1 as the output MIDI port
  • Add new assignment control
    • Control: Bear Phase / Midi Range 0-32
    • Assign to any CC MIDI channel&note that you'd like. Just make sure the tracking Java software expects the same MIDI message.
Note: Virtual MIDI connection is required to read the MIDI message on the same PC. Seemingly, Mac already has OS support for virtual midi devices. On Windows, you can use [[@http://nerds.de/en/loopbe1.html|LoopBe1]] Virtual Midi Driver.

Update: Converted to Midi Yoko, since it support ,multiple channels, and basically can duplicate hardware midi controller's input to two inputs, once for the DJ program, the other for the XTrak Control program.
More details on single-client vs multi-client MIDI drivers is [[http://home.roadrunner.com/~jgglatt/tech/share.htm|here]]. Basically, if your MIDI controller runs on a single client driver (most likely), you need to figure out a way to propagate your midi signals to multiple platforms. For this project, I used multiple virtual MIDI devices, and an additional software that can duplicate - re-route MIDI devices.
[[http://www.hermannseib.com/english/miditrix.htm|Miditrix]]


XBee Configuration


In this project, XBee configuration is static, and based on Series-1 devices.

COORDINATOR (PC)

AP=2 Put XBee in API mode (escape control bytes)
CE=1 Make this radio the Coordinator
MY=1234 Set the address of this radio to any arbitrary two byte value
ID=3332 Set the PAN ID to a two byte arbitrary value. Each XBee in the network must have this same value.
CH=0C // Both radios must have the same Channel and PAN ID to communicate

END DEVICE (Arduino)

AP=2 Put XBee in API mode (escape control bytes)
CE=0 Make this radio the end device
MY=5678 (or any other unique ID if using multiple Arduino's)
ID=3332
CH=0C

Note: For higher performance increase baud rate (38400 works nice).


Hacks


Disabling ACK messages
  • Basically, we don't need to know if the displays receive the beat phase messages. Lost packets are not important since new phase info is continuously sent.
  • So, TxRequest16 sent by the PC includes TxRequestBase.Option.DISABLE_ACK, and the frame no is set to 0.
  • Also, note that messages are sent in asynch mode.
Discovering remote devices:
  • If needed, PC can send a broadcast message asking for ACK message from all Arduino's that can receive the message. Received XBee ID's can be recorded. This "discovery" can be repeated once per minute to see if all is working well.
  • Not implemented / not critical.

Central Gadget Control


Turn on/off vibration on the beat
  • Broadcast message. Broadcasted twice (so that it's less likely to get lost).
  • TODO: Auto shutdown if left open (some message may be lost, always vibrating device can be annoying)

Display options


Basic features:
  • Half/double tempo
    • For animations that are atuo-synched to tempo (beat phase, etc), DJ may want to dynamically adjust the tempo to create faster/slower patterns.
    • Right now, it's mapped to key burrons b & n on the processing app.

1. Beat Phase


The beat position in the deck is transferred to remote display. (0-7, 0 being the down-beat)
The column at this position is lighted. It creates a "flow" with the beat-line.

Features:
  • Patterns can follow scratching!
  • The loops (1/2 - 1/4 1/8 beat) directly reflects on the display.

Colors: TODO
Aim: Emphasize the beat and 1/2 beat

Extension:
  • Display one column per deck
    • Will be useful for the DJ who is beat-matching two tracks
    • Also looks nice when cue-juggling at the other deck!
    • Can use alternating rows and different colors for two decks

DJ Controls:
Set lower tempo : When the track is fast-paced (>120bpm) it moves a bit too fast

2. Immediate Mode

DJ clicks a button, and that fires up a specific short pattern in the display

Patterns:
Strobe (on/of very fast)
Color Rush (?)
Sweep
Expand Circle
Expand Disk

Snow mode:
Inspiration: http://www.instructables.com/id/Lampduino-an-8x8-RGB-Floor-Lamp/#intro

Additional Resources


Wireless Programming of Arduino FIO:
  • When XBee socket is attached, you cannot programm Arduino boards over USB (since they use the same serial channel)
  • As a temporary fix, I first "loosely" connected the XBee to the board, so that it is easy to take it out when new program is uploaded.
  • There are two nice tutorials that show how to make wireless programming enabled for FIO




You go out to an event to listen to some DJ. You -hopefully- hear some amazing tunes. But, you don't know what that song was! This is about to become history. As you get into the event, you are given a small smart gadget (matchbox size). It tracks your movements (with accelerometer), so it knows&records when you are dancing. You can also manually push a button to record a specific moment of fun. DJ gets live feedback from all the devices to his laptop. You hand the gadget back as you leave the event, and later you get the songs that moved you physically or emotionally on your e-mail. To make even things funkier, the device has small LED array that can dynamically change with music, also likely to be synchronized among devices. The device will communicate over wifi to DJs computer. People can bring the gadget up, and you get some special lighting/effect for the event. In short, the gadget helps you get the most out of the music event, connecting you to music live (with lights) and after (with highlighted tracklisting).