I experimented with the 4-wire resistive touch technology before, but it did not allow me to have more than one touch point (mutli-touch). This project uses frustrated total internal reflection (FTIR) and a camera to detect many touch points.

The NUI Group (Natural User Interface Group) is an incredible resource. They have a very active user group and lots of interesting blogs and articles. A good first read is Getting Started With MultiTouch. There are many build logs (like this one) available, for example instructables and Build Your Own Multitouch Surface Computer.

Most projectors and cameras have a ratio of 4:3. Here are some possible dimensions or number of LEDs for this ratio:

n X Width | # LEDs Y Height | # LEDs Diagonal Length Total # LEDs
1 4 3 5 14
2 8 6 10 28
3 12 9 15 42
4 16 12 20 56
5 20 15 25 70
6 24 18 30 84
7 28 21 35 98
8 32 24 40 112
9 36 27 45 126
10 40 30 50 140
11 44 33 55 154
12 48 36 60 168

I decided to build a 24"×18" surface after considering some size constraints and looking what a local hardware store had to offer. I also decide to go with 16 LEDs on top and bottom and 12 LEDs left and right - a total of 56 LEDs. This allows a nice 1½" spacing between the LEDs.

What? Where? Part # How much?
Acrylic Sheet (24"×18"×0.22" & 24"×18"×0.08") Lowe's  
$20
3 × Aluminum Channel for ¼" Plywood (2×4') Lowe's  
$20
LITEON LTE-5208A IR LED, 940nm, 5mm, 40° Viewing Angle, 1.45V, 100mA (1 × px/100) eBay  
$14
Canvas Stretch (2 × 20" and 2 × 26") Michaels  
$8
Microsoft Lifecam VX-1000 Amazon  
$25
12VDC 1500mA AC Adapter RadioShack 273-1775
$23
Adaptaplug 6" Power Leads RadioShack 273-1742
$4
22Ω ½W 5% Carbon Film Resistor - redredblackgold (2 × pk/5) RadioShack 271-1103
$2

Some pictures of the shopping list items:

In the beginning I tried to finish the acrylic surface myself. I bought two sheets - a thicker one for the LEDs and a thinner to cover everything. Especially the thicker sheet was cut pretty badly. It had to be sanded - which took longer then I expected. I used 80, 120, 220, 400, and 600 grit paper. Leave the protective film on, but remove it just a little on the edges for the sanding.

After a few hours I got really frustrated. I was not able to polish the sheet well enough. The edges were still pretty milky and I did not dare to flame-polish them. I found out that a local store (Ithaca Plastics) which offered to sell me a 24"×18"×0.25" sheet for around $60. The edges were crystal clear.

I used a hacksaw to saw off 2×24" & 2×18" rails from the the aluminum channels. Afterwards I used a center punch to mark the holes for drilling. I left a ¾" gap to the end of the rails and marked a hole every 1½", which means 16 holes for the 24" rail and 12 holes for the 18" rail.

I want to solder the LEDs together directly on the rails. To make sure I will not short circuit the LEDs later a put some 1" wide masking tape (Shurtape) over the whole length of the rails and folded it over on the sides. Afterwards I cut out the holes for the LEDs again with an X-Acto knife.

Spacers are needed to prevent to LEDs from being pushed out. I used some ¼"×¼" wood and cut 8 small pieces (0.6"). I super glued one piece on each and of the rails. This provides the distance to the acrylic and also helps that not so much lights escapes on the side.

This is a typical schematic for LEDs in groups. Each LED will draw 1.2V and we want to be as close to the maximum 100mA as possible. A 22Ω will just do the trick and run the LEDs close to 120mA. That means that the whole circuit will draw roughly 12V/1A and you will need a AC adapter that can support this. The problem is, that a schematic like this is not really good to wire around a frame. The next schematic is the same, but more unrolled. There is never more than 3 wires in a place and the positive and negative just wrap around pretty easily.

The LEDs have to be soldered together in groups of 7. Just put the first 7 in the holes and push them down. LEDs are directional components. Always keep in mind:

Cathode Goes to - Shorter Pin Exterior is Flat
Anode Goes to + Longer Pin Exterior is Round

Bent the inner leads to the side, clamp them with two alligator clips and solder them together. Remove the alligator clips and clean up the soldering. Move on the the next two pins. Don't worry about the to outer pins of the group for now.

Do the next group and reverse the orientation of the LEDS for each group. The 24" rails will have a smaller group of 2 LEDs at the end and the 18" rails have smaller group of 5 LEDs. These two smaller groups will be connected at the end (2 + 5 = 7).

The Microsoft Lifecam VX-1000 is already sensitive to IR light, but it can be improved. Just open the camera and unscrew the lens. The IR filter is the reddish square plate, which can easily be removed with an X-Acto knife. Unfortunately I broke my filter, but I was not planning on putting it back in anyway.

The first test was done with the hacked camera and this is the results. The left picture shows my hand hovering over the surface and the right picture shows when I press it down. The contact points are nice and bright, but there is some light leaking from the upper part of the aluminum channels reflecting light before my fingers touch or creating a second bright spot when they touch. There is a small gap on the upper part of the acrylic that needs to be sealed with black silicon. Also, there is a lot of visible light hitting the camera. I need to add a filter to the camera that lets only IR light through.

The easiest way to seal the escaping light is to caulk the gap between the upper part of aluminum channels and the acrylic. I am using GE Black Silicone II. The steps are pretty simple:

After removing the filter the camera still picked up too much visible light. So, I removed the rest of the housing and hot-glued into a plastic box. I got some help to cut a nice hole in the lid for a 55mm Infrared Pass / Visible Light blocking filters - 850nm.

The filter, camera case, and silicone work well. The left image show the nice image the camera picks up - no artifacts. Unfortunately, this all falls apart as soon as I use the vellum as a projection surface. The right image shows the camera image with a lot of pressure applied. The image is just black with light pressure or when my fingers are moving.

To increase the scattering of I had to make a "compliant surface". The best way is described here Tinkermans method: Roll-on Silicone Vellum. I used a bigger sheet of vellum and applied 4 layers of silicon. I had to wait an hour in between layers and at the end. You have to do this outside - the fumes are pretty bad (hence the respirator).

The compliant surface works great. Here are four camera images - the left side show light pressure and the right more (but not excessive) pressure. The camera can also capture the the spots when my fingers are moving.

There different ways how the projector can be mounted. Here is a method that is a compromise between sturdiness, flexibility, and the amount of fabrication. The projector is mounted to an aluminum plate which is clamped to a bigger pipe. A smaller pipe is screwed to some unistrut. This way the projector can be moved up and down (with the unistrut), left and right (with the two pipes), and tilted (with the two pipes).

The first test using the projector showed immediately a hotspot, created by the projector.

This is caused by the projection image bounced off the acrylic back into the camera. The spot must be in the infrared range, because the camera filter eliminates all visible light. I got a 57mm heat-absorbing glass from Surplus Shed for $2, which I just hot-glued to the lens cap (first cut a hole though). Unfortunately, the 2.5mm thickness was not good enough. The hotspot is smaller, but still visible.

I got a second heat-absorbing glass from Surplus Shed - this time a "PRECISION 75MM SQ TEMPERED HEAT ABSORBING SCHOTT KG1 GLASS" for $15 (6.5mm thick with 93% transmission at 375nm).

There is a lot of software available for free. Probably the most import part is the Touchlib from the NUI Group. It makes it much easier to write a multi-touch application, because it takes care of the difficult things and lets the programmer focus on the application itself. It connects to most types of video capturing devices (like web cams), applies filters to enhance the image, detects blobs (finger tips), tracks them, and sends multi-touch events ('finger down', 'finger moved', and 'finger released') to a connected client. It uses the TUIO protocol (which uses OSC internally).

Smoke

The application "demo_smoke.exe" is part of the Touchlib package and it is one of the few demos that does not require to start up the "Vision.bat" from Touchlib before. It is completely self contained. It does not implement any gestures, it just traces any finger movement. You can use as many fingers as you want and more than one person can interact.

Community Earth

Community Earth is an extension of World Wind Java SDK and can be downloaded from the NUI Group project page. It implements a one finger gesture (pan), a two gesture (scale and rotate), and a three finger gesture (tilting). It is important to start up "Vision.bat" from Touchlib before executing "CommunityEarth.exe".

Multitouch Media Application Pro v3

Multitouch Media Application Pro is a project by Laurence Muller. It allows the user to arrange a map, pictures, and movie clips with simple gestures for move and scale/rotation. It can be downloaded from his web site www.multigesture.net.

It is written in Flash and it needs Adobe Air. Unfortunately Flash cannot handle the OSC UDP output, but it can deal with TCP. The good news is, there is a Java bridge program called flosc (the source is an excellent read for anybody who wants to understand the OSC protocol and Flash XMLSockets). That means the different components have to be started up in this order:

  1. "Vision.bat" from Touchlib
  2. flosc (I use "FLOSC-2_0_4-new.jar")
  3. mmapro (you have to install the mmapro first and you can add pictures and movies to the local directories)

Multitouch Puzzle Game

Multitouch Puzzle Game is a project by Laurence Muller. It is much simpler than MMAP - an image is divided into a few tiles (the number depends on the selected difficulty). The user can move the tiles around to assemble the whole picture. There is no snapping or any indication that the puzzle is solved. It can be downloaded from his web site www.multigesture.net.

It is also written in Flash (but it does not need Adobe Air). Again, the startup order is important:

  1. "Vision.bat" from Touchlib
  2. flosc (I use "FLOSC-2_0_4-new.jar")
  3. "PuzzleGame.exe"