Leonardo: Something touched that object!
Adobe AIR: Ok dude, I got it.
We’ve been using the Leonardo version of Arduino board since SleepCycle. It has a keyboard library built in so you can make objects that communicate with Flash or AIR applications running on a desktop machine through the serial port without the hassle of setting up a server or proxy. Actionscript event handlers for the keyboard work well in this configuration (though I’m not sure it would be fast enough for time sensitive <100ms events ).
Word to the wise though – build in a hard-wired kill switch on the Leonardo. During development with the Leonardo running, it will start adding the keystrokes you’re using to communicate events into the document you’re working on. If you happen to be working on the code, it starts entering meaningless characters to it’s own code. Interesting suicidal kind of behavior.
The idea here is to have different objects in an installation react to movement in the space in a way that’s relative to their own location in the space. Video clip here.
I scaled the motion sensing data down and moved it to the bottom of the frame. The objects above are movie clips with their frame numbers displayed. They track the motion data but with an offset based on their position. If motion happens in the middle of the installation, the objects on the edges will point inward toward the motion, while the center objects point straight ahead.
Note that these are all instances of the same object class – so they all have the exact same codebase. They calculate relative zone on the fly.
Made it through the copious hoop jumping to turn a HTML5 project into an provisional app to install on the iPhone by using Adobe’s Phonegap Build service. It essentially wraps HTML5 files in an app wrapper so it gets installed on your device like any other app, and if it’s designed well, looks like any other phone app. Here’s a map of the complex digital signature process iPhones required to install a provisional app on your own device:
To create a zone motion detect system, I took the images created by subtracting consecutive frames, broke it down into zones and searched for the color white.
To break into zones you have to redraw the image one section at a time. The BitmapData class has a masking option in the draw function that allows you to accomplish this by defining a mask area that only redraws the part of the image specified by the mask.
To search for a color in an image and return the boundaries of the rectangle containing that color use the BitmapData.getColorBounds() function.
So essentially you put this in a loop and scan the image. I’m only tracking left to right so my mask defines a rectangle that is vertical slice. Then I move the position of the rectangle in each iteration of the loop.
Since I anticipate much motion in front of the piece I wanted to determine which zone contained the most motion, accomplished with a variable that keeps track of the highest value found in the sweep of one frame and the zone it was found in. Then I recolored that rectangle red for visual confirmation.
See results on Vimeo
I’m working on building a reliable motion tracking system using Actionscript, using an external webcam (Logitech C930e). This requires working with the Camera, Bitmap, BitmapData and Graphics classes.
Basic motion tracking strategies used here:
• capture input from camera
• apply difference filter to subtract from previous frame
• apply threshold filter
Next up is to study the motion of viewers in front of the piece to sort out meaningful data.
Then figure out how to scan the resulting image for white pixels zone by zone, and output the zone with the most activity.
Setting up for opening on Friday, June 13th. Install turns out to be complicated by the fact that we purchased local bricks for the frame – and the best ones were different dimensions than the ones we left in Minnesota. This changed the shape of the rectangle, which meant scaling the masking, which meant scaling the image placeholders, etc.. A nightmare really.
But better than burning all that fuel climbing from St. Paul (elevation 702 ft)
Sleep Cycle will soon be installed in the Soeffker Gallery on the Hamline University campus in St. Paul, MN. Show opens on March 6th. Info about the gallery here
We found out today that SleepCycle got accepted into the Sante Fe International New Media Festival. June 13 – 29th , 2014! See you in Sante Fe.
SleepCycle Version 1.0 is running (from January 3, 2014 until January 19,2014) at the 410 Project, 523 South Front St., Mankato, MN.
SleepCycle is a collaboration between myself and Mika Laidlaw. Seven slip-cast porcelain pillow sleep in the center of the floor on a bed of sparkling black sand. Each pillow’s skin texture slowly progress from youth to age.
Quietness is punctuated by occasional dreams that overtake one of the pillows. If there is no movement in the gallery for a few seconds, the whole set moves into a unified dream state, displaying pieces of an larger image.