Sunday, January 29, 2012

BrainCam

The BrainCam is a wearable camera that edits pictures to reflect your brain state.


A toy EEG device interfaces with the camera, and records the user's brain state while simultaneously taking pictures.  The data is logged onto a memory card using an Arduino microcontroller, and later the images are systematically manipulated to reflect the user's brain state using Processing.  Two MindFlex variables, Attention and Mediation, were used to manipulate the images.  Below I show the unaltered image on the left with the altered image on the right.














Fair Warning:  This is my first project using Arduino / Processing.
For further technical documentation, check out the repository or the hacking instructions below, and feel free to comment with questions.

The Hack
The camera can be built for less than $150
Components:
Arduino Uno
Matel MindFlex                       /   Matel via ebay
Serial TTL JPEG camera      /   LinkSprite via sparkfun
JST Jumper                                /   sparkfun
SD card interface                  /   sparkfun or other
Hookup Wire
9v Battery Holder or other portable power source

Step 1: Hack The MindFlex
Spoiler alert: This is easy!  You only need to solder 2 wires.

Eric Mika and friends of Frontier Nerds hacked the MindFlex a couple years ago and published a well documented tutorial here with a video.  After completing the hardware hack, upload Eric's sample code with the Brain library he developed and open the Arduino serial monitor to make sure the hack is working properly.  Don't worry if you get an error about packet size.

Step 2: Download the latest BrainCam Repository
Download the whole project, you will need to move some files around to make it work properly. Open your Arduino Sketchbook folder (press cmd + K in the Arduino IDE). Move the BrainCam_*_*_*.ino and the EEPROM_reset file into the sketchbook folder. If you already have a libraries folder, copy the contents of the BrainCam/libraries folder into your sketchbook libraries folder.  Otherwise just copy the whole libraries folder over.  If you already have libraries that I have included, you should be sure to replace them.  I have edited the libraries to reduce the amount of RAM used, and the BrainCam will be unstable if you use the old libraries.

Step 3: Wire It Up!
You may want to zip-tie your Arduino to your MindFlex.  Then attach the SD shield, or whatever SD interface you've chosen to use.  The SD uses digital pins 8, 10, 11, and 12.

Connect the jumper to the camera. The libraries set up the camera on a softserial port on pins 2/3.
Connect the camera:
TX -> Digital 2
RX -> Digital 3
VCC -> Arduino 5V
Ground -> Arduino ground

Then connect the MindFlex as in Eric's tutorial:
Ground -> Arduino ground
MindFlex RX -> Digital 0

Power:
I'm using a 9V right now, but they conk out pretty quickly (the system probably pulls ~200 mA)  so I'd suggest a LiPo system for anyone who plans on taking pictures for a long time.

Step 4: Upload the Arduino Code & Take Pictures
Unplug the MindFlex from Digital 0, then upload the BrainCam_*_*_*.ino sketch.  Plug the MindFlex back in, power it up and put a micro SD card into the SD shield.  Reset the Arduino.  Open the Serial Monitor (Ctrl + M).  You should see a message that says INIT for about 10 seconds.  Then the camera will take a picture and print the size of the image.  Then it will read out the MindFlex data about once a second.  The system is based on a constant interval between MindFlex updates.  If it gets out of sync it will print OOS, followed by CLR when it is back in sync.  Once the Arduino finishes uploading a picture to the SD card (this takes ~90s!) it will print the number of the completed image, and the number of the next image.  Then it will take a picture and print the size of the new image.

If your program never passes the INIT stage, then double check that everything is connected properly, and that you replaced any libraries with the ones provided in the repository. Double check that you didn't forget the SD card or forget to power up the MindFlex!

I have added some photo data to the MindFlex data that Eric parsed out.  The first number is the number of seconds that the Arduino has been running, the second is the image number as an integer, the third is the image number as a string, and after that the rest of the data the brain.readCSV() data as described by Eric.

To make the system mobile, unplug from the computer and attach your 9V.  I fixed my camera into a little plastic housing and attached it to the MindFlex headband, it doesn't seem to have altered the function of the mindFlex at all.  Also, the camera logs the current file number into EEPROM which is maintained after a reset - so if you accidentally reset the camera or need to change batteries, you won't overwrite / corrupt existing images.

Step 5: Data
The memory card now contains a LOG.txt file and numerical image files starting with 100.txt.  Place all of these files into the Img_Processing_*_* folder.  Run the RenameImages100.vbs script (Windows) to rename all of the image files from .txt to .jpg.  I'm sure there's an easy way to do this on a Mac, but alternatively you can rename each file individually.  Then open and run the processing sketch (Img_Processing_*_*.pde).  A window will pop up showing the original image on the left, and the edited image on the right with the meditation and attention values to the right of the edited image.  Pressing the left and right arrow keys scrolls through the images.  The Processing sketch is pretty rough right now.  I'm working on adding a better UI, documenting and increasing functionality.

1 comment:

  1. Cool project. Are your readings making sense though ?

    ReplyDelete