top of page
effects of touch
This physical, interactive installation explores the physical, mental and cognitive impacts of touch and human connection between two individuals. The captivating visualizations ventures into the realm of what is conventionally obscured, providing a unique opportunity to experiment with and explore the delicate sense of vulnerability that accompanies this experience.
Demo of the interactive installation
Brainstorming + Ideation
We've all had different experiences, and feelings, relating to touch. Sometimes, we feel fearful or triggered. Other times, we feel calm, comfort and relief. This project came from a pretty personal place as I was interested in exploring how my body reacts to different types of touch, or touch from different people.
I explored all of these concepts in a mindmap I created as shown below.
Mindmap showing my conceptual exploration
I explored different ways in which I could gather a user's touch data, and how I could use this data to generate a visual output. I decided on a wearable glove-like device that consists of pressure sensors and a pulse sensor at the tips of the fingers that one of the users can wear. The same user would also be wearing a BCI device.
I hence made use of (1) Pressure of touch, (2) pulse from user and (3) brain wave data as input.
I originally wanted to create a physical, kinetic sculpture that visualizes the input. However, I decided to simplify things and make use of a simple Touch Designer animation). Each input would control a different factor of the animation (ie. Higher the pressure sensed on the glove device, the bigger the particle size in Touch Designer etc.).
Initial sketch of how I imagined the end-result to look like
Process
Pressure sensors
Pulse sensor
Technical conceptualization of the wearable glove device
Building
(1) Arduino
(A) Pressure sensors
The pressure sensors were pretty straightforward, and I just sewed them onto a glove that fit my hand. I would then take the average of the 5 pressure sensors to determine the changes in the Touch Designer animation.
Above: Wired up pressure sensors
Above: Glove prototype with pressure sensors hot-glued on
Above: Creating the glove
Above: Revised glove, with pressure sensors sewn on instead
(B) Brain wave reader
For this project, I made use of a second-hand, Mindflex EEG reader that I got off Ebay.
I followed AutoDesk Instructable's tutorial, as well as Frontier Nerd's tutorial on how to hack a Mindflex reader and connect it to Arduino. It was a much simpler and smoother process than I thought it would be, and it worked on the first try.
The brain wave reader was able to provide the following results/ cognitive states in numerical numbers: signal strength, attention, meditation, delta, theta, low alpha, high alpha, low beta, high beta, low gamma, high gamma. I was then able to make use of the numerical numbers to influence the colors of the animation- which I will delve more into below.
One difficulty I faced, though, was that my signal strength kept remaining at 200, which (counterintuitively) suggests that the signal strength is at the weakest (0 indicates strongest signal strength). I was hence unsure as to whether the data I gathered was accurate, but the values seemed to change every time I put on/ off the Mindflex reader, so i assumed it was working okay.
Hacking the Mindflex reader
Inside the MindFlex reader
Wearing the Mindflex reader
(C) Pulse sensor
The pulse sensor was pretty straightforward as well, but it was difficult to tell whether the results were accurate or not. The data would then be sent to, and processed in, Touch Designer, and it influences the audio that plays from Touch Designer. I delve into this with greater detail below.
(D) Code
Here's how the Serial Monitor looked like when we uploaded the code. We then proceeded to process this data in Touch Designer via Serial Communication.
My Arduino code + Serial Monitor results
(2) Touch Designer
I then created an interactive Touch Designer animation that took in data from the Mindflex Reader, Pulse Sensor and Pressure sensors to create different visuals. I wanted the inputs to affect the visuals accordingly:
- Brain wave data affects the color of the animation
- Pressure data affects the size of the animation
- Pulse data affects the audio of the animation
(A) Serial Communication w/ Arduino
I referred to this tutorial by Don't Just Exist on YouTube on how to send data from Arduino to Touch Designer. Deriving the data was straightforward. However, the part that was extremely difficult for me would be the processing of data in Touch Designer. I struggled trying to split, select, convert the strings from Arduino into integers that can then be used as numerical data to control the animations in Touch Designer. It was also difficult trying to make the network as efficient as possible, but my network works for now!
My Touch Designer network: processing data from Arduino
(B) Visuals
I made use of Arduino data to control the visuals of the animation in namely two ways:
- The higher the pressure on the pressure sensors, the bigger the animation
- The more focused a person is (the higher the value attached to specific brain waves), the more elements of red in the animation
I mainly followed and modified this Touch Designer tutorial by Pao Olea for the base animation.
Real-time Touch Designer x Arduino communication
(C) Heartbeat audio
The user's heart rate was basically derived from the Pulse Sensor that the user was holding. The data from the pulse sensor is then sent into Touch Designer. The heartbeat audio that plays in Touch Designer is the user's exact heart rate in real time. I wanted to experiment with the sense of vulnerability that comes with making the user's heart beat audible to the other person interacting with them.
Putting all hardware components together
Tutorial
I also created a short tutorial on how I went about creating this project!
Final output
Possible extensions
-
Including more pressure sensors throughout the glove, so that I can track the surface area + movement + speed of touch. This data can then be used in relation to the brain waves + heart rate, and we can see whether or not there is a co-relation between the data sets.
-
Using a physical, kinetic sculpture instead of a digital animation from Touch Designer.
Special thanks to:
-
Julia Daser for helping to document this piece, and for doing all our projects together
-
Torin Blankensmith who was the professor of my Touch Designer course for a semester. Thank you for providing so much guidance with relation to Creative Tech, and for spreading your passion for TD!
bottom of page