vog.photo / Ars Electronica

Life Ink

2022-05 to 2022-09, Austria

Research Question

How much of the creative expression gets filtered out when captured digitally ?

With the digitalization of creation also comes the algorithmic processing of the creative intention. While striking examples today include AI-powered image processing in smartphone cameras, filtering is ubiquitous in creative tools.

Wacom, as the leader manufacturer of pen-and-tablet technology, collaborated with the Ars Electronica Futurelab on examining that question.

To apply this research question, we looked at what kind of “Digital Ink” could encode the information that is filtered out in traditional pen-and-tablet drawing use. In the course of a multi-year collaboration dubbed Future Ink, we looked at unlocking the third dimension of drawing through drones in the Space Ink project, how we could parametrically incubate created patterns of microorganisms in Bio Ink, and finally how we could not just capture the intentionality of the pen, but the inner states of the body and mind at large in Life Ink. This is when I joined the project as its engineering lead, and later the project lead.

In that approach, instead of the mind telling the body how to to use a medium to express its creative impulses, could we get as close as possible to the mind and acquire the creative intention and the states that accompany it, reproducing it in another way ? After using a pen-and-tablet system to draw with space, then with micro-organisms, can we draw with life ?

I joined the project during the base research phase, 6 months before launch. We surveyed a panel of candidate biosignals in a set of subjects undertaking creative tasks, extracting what biosignal-based features we could correlate with the latter. In parallel I evaluated different technological solutions for acquisition, interpretation and visualization. Finally, we produced a room-scale audiovisual performance in the Ars Electronica Center’s Deep Space 8K, interpreting the biosignals of the pianist Maki Namekawa as a monumental virtual 3D sculpture that premiered during the Ars Electronica Festival 2022.

Architecture

In the course of the project, two needs became clear:delivering real-time networking, digital signal process-ing (DSP) and interpretation at more than 1kHz, while being able to modify the computation rules on the fly to fit the artistic research process.

For this task, the need for a custom, real-time, graph computation engine became obvious. Thus, “Sparkler” was born.

Performances

Life Ink was revealed as part of the Ars Electronica Festival’s 2022 Futurelab Night Performances. We designed the performance with professional pianist Maki Namekawa on the grand piano. During the performance, she was equipped with the equipment we designed, acquiring the biosignals of interest. That data was streamed wirelessly in real-time to Sparkler, which interpreted it and sent it to the visualization engine.