Digital - on touching Clouds shown at TanzMainz Festival
02.04.25, SWR Kultur: Mainz tanzt in die Zukunft – Wie die virtuelle Realität den Tanz erobertPremiere of Virtual Frameworks S210
This mixed reality app offers a playful introduction to the potential of upcycling in architecture. The immersand is invited to build new architectures with 20:1 models of formwork designed for the construction site Stuttgart 21 - while experiencing them physically in real size.
Currently shown at the StadtPalais Stuttgart in the frame of the exhibition aus Schalen entworfen.
A piece by Philipp Reinfeld and I, with sound by Franziska Aigner and 3D modelling assistance by Lawrence Forsythe
Premiere of Rose in Your Brain
Very happy to have been able to help Liis Vares and Taavet Jansen shape and implement their XR creation.
"Rose in Your Brain" is a mixed reality performance, a collective thought space where linguistic thinking meets the instinctive physique. It is a physical reading experience, a choreography of thoughts that you can experience within your own body using mixed reality glasses. Plug in — let your body and thoughts carry you.
Something is changing, I can no longer make sense of what here means, as I increasingly feel that while I am here, I am also always a little bit there. There, where my thoughts are, there where my imagination roams — and not only mine, but someone else’s as well, simultaneously. It is real. I am always inside a body, in this form that makes me visible and touchable; yet I am also always in a space, where no one can see me. It is a space of thought, an invisible space, but also an ever-present network, a movement. I am present, held in human.
New trailer of Touching Clouds
Virtu(re)al bodies
Excited to see my article article on XR and dance appear in:
Virtual Ecologies – Digitalitäten und Ökologien im Tanz
Jahrbuch TanzForschung 2023
Yvonne Hardt / Marisa Joana Berg / Anna Chwialkowska / Ulrike Nestler (Hg.)
Warm thanks to the four of them for their support and insightful and rigorous feedback and editing!
After Images | 12.09.24 - 27.04.25
Very grateful to be part of this incredible exhibition!
Artistic Director and Curator: Lisa Long
Assistant Curator: Line Ajan
Curatorial Assistant: Josefin Granetoft
Predominant systems of knowledge have elevated sight above hearing, smell, touch, and taste, despite how seeing is constituted from an amalgam of sensory feedback from the entire body. To question this hierarchy of sight over the other senses, AFTER IMAGES gathers works that rely on materiality, texture, movement, and immersive experiences to convey meaning. The exhibition invites visitors to understand images as just one of the many ways we make sense of the world.
Don't miss the chance to see Touching Clouds at the Julia Stoschek Foundation Berlin!
And of course, none of this would have been possible without the incredible team: Franziska Aigner, Pêdra Costa, Hanako Hayakawa, Göksu Kunak and Simon Speiser.
Mixed Reality Workshop
Excerpt of a mixed reality performance created in the frame of a 10 day intensive workshop organized by NewTalentsRuhr. Grateful to have had the chance to work with these amazing young artists!
By: Scratch, Leo, Selma, Smilla, Julian, Norbert
Sound: akiramus1c
Graffiti: Daniel
Small movements Big questions
26.04.24 19:00 DOCK 11 Berlin
The REALTIME AFFAIRS event series is a regular meet-up for artists, designers and interested audiences who are passionate about audiovisual performances and immersive experiences. The aim is to present formats on the interdisciplinary border of performing arts, gaming and media art.
I will discuss the coding and math behind some popular animaton techniques we used in our recent VR pieces. In this video, for example, we used Perlin noise to animate grass. It was developed by Ken Perlin for the production of nature-like effects on computer-generated surfaces and is used for generating realistic looking terrain, but also the effect of wind on fields of grass.
Dream Stones - New AR creation
Inspired by “Xorandor” by Christine Brock-Rose and Sapara Mythology, Pape and Speiser’s work “Dream Stones” invites into an AR experience in which perception of time between humans and stones are synchronised. Through the collaborative manipulation of the virtual and the physical the viewers step into dialogue with the dream stone worlds.
This new creation extends their inquiry on embodied immersion which they started in Touching Clouds to questions of co-presence, both of humans and other (artificial) forms of intelligence.
Collaboration with Simon Speiser
Dramaturgy: Ben Woodard
I like the Digital but does it like me?
Currently working for the new mixed-reality-project of Julia Krause and Jens Heitjohann, which is about the relationship between future artificial beings and humans. Visitors encounter an ostensibly autonomous avatar in a one-to-one performance via the display of an iPhone. In a try-out of this work at the Heizhaus Berlin, we presented the technical set-up and the possibilities of the newly developed virtual puppet theater setting, which combines the possibilities of automation and live control to create a presence effect. Building on the amazing work of Pangolin Park, I am working on increasing the expressivity and range of behavior of the digital beings, and on integrating the different functions provided by the hardware in order to increase the immersive character of the work in a theatrical setting.
Digitality and performance workshop
Codirection with Laura Cugusi
invited by Theater im Deopt Dortmund and the Akademie für Theater und Digitalität
part of the program New Talents Ruhr
With support of IASA
New AR creation
Collaboration with OBLIQUE SENSATIONS (Christoph Wirth and Markus Wagner)
Music: Christoph Wirth
Design and shaders: Norbert Pape
Coding: Norbert Pape and Markus Wagner
based on Living birds asset by Dinopunch
La visión del Monte
I adapted and developed tools to choreograph the Interbotix robot arm animating Simon Speiser's mesmerizing light sculpture La visión del Monte. In particular, we handpuppeteered the arm, recording the path into a so called BAG file. We then edit the latter using a self-made BAG file editor. Finally, the thus generated movement materials are played back and synchronized to sound files using bash files.
Collaboration with OBLIQUE SENSATIONS (Christoph Wirth and Markus Wagner)
Music: Christoph Wirth
Design and shaders: Norbert Pape
Coding: Norbert Pape and Markus Wagner
based on Living birds asset by Dinopunch
Summertime, math time
A-Life: Life is a property of form
For the development of Touching Clouds, I had to dive into the world of shaders, which are small bits of code executed directly on the GPU. Its parallel processing capability makes surprising real-time simulations of natural process such as swarms and fluid dynamics possible. I am starting to dedicate some time to read up on the surprising cross-polinations of (theoretical) biology, computer science and art. The image below is snapshot of a GPU run fluid simulation based on the incompressible Navier Stokes equations.
What if the complexity of the world around us emerged out fo the interaction of particles following simple rules? 1952, Alan Turing starts off his influential speculative article The Chemical Basis of Morphogenesis with a strong hypothesis:
It is suggested that a system of chemical substances, called morphogens, reacting together and diffusing through a tissue, is adequate to account for the main phenomena of morphogenesis.
In a similar vein but in the 1980s and coming from the now firmly establish field computer science, Christopher Langston notes in his seminal text "Artifical Life":
The most surprising lesson we have learned from simulating complex physical systems on computers is that complex behaviour need not have complex roots.
Polemically, he claims:Life is a property of form, not matter, a result of the organization of matter rather than something that inheres in the matter itself. [...] It is effects, not things, upon which life is based — life is a kind of behaviour, not a kind of stuff [...]. Behaviours themselves can constitute the fundamental parts of non-linear systems — virtual parts , which depend on non-linear interactions between physical parts for their very existence. Isolate the physical parts and the virtual parts cease to exist.
What he calls the virtual is often described in the language of partial differential equations. However, it is usually impossible to write down explicit formulas for solutions of partial differential equations. Today's computing power enables us to numerically approximate solutions and simulate the dynamic processes. One way to do so is by discretization, which means simplifying the problem by splitting smooth spaces into grid-like collections of cells and continuous time into a series of time intervals (steps).
One then translates the partial differential equations into rules defining how cells act upon each other given a short time interval. And voilà, we can now let a computer do the work of computing the state of the cells. Fairly recent developments in GPUs greatly enhance such simulation processes through their 'parallel processing' capability: the state of the cells in the grid are not computed consecutively, but all at the same time! These developments have had an enormous impact on scientific research, but also easthetics. A-Life is ubiquitous in CGI production. Here is another famous example of a dynamical system which can be thought of as the simulation of some sort of bacterial culture:
All possible Rhythms
Working with the Belgian composer Simon Halsberghe to gain a better understanding of the Jacki Lieberman Set. The German percussionist Jaki Liebezeit worked on the set of all possible rhythmical patterns consisting of 1s and 2s, i.e. short and long beats. To complicate the matter considerably, cyclic permutations are considered equal. Formally, this set is defined as follows:
$$ J := \bigcup_{n \in \mathbb{N}} J_n $$
$$ J_n := \left\{ a_0a_1\dots a_{k} \mid a_j \in \{ 1, 2 \}, k \in \mathbb{N} \text{ and } \sum_{i=0}^k a_i = n \right\}_{/ \mathord{\sim}}$$
where
$$ a_0a_1\dots a_k \sim b_0b_1 \dots b_k $$ $$ \iff $$ $$ \exists l \in \mathbb{N} \quad \forall j \in \{0,1, \dots, k\}: \qquad a_{(j + l)\mod (k+1)} = b_j$$
Simon is looking for a constructive way to generate all the patterns of a given length. We have implemented an algorithm, the computing time however quickly exceeds reasonable limits. The follow up question is: How many patterns of a given length $n$ exist? We computed values for $n \leq 26$, but have not yet found the formula for any $n$.
Click here for our intermediate results.
25.05.23: Premiere of Digit(al) - on touching clouds
- a choreographic object
How many angels fit in a cloud? This outdated medieval scholastic problem reemerges today in guise of clouds, bits and data. While much our work consists of manipulating data, we seldom ask: how does it feel like to touch data? What do clouds feel like?
Digitus (lat. “fingerbreadth“, „number“) stems from the Proto-Indo-European *deyǵ- (“to show, point out”). New technologies enable us to, quite literally, touch and handle digital objects. This folding back of touch onto abstraction provides a fertile setting to explore both their materiality and the way they, in turn, move us through space: a choreography.
Digit(al) is an augmented reality piece, as well as a choreographic installation: in a sufficiently large and empty space, informal seating opportunities and headsets are offered to the audience. Those that choose to put on a headset will see the space and people around them enriched with virtual floating stones and leaves, which respond to touch and movement in various ways. The exploration of the virtual space guides them through the real space, producing a choreography that can be watched by other audience members.