Angelique Spaninks had a long talk with architectural designer and Ph.D. researcher Paula Strunden who builds extended reality (XR) models, merging virtual and mixed reality technologies with physical objects and crafts. Paula’s research delves into possible futures of spatial computing and investigates how we can live, work and play within a hybrid physical/digital world. Her latest hybrid installation was called Rhetorical Bodies and premièred at MU as part of the Hybrid Tales for Hybrid Times exhibition. The ideas behind it as well as Paula’s Ph.D. research sheds light on several of the topics within The Toolkit for the Inbetween.

Paula Strunden: The idea to create Rhetorical Bodies stems from the interest to work with embodied virtuality and multi-sensory perception and to think about how we can use and interpret this. I really want to play with and explore our bodies within the virtual in a way that goes beyond this visual representational body that we often refer to that is more like an avatar. So when we think of digital bodies, we often have this kind of image of an actual physical body, but of course our digital body can be a very different thing.

It can be a very fluid and open thing that transcends known visual and physical boundaries. Not only spatially, but also in time. Every person has a body they can feel, but that body also relates with other people and things on another level, it creates connections … I have difficulty describing it, but basically, I feel that our bodily boundaries are very kind of fluid and dynamic and they’re always changing. They are in a certain relation to everything that we interact with, like people, but also of course things and places.

Then I had the idea to mix VR with dancing and create a collaborative experience that plays with these connections. 

So the initial idea for Rhetorical Bodies was to really just concentrate on this feeling of where your bodily boundary sits and how it relates to another person entering that space, and then gradually by moving you are both able to merge with each other and your surroundings.

When I started working on it, many things happened to me at the same time. I think I never had in such a short time, so much sudden experience of letting go and losing – my contract ended, I moved, a relationship ended and I lost my grandmother who was very dear to me. And I think all this very strongly influenced the way how the work turned out to become about letting go. Let go of our body even. 

My grandmother inspired the use of the inflatable wings and with those you go, into the virtual sky and through these portals. There you kind of dissolve and your body comes apart and becomes fluid, a bubbly representation of yourself. And then you start seeing yourself as such and you meet this other person, and you can move and even merge together, and when  you go further down everything starts moving around you and then you’re left to become part of that world.

What I learned is that also in virtual reality, where there often is a lot of technology around it and even when you try to tell a story that is outside of yourself, it is never really outside of reality. So in that sense VR is always a hybrid thing. You bring in your own experience and your own feeling and probably that is somewhere inside of the work that you make, and each visitor brings their experience and reality into it.

Designing for the gap

PS: In a way, I think this kind of work is very close to theater and performance. There you also design for the gap. You create something and then people come to see or to experience it. But you use a certain sense of abstraction or ambiguity and strongly play with the fact that you meet each other in between, in the middle, even though you don’t exactly know where that in between is. I don’t know what kind of people will come, what their memories are, and what the understanding of their bodies is and the abilities of their bodies are.

And I think for everybody, being a fluid blobby merging piece of something, is a new experience. It is very different. That is also so beautiful to see in this kind of installations that everybody co-creates this reality, that it’s not only me as the designer who creates something, but it is also contextualized in a certain way. The person you hire who guides the experience, and the space and the objects in it, everything contributes in a way. And then of course, the people that go in and their constellation and how they relate to each other. It’s all that together that then creates a certain experience. And this makes it a highly subjective, super individual thing.

AQ : For us of The Toolkit for the Inbetween, the in between is an area, the space between the fully physical and the fully digital. How do you look at this space, also considering you are trained as an architect. 

PS: I don’t like at all this idea of VR as alternative to reality or that it is a form of escapism. I don’t think in this quite binary differentiation between the actual and the virtual. I think they’re very much intertwined in the same way that we can have a conversation, but you have your own thoughts, memories and understanding of it and I have mine. I cannot fully grasp yours and you cannot get mine. But it is nevertheless the same real. Like a shared experience of love or an experience of beauty can be very real but will never be exactly the same. And I think these experiential inbetween spaces exist because we experience them extremely strong and we create memories of them.

So for the people that experienced Rhetorical Bodies, they also feel like they’ve been to the same space. I really experienced that while talking to them that they see these environments as a real place. A place that you can go to, that exists in your memory, that you can speak about and that you can share with others.

Places in between thoughts and dreams

And I think coming from the background of architecture, I designed them as places to go to, to be in and to experience certain things. And I think what’s really interesting is that it gives me as a designer the ability to design them as places in between your thoughts and your imaginations. To me the most interesting VR can feel to a certain extent like entering your thoughts or entering your dreams or memories. Something happens within our brains, that’s why it feels so intimate and why it triggers emotions strongly. And I think the moment you start combining these experiences that are perceived as very internal with external stimuli, like touch or smell or certain kinds of movements, one becomes highly alert and experiences very strongly. To me, that’s beautiful because you can reconnect to things and spaces, but also other people in a way that is quite internal. It’s like you extend your body a bit to envelope these things. And this bodily extension creates a certain form of empathy or empathetic imagination, a form of care for things just because it is so much part of what you would usually perceive as internal. I think this is super interesting also in how it affects design decisions.

Female pioneers of multi-sensory interfaces

I am having a public conversation with Jacqueline Morie soon. She’s one of the early pioneers in VR and at one point she systematically analyzed differences in the approach and idea of embodied virtuality. The majority of people building VR in the early days were men. They were building virtual kitchens you could walk around in to design your own, military training scenarios for wayfinding or other more functional VR experiences. But Morie wasn’t interested in those, so she did an in-depth survey where she tried to find artistic, meaningful, very creative virtual reality experiences, multisensory things, with full immersion and interesting interfaces. Of the 100 creations she found that were made between 1985 and 2006-2007 seventy percent were done by women.

Take for example Char Davis, her work is very much about dehabituating the senses and reconnecting sensory modalities to VR in order to understand about being in the world. And I think this goes against the idea of paradoxicality, that it is not something that you try to push these binaries further, but that you acknowledge, okay, I’m here, but I’m there too, at the same time. This is to me the most interesting part of working with VR. And it’s not interesting to overcome that, it’s all about staying within that sensation. 

A dialogue between the body and the brain

Architecture theoretician called Karen Franck wrote an interesting text already in 1996, titled When I enter Virtual Reality, What Body do I leave behind? In this she looks at science fiction literature quoting for example Gibson in Neuromancer where he calls the body ‘meat puppets’ or a ‘flesh cage’. It is his way of defining the virtual body versus the physical one, that should be overcome, because of it’s diseases and aging. While many female sci-fi artists at that point rather embraced the body, and pushed the potentiality of it within a realm that is not constrained by socio political circumstances. They perceive the body, the being in the body, not at all as a constraint, but more as a kind of ability. They try to stretch the possibilities of what can be done within that body.

That is also the way of how I work with VR and use digital tools, in a super bodily experimental kind of one-on-one dialogue. I spend a lot of time just testing and trying things. I rely on how the body responses. How does it tune? How does it change? It’s a very intuitive judgment. Sometimes the medium can make me laugh or cry, and it can make me stunned. I can create things that I don’t understand. For me, this is totally exciting because I am entering a dialogue that is very much between the body and the brain via this technology. It makes you go to places. You don’t know how you got there, but you are trying to understand it, and you can only do that by understanding how our body works, how we perceive reality, how we construct reality, how we constantly stabilize reality. And I think it is also interesting because it’s not about creating alone, but collaboratively.

It’s not about pushing any differences between whether it’s virtual or actual, it’s more about understanding how we are. It’s about being in a certain moment, in a certain time, in a certain presence. It is about staying with the body. It is a very sensual experience. And I do not only mean the common five sense, like touching, seeing, hearing, smelling, tasting but all neuroscientifically distinguished senses, like our sense of time passing by or the sense of our body, our sense of light changing, and our senses of pain and gravity and balance. How we think and understand the world has evolved through the way in which we use our body. And I think in the digital realm everything can become quite fluid and dynamic. There we can connect these senses in a really powerful way. Not using technology to take over bodily experience but to extend and explore it more deeply and through that the realm of the in between. I look for these very small but very significant moments, where people feel present or have the awareness of connection or an understanding you can’t really describe. Like the couple that experienced Rhetorical Bodies together on their first date and fell in love, or the person who felt it came close to a near death experience at the end. I believe these are transformative moments, they suddenly make sense. And of course these moments are really fleeting, 

Finding a language native to the machine logic

But it is not only about our body, there is also the body of the device. The game engine plays a very big part as it defines the logic of the way how this is being set up. There are moments where I realize and have to laugh about how I internalized the logic of how this engine works and not of how my body works in the physical space. So that instead of for example taking off the headset, I wait for something that I know will happen when I do something within the experience. Then I start following the machine logic. And I think this is really interesting as a a designer to find out a language that is more native to the medium. A language that works within its logic and not to blindly follow it, but to be able to understand it as well as have a critical standpoint on it and be able to discuss it, both with the machine, with technology, and each other.

I think a lot about the way how we are being perceived as individuals. Like our fingerprints how they are being used in order to prove that it’s you and not me. And I think everything about us unique. Like within split seconds, machines can recognize whether it’s you or me wearing the headset by the way how you pick up something, the way how you grasp something. The whole way of how we interact with our environment and the things within it and how we understand them is highly subjective. I mean, we just know so little about how we think and how we feel and how we sense certain things. But our body is constantly giving clues about that, to ourselves, but also to others. Most of our communication and our interaction with the world is on a level that we are not aware of.

Technology is bringing the unconscious back

And it has evolved not consciously and that might be the in between also. Technologies in general allow you to go more into this direction. It can bring the unknown and the unconscious back into the quotation, allowing you to take certain decisions in design more intuitively.

It may sound a bit too vague, but I think most of the in betweenness we don’t know, we cannot say, okay, this is a point, and this is a point and there’s something. We only have kind of a cloud of that environment and we don’t know where it’s in between because we don’t know where it goes or how far it goes.

AQ: Is there enough space or room for experimenting and researching these kinds of things? Not only supported by big tech industry but especially for critical artists and designers.

PS: I find it very problematic that big tech companies are ruling VR. I don’t work with Oculus personally, but I think it’s like the biggest device and they are very much leading the content that’s being experienced by people. And I think the more you can help breaking and opening that up and also questioning that, I think the better. 

Many artists try to do that in a certain way, but you can see in the aesthetics that the work is often still defined by the hard- and software available. And there can or should be much more experimentation. But we must take into account the difficulties in developing and exhibiting these interactive and especially experimental works that are often still in progress too. I think there are so many variables, and it is often so fragile and constantly changing. Even for me as a maker it is super difficult to recreate an experience from a year ago.

The Netherlands has a few places, like MU and V2 and The New Institute, that make these kind of complex research projects possible and connect them to an audience. Digital culture does not feel like a niche thing here. And I sense that because of this, people are generally more open to it and willing to find out more.

Because the big thing with immersive interactive work is of course that you need to involve people to know if and how it works or not. You cannot do that by yourself in your studio. You need to have a dialogue with people while developing work. It’s a weird form of art because it’s so dependent on the interaction and the users. 

Even as an art form, it’s very much in between. And I think to have places where this in between is being shown, is super important. It is not theater, you don’t have a set stage to design for, and it’s also not an artwork that sits somewhere, and it’s not a game either that that can be bought and run as a product, and it’s not a performance that you can be in the presence of but are not really part of. It is in between all of this, really.

The body as input device

I like the idea of hacking, of play and experiment. Using technology in a more creative way, but I stil sense that is very much under explored. 

The argument in my PhD is that this comes out of over concentration on the visual. And I think the moment you start bringing in these other sensory modalities it opens up very quickly. But the concentration on the purely visual is an effect of these low cost devices that came on the market, as they were just reduced to a headset and a pair of controllers.

But historically, all the experimentation with virtual technologies was very much about how you interact through bodily movements and always through touch. VR needs to have the body be the input device and the space as the interface. Then it becomes spatial computing or like embodied computing. Then you get this relationship between your body and the things you interact with and the spaces you’re immersed in. And it doesn’t matter so much anymore whether it’s like a photorealistic environment, it will be more about this relation.

The underestimated role of the guide

An important last element that I think is a bit underestimated is the role of the people that can guide and take you through a hybrid or inbetween experience. Onboarding and understanding how to work with the technology is an important part of taking care of people. And if it works well it can be half of the experience, but at the moment, this is still something that is very little invested in. You have museum pedagogy and guided tours, but to be able to enter this in between space you need more dedicated care.

This is of course super expensive to do, but I think in the longer run it also brings a lot of knowledge and experience. It offers the option of building trust and letting go. And I think your emotional availability before you go into an experience, like mine but also others, strongly affects what you see and hear and dare to do. And when this is carefully guided when you enter, but also when you come out afterwards, by having a conversation with a dedicated guide adds to the overall experience and knowledge that is being build.

So not only to have places to show this type of work and dive into the in between, but also experienced people to onboard and guide you that do not change every day is something to wish for when you are exploring unknown territory. You can’t give people like a litre of water and say, like, go and explore. You need to help them with what you know already and how to help them along to get the most out of it.