The Muted Residency tied together Christophe Monchalin and the research project Content4All. Christophe Monchalin is a Brussels-based artist. He is specialized in the fields of motion design and digital art, with a background in science and robotics. Through his various visual and interactive works, he digs into the very nature of human feeling and behaviors. Content4All explores the relationship between how human beings describe things and the visual and linguistic paradigms. The vision of the project is to be able to translate spoken language automatically in sign interpretation. In this context, they worked on the creation of a photorealistic 3D avatar, which implied lots of challenges around data collection, user experience and aesthetics. At the meeting point of those two entities happens Muted. Muted is a poetic virtual reality experience that explores the artist’s subject of choice, through the story of abandoned siblings that were forced to invent their own sign language in their childhood. The story, being narrated as they are now adults, is narrated through their language, expressing feelings that the spoken word can’t express, enhanced by dreamlike visuals in a 3 aesthetic phases scenario, evolving from filmed dancers to 3D poetical models.
On Content4all’s side, the challenges were multiple, the biggest being the creation of a photorealistic 3D avatar, which implied lots of sub-challenges around data collection, user experience and aesthetics. The goal of the artist, on his side, was to find the closest, the most relevant visual form to follow the function: how can the body convey the message in the best possible way? How can the medium fit as good as possible the message, possibly even enhancing it? These questions were dug using the different technologies and facilities made available by the research project. The innovation in MUTED residency did not lie into how to render photorealistic 3D avatar but rather on how to convey sentiments and emotions with a semi-realistic avatar. This was done through the real-time rendering in a 3D virtual environment of an avatar point cloud. More specifically, during the residency, a dancing choreography was recorded using the HHI volumetric studio. Recorded files were then converted into a point cloud to create the avatar of the dancer. The recorded avatar was then rendered in real-time into a 6 degree of freedom virtual reality environment with which ones can interact with it by touching it or moving around it.
Interview with Christophe Monchalin at CENTQUATRE-Paris, during the STARTS Residencies Days 2020