CONTENT4ALL stands for “Personalised Content Creation for the Deaf Community in a Connected Digital Single Market” and it is a project financed under grant nr. 762021 within the framework program H2020 of the European Commission.
In 2001 the Convention of the United Nations prepared an agreement on the rights of persons with disabilities, which was ratified in 2008. Applying this legislation into the domain of TV, for Deaf people spoken TV content must be represented via sign language, whose production costs are very high. Hence, a lot of broadcasters use subtitling as a convenient solution. But for most Deaf people sign language is their mother tongue, written language approaches as a foreign language for them. Therefore, full accessibility of TV content for the Deaf can only be provided via sign-interpretation.
The CONTENT4ALL project aims at making video content more accessible for the sign language community by implementing an automatic sign-translation workflow with a photorealistic 3D human avatar. A low-cost solution for personalized sign-interpreted content creation can address both of these problems, leading to greater accessibility to media content for Deaf users. CONTENT4ALL proposes such a solution to the problem in the short-term, which is also commercializable, and proposes innovations to technologies that can lead to automated sign-translation capabilities in long-term.
Muted
Muted is an interactive installation of virtual reality that poetically tackles the subject of abandonment through weightlessness and sign language.
This is a story of a young girl who remains silent. Faced with abandonment, she imagines a sister, creates a whole universe and invents a sign language to communicate. The experience will take you to the depths, to the bottom of water, slowly, almost weightlessly along a thread as an apneist would do. The installation favours sensation as a journey in this story where violence is done through absence and silence.
Muted is not a virtual reality film, but a six degrees of freedom interactive experience that allows you to be fully integrated into the universe.
The artist wants to address the notion of abandonment through the impossibility of the spoken word. Two dancers will perform a choreographed sign language and from their movements, poetic illustrations and animations (2D & 3D) will appear in space. The whole experience will build a small story, made understandable without words.
Christophe Monchalin
Christophe Monchalin is a digital artist and motion designer, based in Brussels. Graduate engineer in computer sciences and robotics (IMERIR / France), he develops his digital work through visual and interactive art. His artistic research explores mainly human feelings and behaviours.
The Muted Residency tied together Christophe Monchalin and the research project Content4All. Christophe Monchalin is a Brussels-based artist. He is specialized in the fields of motion design and digital art, with a background in science and robotics. Through his various visual and interactive works, he digs into the very nature of human feeling and behaviors. Content4All explores the relationship between how human beings describe things and the visual and linguistic paradigms. The vision of the project is to be able to translate spoken language automatically in sign interpretation. In this context, they worked on the creation of a photorealistic 3D avatar, which implied lots of challenges around data collection, user experience and aesthetics. At the meeting point of those two entities happens Muted. Muted is a poetic virtual reality experience that explores the artist’s subject of choice, through the story of abandoned siblings that were forced to invent their own sign language in their childhood. The story, being narrated as they are now adults, is narrated through their language, expressing feelings that the spoken word can’t express, enhanced by dreamlike visuals in a 3 aesthetic phases scenario, evolving from filmed dancers to 3D poetical models.
On Content4all’s side, the challenges were multiple, the biggest being the creation of a photorealistic 3D avatar, which implied lots of sub-challenges around data collection, user experience and aesthetics. The goal of the artist, on his side, was to find the closest, the most relevant visual form to follow the function: how can the body convey the message in the best possible way? How can the medium fit as good as possible the message, possibly even enhancing it? These questions were dug using the different technologies and facilities made available by the research project. The innovation in MUTED residency did not lie into how to render photorealistic 3D avatar but rather on how to convey sentiments and emotions with a semi-realistic avatar. This was done through the real-time rendering in a 3D virtual environment of an avatar point cloud. More specifically, during the residency, a dancing choreography was recorded using the HHI volumetric studio. Recorded files were then converted into a point cloud to create the avatar of the dancer. The recorded avatar was then rendered in real-time into a 6 degree of freedom virtual reality environment with which ones can interact with it by touching it or moving around it.
Interview with Christophe Monchalin at CENTQUATRE-Paris, during the STARTS Residencies Days 2020
