Tech Project
Description of the challenges faced by the Tech Project
VR-Together will offer new ground-breaking virtual reality experiences based on social photorealistic immersive content. For this purpose, it will develop and assemble an end-to-end pipeline integrating state-of-the-art technologies and off-the-shelf components. The challenge of VR-Together is to create photorealistic truly social virtual reality experiences in a cost-effective manner. Immersive media production and delivery through innovative capture, encoding, delivery and rendering technologies. The artist will have access to hardware and all the related software necessary for his development work. The technical components available will be the ones supported by the project's three distinct self-representation configuration modes. Namely: -Point cloud configuration -Time varying mesh configuration -Web based configuration Each configuration has a different software implementation but they all follow the general architecture as it is shown on the third picture of this page's project description. The artist will be guided through the complex aspects of each configuration in order to develop his work without the need of explicitly understanding the underlying technology but merely the end-user projection; the layer in which it is expected for him/her to realize a new creation.
Brief description of technology
VR-Together is a project developing an end-to-end system for the production and delivery of photorealistic social immersive Virtual Reality (VR) experiences. The software architecture of project's (i.e. from capture to consumption) chain is comprised of 5 main components. These components serve as a basis of the integrated platform structure, and each one of them is a conceptual entity related to a general task within the end-to-end communication system. -The Capturing component captures / produces the visual and audio data, performs content reconstruction tasks and synchronizes the captures audio and video signals. -The Encoding & Encapsulation component encodes, encapsulates and packages the audio and video signals received from the previous component into one single stream. -The Delivery component makes this content available for consumption on the network. -The Orchestration component provides end-users with the information necessary to initiate a communication session over the VR-Together platform. -The Play-Out component is responsible for rendering and presentation of the immersive contents. It includes the contents for the shared virtual scenario and end-users’ representation. It includes modules for unpacking, demuxing and decoding and synchronizing the audio-visual content. End-User representation is seen in 3 ways: -Time Varying Meshes -Point clouds -2D Video
What the project is looking to gain from the collaboration and what kind of artist would be suitable
VR-Together is a project aiming to achieve high level of immersion by employing photo-realistic end user representation. In addition to that users are brought together in a virtual environment platform, where they are expected to interact and perform together an action that would otherwise would not have been able to be complete. While working on VR-Together we have discovered that the level of the quality of the virtual world in which the users are put largely defines the feeling of immersion of the experience. Furthermore, it would be great if an artist, given the current characteristics and limitation of the project, would be able to conceive and develop innovative and unique ways (such as a new narrative) in order to intensify immersion as well as users' collaboration. Ultimately the goal is for the artwork to reflect on current and new use cases and identify new aspects of the VR-Together's technology usage.
Resources available to the artist
The artist will have access to an office space where technical experts of the project will be able to provide him with all the necessary information required to contribute to the project and develop his/her exhibit. Depending on the location of the artist access to a lab node where the VR-Together equipment is deployed might be provided. The basic equipment available for the artist will be: -a head mounted display (e.g. Oculus Rift) -a set of image capturing hardware components (Xbox ONE Kinect or Intel RealSense) -a workstation capable of managing the high processing needs of the VR-Together project. -all the related VR-Together platform software components The artist can be hosted in the premises of: -i2CAT, in Barcelona (Spain) -CWI, in Amsterdam (the Netherlands) -CERTH, in Thessaloniki (Greece)