Tech Project

Description of the challenges faced by the Tech Project

The project addresses scientific, pedagogical, and technological challenges. Scientific challenges include gaining a deeper understanding of multisensory integration in typically developed and impaired children and identifying the most suitable modalities to teach specific concepts. Pedagogical challenges include the definition of a multisensory embodied and enactive pedagogical framework for teaching and learning arithmetic and geometric concepts, to be applied in a similar way to both typically developed and impaired children. Technological challenges include the development of software modules for real-time analysis of nonverbal motoric affective and social interaction, of software modules for sonification and active listening to sound and music content, and for visual and haptic feedback, of an integration platform and of prototypes of serious games.

Brief description of technology

The weDRAW project is developing: - A library of software modules including novel techniques for (i) real-time analysis of expressive and affective nonverbal motoric behavior and (ii) for real-time analysis of social interaction (e.g., entrainment, leadership). - A hardware and software integration platform. - Three serious games to learn arithmetic and geometry from music and drawing in individual and social settings. Technologies need to be designed, developed, adapted, and customized for both typically developed and impaired children. A main objective for the weDRAW project is indeed to define and develop a multisensory embodied and enactive learning and teaching ecosystem that applies in the same way and provides the same opportunities to both typical and impaired children, thus breaking the barriers between them and fostering inclusion. Technologies the Consortium will make available to the artist include: - An integrated hardware and software platform, supporting multiple different input and output devices, and scalable to different learning environments (e.g., school and home), integrating the above-mentioned libraries and supporting the design and development of serious games. - A haptic robotic platform with audio-visual and tactile feedbacks. Consolidated standard platforms such as EyesWeb XMI and Unity are adopted in the project. The focus is on nonverbal communication.

What the project is looking to gain from the collaboration and what kind of artist would be suitable

We expect the artist in residence will: - Actively participate in the interaction design process of the serious games envisaged in the weDRAW (participatory design) by providing an alternative perspective with respect to those of teachers, children, and scientists. - Contribute to the design and development of the artistic content for the serious games and the demos the project will produce, by creating, in collaboration with the weDRAW partners and in the framework of the interaction design process mentioned above, the sound, music and/or visual content. - Participate in the public events the project will organize, by contributing to their design, to the demos that will be presented, and to foster the visibility of the project in the artistic community. - Participate in the evaluation of the project, of its outcomes, and of its prototypes, by providing a qualitative and quantitative feedback, under an artistic perspective. This will be complementary with respect to the formal quantitative evaluation the project will carried out at schools.

Resources available to the artist

The artist in residence will mainly work at the premises of the Casa Paganini – InfoMus research center at University of Genova (www.casapaganini.org). This is an ancient monumental building located in the historical center of Genova and endowed with a technological infrastructure. Besides education, Casa Paganini is also active in scientific and technological research in the area of performing arts and already hosted visiting artists. The artist will enjoy the facilities available at the center, including technical equipment (personal computers and workstations, advanced technologies for movement analysis, professional multichannel audio systems, professional video-projection systems, fast internet connection) and the staff of Casa Paganini – InfoMus will support him/her in the activities connected with the project (e.g., in exploiting at best the EyesWeb XMI platform).