Tech Project

GOPROSOCIAL
BrainHuman-machine interactionNeurocinema
Open Call
In Residency
Residency Outcome
An interactive neurocinematic system that uses media for assessing and training emotional dynamics
From Jan. 1, 2017 to Dec. 31, 2020
  •   A student project from Neurotheatre EID summer school 2018, image by Svetlana Degtyareva
    Credits: A student project from Neurotheatre EID summer school 2018, image by Svetlana Degtyareva
  •   Neurocinema architecture, image by Aleksander Valjamae
    Credits: Neurocinema architecture, image by Aleksander Valjamae
  •   DEMULTIPLEXIA performance in Brussels, image by Erik Norkroos
    Credits: DEMULTIPLEXIA performance in Brussels, image by Erik Norkroos

Description of the challenges faced by the Tech Project

The driving research hypothesis is provided by Kahneman’s dual process model of fast vs. slow decision making processes that have different neural correlates thus allowing their separate monitoring and manipulation. Our research questions are: how distinct are these neural correlates? What are their temporal patterns? Can these be reliably monitored in real-time? Can training change fast vs. slow decision making dynamics? Can these effects be long lasting? Specific scientific objectives include 1) real-time monitoring of emotional dynamics, individually or in groups (collective brain, hyperscanning), 2) updating neurocinematic system narrative engine, 3) testing the possibility of changing/disrupting prosocial behavior using it.

Brief description of technology

In an interactive neurocinematic system changes in the psychophysiological reactions of participants (enactors) that are assumed to represent implicit and unconscious reactions of the mind, determine the changes made to the narrative presentation of media in real-time. GOPROSOCIAL will follow database cinema approach (Manovich, 1998) creating a novel audio-visual content to explore emotional narratives for diagnostics of and, ultimately, changes in emotional processing. We will use wireless EEG systems (2 gtec Nautilus systems) and peripheral physiology (EDA, ECG - bitalino) signals to monitor users emotional states. Eye-tracking technology (Tobii Pro Glasses) could be used to determine most effective visual features of videos. Multisensory media effectiveness, narrative structures, individual differences in physiological responses are all open questions ready for artistic exploration. Latest neurocinema development was part of the neurotheatre performance at Bozar Electronic Arts Festival, BEAF'17 in Brussels (http://www.bozar.be/en/activities/129199-neurotheatre-collective-demultiplexia)

What the project is looking to gain from the collaboration and what kind of artist would be suitable

The project is looking for media artists, theatre directors or filmmakers who will experiment with the concept of neurocinema/neuroart. We expect our residents to have technical skills, especially in processing audio-visual media, filmmaking/editing, and/or interactive music/video, algorithmic composing etc. We are interested in exploring different emotional narrative structures within installations, experimental cinema formats, theatre performances etc. We are aiming for a high-impact performance/installation to be shown at a major media art venue or a festival. This should involve audience collective experience that will be interactively driving the dynamics of the performance/film/installation. This work should allow envisioning and reflection on technology-infused futures, questioning the societal and ethical implications of soon-to-be-prevalent technologies such as AI, brain-to-brain communication; and technology-enabled scenarios such as collective minds and decision making.

Resources available to the artist

HCI@TLU group is located in a brand new building in the centre of Tallinn. HCI@TLU has a number of labs. Interaction Design Laboratory is equipped with state of the art hardware and software and has numerous types of input devices. User Experience evaluation lab has hardware and software (a number of wearable EEG systems – g.Nautilus, Tobii Pro Glasses 2 wearable eyetracker, various physiology sensors, behaviour analysis software) and a comprehensive set of alternative input-output devices (Phidgets, Arduino, Sifteo, OpenBCI). Wide network of contacts with local artists groups (e.g., www.mimproject.org) and theatres (NO99, Vaba Lava) will enforce artist residence work.