INSIDE OUT 2.0

 

 

Inside Out 2.0 is a research project based on the exploration of a musician's emotions while performing a musical composition. This is a performance in which the individuality of the musicians becomes an equal part of the piece, creating a unique audio-visual composition.  It is an attempt to open the listener's consciousness to other connections than the ones that normally arise.

 

Inside Out 2.0 develops a sensory system for understanding classical music, offering a way to visually interpret emotions within a musical performance, based on the introduction of modern technologies. How does a musical performance change, revealing new ways of interacting with the audience? Can the listener understand music not only through his individual perception system, but also read the perception of music by the performer himself?

 

Our team explores contemporary media technologies in music performance. The musicians are fitted with special neurosensors as they perform. Information about the electrical activity of the musicians' brains is visualized and classified by neural networks into a dynamic digital canvas. The Database for Emotion Recognition System Based on EEG Signals and Various Computer Games (GAMEEMO) 1 is used as a dataset for the development of emotion recognition system which has a direct affect to a visual part. 

 

Three digital algorithms are involved in the creation of the visual image: a pre-trained StyleGan2 ADA model of generative-adversarial network 3, an algorithm that synchronized a sequence of visuals that GAN generates to music, and an adaptive algorithm that reads real-time data from the neurosensors, classy them and translates it into a reactive system of patterns, colour codes and noise dissonances that change the pre-trained image in real time.

 

To create a visual narrative we use large datasets of artistic works2of different periods and styles, correlating them with the music of same epochs. Music of the early 20th century can be seen through the works of Expressionism, Rayonism and Poyntellism. Musical compositions of mid-20th century are expressedwith the works of the Futurists, the Post-Impressionists and the Surrealists. The second half of the century reveals abstraction and surrealism, reflecting on the works of Marc Rothko and Salvador Dalí.The end of the century presents the interpretations onPop Art, Minimalism, Feminism and Conceptual Art, Neo-Expressionism, ArtePovera, revealing interpretations on Jean Michel Basquiat, Roy Lichteinstein, Andy Warhol, and many others.

 

The music programme is based on the XXth century including pieces by Kluzner, Ysaÿe, Pärt, Desyatnikov, Sollima, Bartok and Yevlakhov. Every work that we have in the program has internal emotional contrasts. Such contrasts during performance are always difficult to perform, due to the fact that you need quickly to switch to another state.The piece played is unique each time, revealing the individuality and community of both musicians and listeners.

 

During a performance we can watch live how the emotional background of the musicians changes as they play their instruments. The emotions of each musician influence the visuals, creating an overall visual composition. The musical combinations create visual coherence.

 

Authors:

Egor Zvezdin, Tatiana Zobnina, Morgan Korolkov, Maria Kuptsova, Marina Muzyka, Ilya Domnins

Performers:

Natalia Uchitel, Ilya Izmaylov, Anita Ozheshkovskaya, Dmitry Stopichev

Producer:

ArtMix (Egor Zvezdin and Margo Bor)

 

 

 

 

1 Alakus, Talha; Gonen, Murat; Turkoglu, Ibrahim  (2020), “Database for Emotion

Recognition System Based on EEG Signals and Various Computer Games - GAMEEMO”, Mendeley Data, V1, doi: 10.17632/b3pn4kwpmn.

2https://www.wikiart.org/

3 Original code by Tatyana Zobnina based on Lucid Sonic Dreams and styleGAN NVIDIA Labs