Cortical

Fusion

Background

When senses fuse

Cortical Fusion is an artistic commentary on a neurological phenomenon known as "synesthesia" or the blending of the senses. Certain individuals, known as "synesthetes", experience an exaggerated version of this effect, but studies have pointed to the fact that each of us possess some degree of cross sensory associations. Examples of this can be seen all over our descriptive language, which is why we understand what it means for cheese to taste "sharp", how someone can be "green with envy", what a "loud" color is, and what it means to give a "warm" welcome.

Does this sound sharp to you?

The "Bouba/Kiki" effect as described by the German psychologist Wolfgang Kohler further demonstrates that these associations are a common feature of human perception. Kohler would show people a set of two shapes- one jagged and pointy, the other rounded and smooth. He would then ask the subjects to identify which one of the shapes was known as "Bouba" and which one as "Kiki". Amazingly, over 95% of people agree that "Kiki" is the sharp shape, and "Bouba" is the rounded one. This test has been repeated all over the world to similar results, suggesting a universal correlation between the seemingly unrelated senses of sight, sound, and texture.

Synesthesia in the arts

Artists have made reference to these sensory pairings since the birth of art itself. Poetry for instance, often relies heavily on metaphors and associations made between different emotions and senses. Similarly, the ancient genre of musical drama can be seen as an attempt to link sight and sound in order to convey feeling.

Sight, sound, movement, taste and smell

In 1903 the Russian composer and synesthete Alexander Scriabin began sketches for a multimedia composition entitled "Mysterium", perhaps the most historically elaborate endeavor of this type. It was supposed take place as a week long celebration at the foot of a mountain with different musical pieces corresponding to different times of day. Specific incenses would be burned at different moments in order to incorporate the sense of smell. Similarly, certain foods would flavor parts of the piece and each member of the audience would take part in the performance during sections of dance. Sadly, this work remained unrealized at the time of Scriabin's death in 1915.

A more familiar example

Closer to our own time, the Disney movie "Fantasia" is a familiar example of synesthetic art: figures on the screen are used to create a visual counterpart to a musical accompaniment. Sonic and visual motion are correlated in a way that is instinctively logical and appealing.

Taking the next step

Cortical Fusion sets out to expand on this theme, working tactile perception into the equation and literally immersing our audience members into a three-dimensional synesthetic environment, which records and reacts to people's actions within it.

Art and Architecture

The Philips Pavilion at the 1958 World's Fair is a great milestone in this genre of multimedia architecture. During the exhibition, audience members were led inside a large hall in which looping video projections and sculptures were viewed alongside musical compositions by Iannis Xenakis and Edgard Varese. They were then allowed to walk around in order to observe different moments of lighting and acoustics within the space. Likewise, the art of Cortical Fusion is deeply reliant on the viewer and his or her path within it. Attempting to demonstrate and create new relationships between sound, light, and texture, each experience of this work will be affected by which areas of perception our audience member chooses to focus on, and how he or she reacts to what is presented.

Borrowing from the past and present

Influence for individual sections within the work come various sources. Sonically, the complex, slow, drone-like installation music of La Monte Young provided a valuable framework- especially since this type of almost static music is able to serve as a controlled variable through which to communicate with the other senses. Visually, the the light constructions of James Turrell and Olafur Elliason were a model for the way in which we wanted to place the audience inside of an unfamiliar, yet powerful optical environment. Texturally, Ernest Neto's work with foamlike sculptures influenced our design for our interactive movable architecture.

Studying your every move

Addressing the contemporary interest in surveillance and data visualization, viewers who enter the piece are tracked and recorded by the system. This data is collected, filtered, and analyzed in order to better understand the way in which we form associations between the different senses with which we perceive the world. The work of Scott Snibbe is a great example of how collected input from the audience members themselves can be recycled into the art itself. His 2004 piece "You Are Here" literally traces paths of audience members walking through the exhibition. At a specific location, he has set up a screen in which they can see this movement drawn on an overhead view, teaching audience members about movement, public space, cultural norms, and their individual path within this framework.

Making connections, drawing conclusions

In Cortical Fusion, we trace individual and group movements in order to establish connections between specific stimuli. In doing this, we hope to have formed an interesting addition to the long line of artistic pieces directly confronting and working within the blurred modes of perception with which we humans view the world. Additionally, from this mass of information, we hope to discover yet unknown correlations between stimuli.