How is it that information can be spatialized, and what does it mean? In other words, once proper tools are developed for interacting with information on the spatial level, what sorts of creative processes will arise? From the experimental engineering of Richard A. Bolt and his team at MIT in the 1970's to the artistic explorations of Marcos Novak and his architecture of virtuality, the questions of spatiality and spatial computing have opened up previously unknown modes of thinking and interacting.

Richard A Bolt

"Spatial Data - Management" (1979)

"We have attempted to create an interface which is not a tiny, narrow-band 'port-hole' into an information bank, that bank itself an abstractly addressed set of intangibles. Rather, we have attempted radically to recast the setting as an 'informational surround' wherein the user is directly engaged with data bodied forth in vision, sound, and touch, data inhabiting a spatially definite 'virtual' world that can be interactively explored and navigated." (Richard A Bolt)

One of the most influential voices in the field of multimodal interfaces, Richard A Bolt worked in the design of systems that enabled the user to interact with computers through natural gestures rather than simple input devices like the mouse and keyboard.

As a contributing member of the Architecture Machine Group at MIT, helped in the implimentation of new computer interfaces that reacted to voice, gesture, and gaze, among other things.

Worked in close collaboration with his mentor Nicholas Negroponte in the creation of the media room, also known as the "Put-That-There" System.

The two engineers together founded the Media Lab at MIT, heading its Advanced Human Interface Group.

Groundbreaking research resulted in the design of "the first hypermedia system to organize information for retrieval within a 3-D virtual environment" (Randall Packer and Ken Jordan 2001). Named the SDMS system (aka. Spatial Data-Management System), the idea was to incorporate "a multidimensional window into data" (Bolt 1979). In sharp contrast to the later work of Alan Kay and the GUI, Bolt and Negroponte were more interested in the developments of virtual reality as a commercially viable interface.

The results of their work brought them to consider the question of "how data might be presented as symbols within an engaging, multisensory, navigable environment" (Randall Packer and Ken Jordan 2001). The term Dataland was defined as the model for the information landscape they developed for SDMS, a place to organize and manipulate information spatially.

Accessing data by
"going to where it is"

Rather than "pulling data in" (Bolt 1979), which occurs by default in systems that attempt to organize information through hyperlinking (i.e., systems that reference by name or symbol), SDMS-based applications placed the user immediately within a constant tactile interactive environment. The user would then access data through manual gestures and movements, and by memory.

Perusing data was just as important as finding data. In order to completely interact with data objects within the system, the user would be forced to 'open up and use' the information, instead of the more common, and passive, representations afforded by common graphical user interfaces.

A famous greek poet and teacher of public speech and rhetoric, Simonides taught his students to memorize large poems and speeches through a technique later coined the "Simonides Effect". The theory and practice came from the understanding that humans seem to naturally organize information spatially in their imaginations. Through the creation of a mnemonic engine, a mental landscape where information was laid out spatially, the user could enter that space and pull out information stored in remote locations of an imaginary room. Thus, the student did not memorize the speech or poem per se, but instead memorized the path that they walked through the mental architectural space, and this was enough to fit the necessary pieces together into a complete work.

Bolt and Negroponte found that a familiar spatial layout is sometimes more important to the human mind than the exact contents of the objects in that space. Similar to the ideas of Simonides, Bolt realized that the retrieval of information in a virtual reality should mirror the way we organize our physical world, through the use of spatial relationships and motor memory.

Work such as that proposed by the engineers at MIT attempted to open up the limited channels of communication, by tapping into the the spatiality implicit in human symbolic retrieval.

Spatial Data Objects

- Book

Unlike the endlessly scrolling views of a computer monitor, the book as a virtual object would take on characteristics similar to its real-world counterpart. Such simple animations as 'open-up' and 'page-turning' would increase the functionality of the medium. Letting the user know "where you are" in the book, as well as the use of 'dog-ears' and 'tagging' increases the interactive feel of the data object.

- Transparent Ink

Hyperlinking text documents is one way to traverse information, but we are still ignoring the basic human impulses triggered in readers when they are reading documentation. The ability to annotate, comment, and append their own ideas is lost when their is no ink. Transparent and erasable ink solves this problem, and opens up an even deeper interaction with text documents and 'book' objects.

- Sound

Just as we are capable of searching and browsing through text and image catalogs, it is useful to mix sound in with phsical objects, so that we may be able to organize information through sonic cues as well. Speech and music localized and syncronized around images and text would invite the possibility for 'browsing' through sets of auditory messages.

Wired Magazine - many articles have been written for Wired by Nicholas Negroponte on multimodal interfaces
VR Interaction Techniques
Human - Computer Symbiote

Marcos Novak

"Liquid Architectures in Cyberspace" (1991)

"Liquid architecture is an architecture that breathes, pulses, leaps as one form and lands as another. Liquid architecture is an architecture whose form is contingent on the interests of the beholder; it is an architecture that opens to welcome me and closes to defend me; it is an architecture without doors and hallways, where the next room is always where I need it to be and what I need it to be." (Marcos Novak)

Marcos Novak doesn't see himself as an architect in the regular definition of the word, but rather redefines himself as a 'trans-architect', or one who designs spaces and the objects that fill and create those spaces directly in virtual reality. Whereas an architect is interested mainly in the economy of construction, the trans-architect explicitly invokes his space to perform and transform, as an immersive, iteractive, lively creation in 3-D virtual space.

He invented a set of conceptual tools for thinking about and constructing objects and territiories in cyberspace. Through the use of the computer and its ability to completely ignore the laws of physics and Euclidean Geometry, new forms can be created. Novak calls these liquid architectures, since they combine time and space with little or no rational constraints.

The results of his studies has led him to create previously unimaginable, unfathomable objects in virtual space, and then have them milled into real works of art that can be touched and experienced in the real world. Architectures that exist both 'here' in the physical world and 'there' in the virtual world, forever transverging the boundaries between the imagination and the possibility of creation.

What is Cyberspace?

As William Gibson put it is his novel All Tomorrow's Parties, we are now entering "the infinite plasticity fo the digital". Information is now completely under our control, and we can exist in it as well as beside it.

Novak defines cyberspace as a 'spatialized' visualization of all information. This is a place where information is not looked at, but 'entered'. It is not a place that exists out in the world, but within our own minds, and it is the result of our choice that we enter it and modify it.

By modifying the normal mapping of data to representation, we can explore unknown realities that effectively place the body into landscapes previously limited to the mind.

"...cyberspace, in its literal placement of the body in spaces invented entirely by the mind, is located directly upon this blurring boundary, this fault" (Novak 1991)

Unimaginable Complexity

In this age of so many new media forms, where video, audio, haptic, and other sensory systems are being developed in tandem, it is the job of the trans-architect to mix these forms together. To take the results of their mixtures within the realm of virtual space, and then bring them into the real world in some new form, as embodied fiction.

"Magic, or the promise of control over the world by the will alone" (Novak 1991). Even as magicians have attempted to bring the world of fascination into the world of true reality, technology has been, and will continue to be, the key to true magical creations. Mixed media is an emergent form and is only bound by the human imagination.

Imagining Navigable Virtualities

The human is on his way to becoming a new form, one very different than any other. Novak shows symbolically in his own work that the time we have entered is different from any other time in history. We are now altering the very foundations of our existence, and have to make choices as to what sorts of realities we will create for ourselves.

Allogenesis, or the production of the alien, is our ongoing struggle to surpass the very limitations of our own bodies. Just as media forms have taken on liquidity, so have our minds.

We are now free to explore the spaces of our minds, to navigate and control space, and to orchestrate and compose time, like music.

"Navigable music is not an organization of sounds in time, it is the organization of a matrix of sonic, visual, behavioral, and other possibilities. Actions within that matrix may contain every aspect of conventional music, because what is experienced within this landscape depends entirely upon the user's individually selected, unforeseen, interest-driven trajectory. If I prefer a beat, I remain within the part of the landscape where I first encountered a certain rhythmic pattern. If I leave a phrase, I can always return to it. I can choose extreme monotony, by remaining in one place, or extreme variety, never returning to the same place." (Novak 1995)

As a music composer, I desire the mixing of digital sound synthesis and physical forms in the creation of new works of art. Thus, I propose the development of an environment for composing music through the use of the human body as it interacts with objects in a 3-D virtual space. This space reciprocally interfaces with the human body, engaging the user in a dialog that can be recorded, mixed, and performed.

Space Clay

This object is physical, in the sense that it can be manipulated by the human hands, but it is not necessarily tied to the laws of physics. It can be stretched, pulled, squeezed, twisted, ripped, squished, etc..., all under absolute control by the composer. The choice of sonic parameters is set by the user before each session, and then is heard in real-time as the clay is 'acted upon'.

Space Dust

In a sense, this is the opposite of Space Clay. Instead of interfacing with an object directly out in virtual space, the body is surrounded by virtual air molecules. Sound synthesis parameters are set up the same way, but in this case the user interface is his own body. The space around the body is then set to be the composer of the music, as the body attempts to control the radiation of sound moving, colliding, refracting, defracting, and diffusing through space.