SMART MONTAGE: The New Mobile Dialectic
Introduction
The convergence made possible through the Internet and “new media” has been widely talked and theorized about. Yet, the convergence of visual material possible through the semantic web,[1] is altering the grammatical structure of how we view and absorb visual information. The semantic web has now expanded to the iPad, Smart Phone, and Web TV where users can access the same media using different devices at different times. The implications are only now rising to the surface but it appears that not just the structure or narrative grammar is expanding, but the nature of storytelling itself.
I’m choosing to speak about one segment of this vast expansion, how these convergent forms create a new 21st century visual grammar or syntax. For the purposes of contextualizing what I’m talking about, I’m going to coin the grammatical phenomenon smart montage. In coining this term, I am quite consciously being a bit cheeky. I am aware that the development as well as the term may soon be dated, but I expect this, as smart montage may be just a step in the evolving new media language.
Smart Montage is a new-media visual language development, building off of, and integrating, a number of existing concepts and technologies such as video mashups, cluster video, intellectual montage and spatial montage and can be output to both traditional devices, web and mobile devices utilizing the latest semantic[2] technologies. Many of these developments are not new, but rather have been bubbling up from media experiments, such as Eisenstein’s attempts at an intellectual montage, since the beginning of cinema. Digital editing technology and the semantic web, however, have created a moment where all these techniques can converge to push the language of editing into smarter, more mobile forms.
Defining the Visual Dialectic
To understand the possibilities of smart montage, it is critical to look to the past and to the birth of the visual dialectic. Eisenstein saw montage as a dialectic, which could serve not just a narrative purpose, but a wider Marxist dialect.
In the realm of art this dialectical principle of the dynamic is embodied in CONFLICT as the essential basic principle of the existence of every work of art and every form. FOR ART IS ALWAYS IN CONFLICT: 1. because of its social mission. 2. because of its nature, 3. because of its methodology.[3]
Eisenstein shifted from his contemporaries, Pudovkin, Kuleshov, and Vertov in defining montage as “not an idea composed of successive shots stuck together but an idea that DERIVES from collision between two shots that are independent with one another (the ‘dramatic’ principle).”[4]
Intellectual Montage
Eisenstein became less interested in the shock and drama, which he so adeptly crafted, and more interested in how to communicate ideas through a visual language. He developed what he labeled, “intellectual montage” which was a new visual syntax for relaying Marxist theory.
A purely intellectual film, which freed from traditional limitations, will achieve direct forms for thoughts, systems and concepts without transitions or paraphrases. And which therefore becomes a SYNTHESIS OF ART AND SCIENCE.[5]
Eisenstein’s attempt at intellectual montage is most applicable in discussing contemporary techniques in spatial montage. Spatial montage is the technique of placing multiple images on the screen at the same time as seen on a TV show such as 24 and can be a technique that incorporates the failed attempts of Eisenstein as part of a smarter montage. Eisenstein through intellectual montage attempted to use film “to develop and direct the entire thought process.”[6] October was Eisenstein’s attempt to apply his theoretical principles regarding film as an intellectual dialectic but, however, audiences found the experiment almost unreadable. As Murray Sperber points out in his article in “Jump Cut” on Eisenstein’s October, most critics were unable to appreciate Eisenstein’s attempt at a new cinematic form.[7]From a physicality standpoint, Eisenstein’s intense use of symbolic imagery to push a political ideology either appeared too obvious or he veered too far away from the emotional and shock cinema which he was so adept at. His epic, October created a semiotic statement, rather than a cohesive fused narrative about the 10 Days that Shook the World.
Extensions of Intellectual Montage: Smart Montage
Let’s fast forward for a moment to contemporary film practice. For most of cinematic narrative history we embraced the importance of emotion within the narrative construct; exactly what Eisenstein stated was effective about cinema. Yet cinema did not embrace or develop intellectual montage beyond Eisenstein’s own experiments.
Film language, however now has a possibility of living in a shared temporal and spatial frame of video commonly known as spatial montage. Spatial Montage was a term made popular by new media theorist Lev Manovich and is seen in examples such as the television series ‘24’ or in a hybrid media forms such as a CNN broadcast which fuses multiple streams of video, text, photography and graphics working with sound which may or may not be sampled. Multimedia, installation, and experimental artists have long used spatial montage, however the technical ease in how the techniques may be applied with present day technology is what has made them explode into mainstream use. By using spatial montage, media makers can create a new smart montage that creates visual intellectual statements using symbols and ironic commentary while still keeping the audience engaged within a narrative. Smart montage is a culmination of editing techniques that uses new technologies to successfully resurrect previous montage theories, such as Eisenstein’s Intellectual Montage.
The Question of Visual Dissonance and Narratives
The smart montage phenomenon can be used to create new narrative forms as well as a visual dissonance that is not dissimilar to the narrative dissonance aesthetic.[8] At moments the mash ups, sampling or spatial montage form itself creates an emotional reaction as a sum of all images meant to be read as a discontinuous whole.
Early Examples of Smart Montage: Manovich’s Soft Cinema
Manovich discusses in his essay Database as a Symbolic Form how the new media narrative can make use of elements, which are entirely organized by a software database, and construct narratives using random algorithms. The result of this discussion was the ‘Soft Cinema’ project, which originated via computer and released on DVD in 2005. Manovich’s primary focus of his co-authored Soft Cinema piece was how traditional cinema can be altered through the use of a computer database. He states, “The Soft Cinema project is interested not in the digital computer per se, but rather in the new structures of production and consumption enabled by computing.”[9]
This means that the three film narratives, which they construct, and can be seen on DVD (although it originated in kiosk form), has film narratives which are based on traditional genres, however by virtue of a database can “construct a potentially unlimited number of different films.”[10] In the Soft Cinema visual framework Manovich and his collaborator Andreas Kratky follow what they describe as “the standard convention of the human-computer interface, the display area is always divided into multiple frames.”[11]
From an intellectual montage standpoint, what is significant is how even with a database driven narrative there can be a dialectic between the various frames viewed on the screen at the same time and connections are made even if by random. This provides the groundwork for further pushing this medium into the realm of smart montage.
New Narrative Experiments
Bruce McDonald’s spatial montage narrative, Tracey Fragments, exploits spatial montage to tell the story of a young teenager’s search for her nine year old brother. Based on the book and screenplay of Maureen Medved, the story is told in flashback and exploits spatial montage to show Tracey’s fragmented identity and emotional state. Most of the film uses the technique to show parallel action, yet through its use of symbolic imagery within the scene we gain a larger intellectual and emotional sense of Tracey’s state of mind and in doing so there are moments of smart montage. What is intriguing about this experiment is that through spatial montage, McDonald attempts to reflect the fragmentation that is part of the narrative style in Medved’s novel.
Ultimately, it’s a toss up whether the fragmented narrative successfully sustains the audience through its full length. But, it is one of the boldest narrative experiments to date. Tracey Fragments also opens up the question of eye tracking within spatial montage. The sizes of the images within the screen themselves as well as their position in the screen can help point the audience to the significant moments and what the eyes should focus on while the background creates a fragmented dissonance and noise.
Music Videos
Music videos for decades have driven innovations in editing form from pacing to associational edits. Ken Dancyger in his book The Technique of Film and Video Editing discusses how the music video influenced the downgrading of plot and time and place shifting.[12] Music videos created a disjunctive editing style that “obliterates” time and space and focuses on feeling states.
To counteract the impulse to organize those images and sounds into the narrative that may not be present, the filmmaker must challenge the impulse more deeply. She must undermine the sense of time and space in the MTV-style film or video.[13]
And yet Dancyger points out that these notions are already reflected in the work of Kurosawa, Resnais, Fellini and Wenders.[14] In fact these tenets originated in Soviet montage and in particular intellectual montage.
Pet Shop Boys video Integral[15] makes use of a stop motion camera capturing the rapid pace of everyday life and juxtaposes it with a graphically pixelated images to reinforce the association themes of the song as well as the artists themselves. Integral pushes spatial montage further with fast edits further expanding viewers ability to absorb simultaneous multiple visuals and messages and create a wider opening for smart montage. Furthermore, through embedding QR codes, smart montage can provide users with additional data and interactivity.
Eclectic Method[16] is a “vj” group that has gained popularity for its immediate cultural remixes in clubs in New York, Los Angeles and London. They have also worked to remix popular media such as the Tarantino MixTape[17], which utilizes spatial montage, repetitive action and music, to juxtapose scenes out of multiple Tarantino movies into a cohesive and yet disjointed whole.
After the passing of Michael Jackson, Studio Brussel[18] created a site to pay tribute and eternalize Michael Jackson’s moonwalk. They asked users to capture their own moonwalk and upload it to the website, “Eternalmoonwalk.com.”[19] The site samples all of the clips to create a multi-frame or spatial montage of users “moon walk.” This site opens up the question of interactivity within the narrative and how this one action can change according to time and place.
Artist Fredo Viola of Woodstock, New York has gained popular recognition through his music that combines and layers multiple tracks of his singing. He launched a website last year, TheTurn.TV[20] that uploads his music videos or what he calls cluster video[21] revealing the visual multi-frame process and spatial montage of how he produces his multimedia compositions. Viola intentionally works with multiple frames in his work as a result of his experience as an editor and motion graphics artists.[22] Sad Song, the song that brought him recognition can be seen as a cluster video of 15-second videos that are composited in After Effects.
Rather than creating a kind of visual and aural dissonance, Fredo uses smart montage to create a harmonic visual and aural consonance that brings us into a rich multidimensional emotional experience.
Found Footage and Video Mashups
Found Footage filmmakers remixed cinematic cultural symbols a long time before the web yet the access to such footage was always limited and guerilla style at best. Media ready for recycling is dumped everywhere on the web. And a new media and found footage hybrid has developed through the evolution of the video mashup. A video mashup uses multiple source materials to create new and powerful social or intellectual statements using digital video, which is often then posted on the web. Influenced by both the concept of the software mashups and found footage filmmaking, media makers have created a new form of storytelling through user generated content. Many of these mashups have a social implication or message and make use of spatial montage to do it. Furthermore because of new media tools, the general public is invited to participate in a remix culture on the web. For instance remixamerica.org[23] and totalrecut.org[24] provide users materials, instructions and how to videos on how to create their own mash up.
Alice in Wonderland Remix
The website Moving Web now includes a category on spatial montage and multi-frame work[25] that tracks this growing form of storytelling. One example is the Alice in Wonderland Remix[26], which samples the story of Alice using spatial montage.
By keeping a consistent soundtrack and cutting on beats, we gain emotional dissonance and entirely new side to Alice stuck inside the rabbit hole.
Wreck & Salvage Collective
The Wreck and Salvage Collective describe themselves as “a collective of three like-minded artists producing original video content for the Internet. We share a love of rust and rotting wood, postcard landscapes and decaying industry.”[27] They publish several series of mashups via their website and a channel on vimeo. What is notable about their work is that they make significant use of spatial montage, which can be seen on both the web and mobile phone so the spatial montage technique is not limited by size or platform. Their work samples commercials, original video, motion graphics, and “other forms of new and old media technology.”[28]
Call and Response 100 Farewells
The, short Call and Response[29], uses spatial montage and repetition to define “the fine line between catchy commercial jingles and mental paralysis.” The piece makes use of smart montage to create an intellectual message about the hypnotic effect of advertising. In Wreck and Salvage’s video-goodbye to former president George Bush, in 100 Farewells[30], they repeat the moment when an Iraqi journalist tosses the infamous shoe and use spatial montage to reinforce the repetition of the action. This use of repetition is reminiscent of Soviet montage. Both shorts reflect a smarter use of montage to create hybrid social statements through their use of spatial montage and the simultaneous editing of images on the screen at the same time.
Conclusion
Spatial montage and emerging new media genres open up the possibility of creating the intellectual montage hybrid of smart montage, which Eisenstein originated.
Moving beyond linear narratives, new media artists can create a smart montage, which embraces the principles of intellectual montage while creating a possibility of simultaneously preserving the narrative and emotional engagement of the viewers. Artists use symbols to reinforce the intellectual message while placing it within a narrative by having the images appear simultaneously in a spatial montage format. This new form is in its infancy, but the possibilities and successes may alter the future of narratives and the semantic visual form.
Bibliography:
Dancyger, Ken. The Technique of Film and Video Editing: Fourth Edition Burlington, MA , Oxford, UK: Focal Press, 2007.
Eisenstein, Sergei.“The Dramaturgy of Film Form.” In Writings, 1922-1934. 1st ed. Edited by Richard Taylor. Bloomington and Indianapolis: Indiana University Press, 1988.
Manovich, Lev. Soft Cinema: Navigating the Database (booklet) Cambridge, MA: The MIT Press, 2005.
Meadows, Mark Stephen. Pause and Effect: The Art of Interactive Narrative. Berkeley, CA: New Riders Press, 2003.
Sperber, Murray. “Eisenstein’s October.” Jump Cut: A Review of Contemporary Media, March 1977, 15-22.
Weblinks:
Alice in Wonderland Remix. From yooouuutuuube.com, http://www.yooouuutuuube.com/v/?rows=18&cols=18&id=pAwR6w2TgxY&startZoom=1 (accessed March 16, 2010)
Eclectic Method. http://www.eclecticmethod.net/ (accessed March 16, 2010).
Eclectic Method. The Tarantino Mixtape. 7 min., 16 sec.; online video. From YouTube, http://www.youtube.com/watch?v=uEIPCOwY4DE (accessed April 16, 2010).
Moving Web. Archive for the ‘Spatial Montage (Multi Frame)’ Category. http://www.movingweb.org/category/spatial-montage-multi-frame/ (accessed March 16, 2010).
Remix America <http://remixamerica.org>
Studio Brussel. Eternal Moonwalk. http://www.eternalmoonwalk.com (accessed March 16, 2010).
Total Recut. Total Recut Homepage. http://www.totalrecut.com (accessed March 16 2010).
Viola, Fredo. Fredo Viola Bio. Because Music. http://www.because.tv/en/artists/fredo-viola/index.php/ (accessed March 16, 2010).
Viola, Fredo. The Turn. http://www.theturn.tv/ (accessed March 16, 2010).
W3C. W3C Semantic Web Activity. W3C Semantic Web. http://www.w3.org/2001/sw/ (accessed March 16, 2010).
Wreck and Salvage Collective. 100 Farewells. From Youtube.com, http://www.youtube.com/watch?v=hafjz0eWLBw (accessed March 16, 2010).
Wreck and Salvage Collective. Call and Response. http://wreckandsalvage.com/pure-salvage/call-and-response-2/ (accessed March 16, 2010).
Wreck and Salvage Collective. Wreck & Salvage: About Us. http://wreckandsalvage.com/about/ (accessed March 16 2010).
[1] The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries, as defined according to the W3C. “W3C Semantic Web Activity.” W3C Semantic Web. http://www.w3.org/2001/sw/ (accessed March 16, 2010).
[2] Also referred to as Web 3.0
[3] Sergei Eisenstein, “The Dramaturgy of Film Form,” in Writings, 1922-1934, 1st ed., ed. Richard Taylor (Bloomington and Indianapolis: Indiana University Press, 1988), 161.
[4] Sergei Eisenstein, “The Dramaturgy of Film Form,” in Writings, 1922-1934, 1st ed., ed. Richard Taylor (Bloomington and Indianapolis: Indiana University Press, 1988), 163.
[5] Sergei Eisenstein, “The Dramaturgy of Film Form,” in Writings, 1922-1934, 1st ed., ed. Richard Taylor (Bloomington and Indianapolis: Indiana University Press, 1988), 180.
[6] Sergei Eisenstein, “The Dramaturgy of Film Form,” in Writings, 1922-1934, 1st ed., ed. Richard Taylor (Bloomington and Indianapolis: Indiana University Press, 1988), 180.
[7] Murray Sperber, “Eisenstein’s October,” Jump Cut: A Review of Contemporary Media, March 1977, 15-
[8] Daniel C. Melnick has an interesting discussion of this in his book, Fullness of Dissonance: Modern Fiction and the Aesthetics of Music, 1994.
[9] Lev Manovich, Soft Cinema: Navigating the Database booklet (Cambridge, MA: The MIT Press, 2005), 40.
[10] Lev Manovich, Soft Cinema: Navigating the Database booklet (Cambridge, MA: The MIT Press, 2005), 40.
[11] Lev Manovich, Soft Cinema: Navigating the Database booklet (Cambridge, MA: The MIT Press, 2005), 40.
[12] Ken Dancyger, The Technique of Film and Video Editing: Fourth Edition (Burlington, MA , Oxford, UK: Focal Press, 2007), 188-189.
[13] Ken Dancyger, The Technique of Film and Video Editing: Fourth Edition (Burlington, MA , Oxford, UK: Focal Press, 2007), 189.
[14] Ken Dancyger, The Technique of Film and Video Editing: Fourth Edition (Burlington, MA , Oxford, UK: Focal Press, 2007), 189.
[15] http://www.youtube.com/watch?v=b7TSDUHhPIw
[16] Eclectic Method. http://www.eclecticmethod.net/ (accessed March 16, 2010).
[17] Eclectic Method. The Tarantino Mixtape, 7 min., 16 sec.; online video; from YouTube, http://www.youtube.com/watch?v=uEIPCOwY4DE (accessed March 16, 2010).
[18] Studio Brussel
[19] Studio Brussel. “Eternal Moonwalk.” http://www.eternalmoonwalk.com (accessed March 16, 2010).
[20] Fredo Viola. “The Turn.” http://www.theturn.tv/ (accessed March 16, 2010).
[21] Fredo Viola. “Fredo Viola Bio.” Because Music. http://www.because.tv/en/artists/fredo-viola/index.php/ (accessed March 16, 2010).
[22] Fredo Viola. “Fredo Viola Bio.” Because Music. http://www.because.tv/en/artists/fredo-viola/index.php/ (accessed March 16, 2010).
[24] Total Recut. “Total Recut Homepage.” http://www.totalrecut.com (accessed March 16 2010).
[25] Moving Web. “Archive for the ‘Spatial Montage (Multi Frame)’ Category.” http://www.movingweb.org/category/spatial-montage-multi-frame/ (accessed March 16, 2010).
[26] “Alice in Wonderland Remix.” From yooouuutuuube.com,
http://www.yooouuutuuube.com/v/?rows=18&cols=18&id=pAwR6w2TgxY&startZoom=1 (accessed March 16, 2010).
[27] Wreck and Salvage Collective. “Wreck & Salvage: About Us.” http://wreckandsalvage.com/about/ (accessed March 16 2010).
[28] Wreck and Salvage Collective. “Wreck & Salvage: About Us.” http://wreckandsalvage.com/about/ (accessed March 16 2010).
[29] Wreck and Salvage Collective. “Call and Response.” http://wreckandsalvage.com/pure-salvage/call-and-response-2/ (accessed March 16, 2010).
[30] Wreck and Salvage Collective. “100 Farewells.” From Youtube.com, http://www.youtube.com/watch?v=hafjz0eWLBw (accessed March 16, 2010).
