Mobile Audience Participation with Jonah Bokaer and the Ferst Center

Audience interaction has become a pretty major trend this year and arts organizations around the country are experimenting with different ways to engage their audiences. But how does one go about making their experience a participatory one? Artist Jonah Bokaer and the Ferst Center for the Arts at Georgia Tech decided to approach the issue using mobile technology. Jonah Bokaer is an award-winning choreographer and media artist participating in the Ferst Center's first ever year-long dance residency program, ARTech. Jonah worked closely with students from the Georgia Tech Music Technology Program to develop MassMobile, a smartphone app that acts as an interactive platform for audience members to participate by affecting the stage in different ways. On April 2nd, MassMobile will premier with Jonah's new work FILTER at the Ferst Center.

I had to opportunity to speak with Jonah as well as Stephen Garrett, one of the graduate students on the MassMobile development team, and dancer Adam Weinert, who can be seen in the sneak peek video at the end of this post.

Where did the idea to include the audience in the performance, through a mobile app, come from?

Jonah - Well, in 2004 I made a work called RSVP, which planted 12 cellular devices in the audience. We used those devices and their ringtones to create the music for the piece. And that was engineered from offstage by the composer. But, when doing some initial research down here and planning this production called FILTER, I wanted to put forward the idea or the proposal that there could be interaction or intervention even, with the show.  And maybe Stephen can take it from here, but we spoke with Jason Freeman and the Ferst Center and this relationship started.

Stephen – Yeah, and I think Jason’s idea, who is my advising professor on my master’s project, was to create a system that allowed for a rapid deployment of audience interaction devices through peoples’ mobile phones. So the idea came from Jason through some previous mobile development work and with several other projects, to create an iPhone and android application that will allow any type of input that the phones will allow. Whether that be an accelerometer, motion data, text messaging, touch data, drawing…anything that the phone will accept, to try and incorporate that into the performances in some way.

What are some of the direct ways that the audience is affecting the performance through MassMobile? Does it affect the lighting or are there sound elements that change?

Jonah - Well, one thing I just want to clarify is that this production will premier in it’s full form here in Atlanta on April 2nd. So we’ve had a six or seven day production period in Avignon France and we premiered the choreography, but MassMobile and these technological components have not yet been unveiled and actually this week we are holding a workshop with invited public to interact with it further.

So in terms of how the audience affects the performance? We’ve identified four ways for potential interaction. In particular I wrote in one section of the performance which is very much open-ended and is a 9 minute section which can basically be reconfigured a bit.  Specifically the lighting is influenced. We worked closely with Aaron Copp, who is my lighting designer, to integrate that into the performance.

Stephen – And certainly MassMobile itself, the way that the app can interact with a piece, is not set in stone. It’s a creative conversation that is always happening. But specifically for this piece, for one of the sections that involves interaction during the pre-show, the audience is quite literally plugged into the lighting board through their app.

The actual technical details of the connection is that the mobile app is connected through a client server relationship to one of our servers here at Georgia Tech. Then there is a laptop that sits in the lighting booth that constantly pulls for information. Based on what items people have selected in the app, the laptop is connected directly to the lighting board and will instantly turn lights on and off. You can change color palettes and things like that pretty much instantly.

Did you build MassMobile specifically for this performance or is this a platform that could be used for other performances/performers?

Jonah – I would almost say that it is still in progress. But, you know, Aaron Copp and I have known each other since 1999 and have been building work together for a long time and often in our creative discussions we would say, “Is there some way to interface or interact?” As opposed to working with just video or with sound, lighting is so integral to these performances that MassMobile seems like a natural fit. I think that for light and choreography I am working in very particular ways. For larger applications of MassMobile I know there are many options.

Stephen – So MassMobile itself is the platform and the app, and is still in pretty active development. Working with Jonah and his FILTER has been a great way to beta-test the app and see it grow to more of its current capacity. My professor Jason Freeman already had the plan to use the application in a live music notation work with saxophone where the audience could directly affect the notes that the performer is seeing onscreen. So there are other uses for MassMobile in the arts imaginable.

Could you speak to the importance, for you, of having participation from the audience?

Jonah – Well, I should say that my degree is not in choreography, it is in media and visual arts. For my thesis I focused on the myth of interactivity and focused on a lot of mid to late video work in the 1980’s and 1990’s. My sense was that there is a certain parallel between what we could call this myth with interactivity with video, but also with performance.  Just because it’s time based, it’s not necessarily interactive. This is one area of focus of mine in terms of creating performances.

I guess the impulse to say that an audience could interact, is still pretty much in progress. So far, at least in this stage in the piece, we’ve only had pre-show interaction. It’s kind of an open question.

I think some major conventions of theater are called into question, for example having the phones on during the show, the light of a cellular device on in the theatre. Some artists would consider this distracting, but in this case we are actually inviting it.  There are certainly questions of attention and attention span. We’re sort of walking into this project and saying that it is assumed that this will be unusually structured.

So why create such an unconventional performance? Well really, the participation is a part of why we wanted to open this out. Because instead of really disrupting or undermining a performance, we wanted people to be able to engage and help the author.

Stephen – The way we’ve enabled that interaction is a very direct and literal effect the audience can have with the lighting of different parts of the stage. [The pre-show] is an opportunity for people to come in and the curtain is open and they can see the initial set and it gives them a chance to explore. Our hope is that creates a deeper and more meaningful bong throughout the rest of the performance. That [the audience] has really explored the set and can experience it on a different level.

In addition to the interaction between the viewer and the stage, do you think that this program will cause the audience to interact with each other?

Jonah – The way that we piloted MassMobile was to have six mobile devices distributed internally to members of the creative team, producers and close colleagues over the course of a few days. I did note, on a couple of occasions, the social interactions that would occur person to person. Maybe Stephen would like to address any interactivity within the platform itself.

Stephen – We did see people that were using the app that were sitting next to each other sort of communicating in person and coordinating their efforts, but at this moment that’s not something we’re doing. [Interactivity within the platform] is something we are looking at in the future of MassMobile. More specifically looking at how do we enable or make sure that people have their own voice in their interactions with the performance.

From a performer’s perspective, what is it like working with something like MassMobile?

Adam – From a performer’s perspective, I think what excites me most about this kind of initiative is that as a performer we’re always looking for new ways to engage the audience and make each articulation of the performance event unique and special and fresh. I think that MassMobile can be a unique way to do that.

The other thing I really appreciate is how we’re using this mobile technology and multimedia to make more personal the performance. Which I think is rare and hard to accomplish.

If you live in the Atlanta area I highly recommend heading over to the Ferst Center for the premiere of FILTER on April 2nd. You can find more information about performance times and tickets at the Ferst Center's main website here. Until then, he is a sneak peak of dancer Adam Weinert and MassMobile.

MassMobile App Sneak Peak from Ferst Center on Vimeo.

Changing Up The Check-in: Foursquare Updates Specials

Image via foursquare.com

Foursquare, the popular location-based social media platform, decided to shake things up a little at the annual South by Southwest festival last week. Rolling out their new 3.0 version, Foursquare has made some significant changes to their platform in the way it operates, most notably in the way it recommends venues and offers specials.

I'll be covering the changes to the recommendation engine in an upcoming post, but today I really want to talk about the new ways arts organizations can offer specials on Foursquare. A "Special" on the platform refers to an incentive that organizations and brands can attach to a venue that is unlocked when a Foursquare user "Checks In" and meets a certain criteria. Up until last week, these specials were limited to either the frequency a user checked in (1st time, 5th time, etc.) or if the user was the "Mayor", a title held by the user that checks in the most frequently at a particular venue.

While a great idea in concept, personally I felt the result was a little lackluster in practice. Becoming the Mayor of a venue is tough, requiring checking in at that venue a large amount of times to temporarily steal the title. While this may be easy to do for somewhere like a local coffeeshop, frequented everyday, it is unrealistic to offer this incentive for an arts organization. Then there is the average check in, some organizations have offered incentives like reduced ticket prices for having a certain amount of check ins, but this is often a one-time special. The hesitation to use a more tantalizing incentive may be the fact that organizations want a little more involvement than having a user hit a button on their cell phone once.

The game has changed with the release of Foursquare 3.0. More types of specials have been added and split up according to whether they are loyalty specials or customer acquisition specials. Plus, nearby specials now appear when a user hits the "Places" tab in Foursquare's mobile platforms, showing users which nearby venues are offering incentives for their foot traffic. Here's a look at how the new specials work:

Customer Aquisition

The Check-in Special - This is the run-of-the-mill, every-time-you-check-in kind of special. It allows organizations to attach an incentive that users can claim with every check-in and is the one most often seen on the platform. Many organizations may still want to take advantage of this by offering cheaper incentives, such as a dollar off admission, to encourage visitors to adopt Foursquare and take advantage of more rewarding specials in the future.

The Newbie Special - Most of the new specials say it right in the name and this is no exception, this rewards users only for their first time check-in. Organizations may want to take advantage of this for the same reason as the regular Check-in special, to encourage initial use of the platform.

The Flash Special - Now we start to get into some of the more interesting specials. The Flash special works like a flash sale, once a certain number is reached the special expires. It is first come, first serve. For example, say your special events are dragging a little bit for the first few hours, on average. A Flash Special can be used to create some interest in showing up first, such as "First 10 people to check-in unlock a private meet and greet with ______ !".

The Swarm Special - Swarm specials are all about building a crowd. Swarm specials are only unlocked if a certain number of users, set by the organization,  check-in at that organization's venue within a set three hour period. This can be a great way to reward a large crowd with an incentive that would be more attractive for a group event than for an individual. For example, an organization could  have a special encore from a performer if that night's swarm special is unlocked.

The Friend Special - This is by far my favorite new special. Users can only unlock this special by having a certain number of their foursquare friends, not just users that happen to be at the venue already, check-in alongside them. When paired with a great reward, this can be a great way to incentivize people to bring their friends along to your venue. The number of friends required to unlock the special is completely up to the organization.

Rewarding Loyalty

The Mayor Special - Hey now, we can't forget about the Mayor! This special works the same way as the old special, it is unlocked by the current Mayor of the venue. The Mayor is determined by which user checks in the most times in the previous 60 days and the check-ins only count once per day towards mayorship. This is so a user cannot stand in your lobby and just hit the check-in button over and over again until they have the mayorship.

Loyalty Specials - A little bit like the regular check-in special, this special rewards frequent visitors who are regularly checking in. One version of a Loyalty special gives users a goal of total check-ins to unlock a reward. Since you can set the number, organizations may want to pair this with a sweet reward for that user who visits over 100 times. Again, only one check-in counts towards this special per day. Another parameter organizations can set with Loyalty specials is the time that user has to reach the set goal, such as having a goal of 10 check-ins for one month. This type of special could be paired up with a more temporary event like an exhibition or a performance series.

It will be interesting to see if and how arts organizations take advantage of these new specials that Foursquare has launched. The biggest advantage to the service is that it is one of the few social media platforms out there that require users to physically be in the space. Coming up in a future post, I will go into some of the other improvements Foursquare has made to its platform, including the ways their new recommendation engine will help direct users to your organization.

Instant Encore: Classical Music goes Mobile

Margo Drakos, co-founder/COO of InstantEncore

Margo Drakos is a woman on a mission. The co-founder and COO of InstantEncore wants to take classical music directly to its audience via a host of online services, including a digital strategy package for powering custom websites for organizations, a website builder for artists and the development of mobile apps. Recently, the company has been busy building custom mobile apps for Android, iPad, iPhone and all smartphones. InstantEncore’s mobile apps have previously been featured on this blog in Tom’s article 10 Arts and Culture Mobile Apps from 2010.

InstantEncore currently powers about 100 iPhone apps and 50 Android as well as hosting mobile web apps, which make an organization's website functional on a mobile phone. Their platform powers the app for the popular YouTube Symphony Orchestra, which, at over 125,000 downloads, and beats both the Taylor Swift and Linkin Park apps by about 90,000. Instant Encore also hosts the apps of notable organizations like the NY Philharmonic, Chamber Music Society of Lincoln Center, Cleveland Institute of Music and Houston Grand Opera.

I talked with Margo recently to catch up on Instant Encore's latest endeavors.

What is InstantEncore? InstantEncore.com is a classical music platform that enables artists and arts organization to harness the power of technology to connect with their fans anywhere, any time. We are the infrastructure! We have created the only classical music-specific digital asset management system that essentially allows our Partners from a broad spectrum of the performing arts world to enter digital content – event listings, ticket selling, audio recordings (streaming, download, public or private), video (live or on-demand), news, blogs, and photos - one time, and have all of their content published in real time to their own website, mobile apps, Facebook or Twitter accounts.

How did the idea for InstantEncore come about? Two things happened: I was playing cello for a living and I had sort of grown frustrated with the disconnect between musicians and the audience, where the field was going, and how technology was disrupting the traditional models. I felt passionate about re-personalizing the concert experience without losing artistic integrity.

An InstantEncore concert card offering a music download.

Meanwhile, I had the pleasure of meeting some wonderful engineers and they had a really powerful search engine specific for classical music that resulted in a high-end jukebox called Maestro. It had originally been created to help catalog vast CD collections, like that of our chairman/CEO. I was talking to them about how sometimes as a performer you would go to a live event and afterwards people would say “I loved the concert. How can I hear it again? How can I get a recording?” And of course I was always trying to sell some unrelated CD. So I talked to these engineers about this predicament that a lot of musicians always find themselves in, and so we actually created this concept of the “Instant Encore”—where you can take a card at the concert, go home, and download the content that you just heard.

How has InstantEncore evolved from the days of concert cards to now? Instant Encore started with a very powerful search engine that had organized and standardized all classical music meta data, and our objective was to build the tools that would enable fans to connect with the music and artists they love, in a personal, immediate way. We wanted to leverage technology to extend and enhance every aspect of the live concert experience. We are committed to providing the 21st century tools that will save organizations time and money by automatically or quickly powering their digital assets to connect with fans and engage sponsors.

How does the Instant Encore component fit in with an organization’s existing social media presences and organizational website? Our goal is to streamline redundancies and save resources. Right now, you have a lot of people that are trying to update their website, spending a great deal of time developing a content management system internally instead of spending development time and resources on the front end. Then there is the mobile space--if an organization does not have a good web-browsing experience from a smartphone, people will close the site. Then organizations are manually pushing content that is often a PR push to Facebook and then to Twitter as well. So there’s a great amount of time and effort in trying to maintain all these very necessary social network platforms.

Obviously details about classical music can be a nightmare to organize or for data entry. We have a very standardized way—unless it’s a world premiere, it’s all in our search engine’s system. For example, if you start to type in “Beethoven Symphony No. 5”, it’s going to have the opus number and the key and all the movements and you just click on that and it’s automatically transferred. So, you’re never entering all that information yourself, which is 1) time-saving and 2) tagged at the most specific level, which allows people that are not looking specifically for you to find you and you that already know you and want to find you to customize their experience.

Our system is set up so that you as a Partner can enter an event listing in moments in our secure Control Panel. By that one simple event creation in our content management system, it updates in your website with our webbuilder or more advanced API (Application Programming Interface—see the end of this entry for more information). posts to the partner’s social networks, mobile apps, etc. You can do a host of different things—tag your YouTube channel. You can manually upload music for streaming or download purchase. Or you can create a private download code campaign for donors or concertgoers. One of our most valuable features is this web crawler that goes through hundreds of RSS feeds and finds articles specific to arts organizations and tag any article from newspapers or blog. This can automatically appear in your app or website as the latest news, so you’re not having to physically manage your app or site but your content is current.

The music player feature on InstantEncore's Android app (left) and the events feature on InstantEncore's iPhone app (right).

So basically it’s just a one-stop shop. It’s a digital asset management system where you come to enter your content in one place and it goes to all the platforms that you’re using. We’re just an aggregator and publisher of that content… Our concept was that it should be something that a two-person organization can use our tools and manage and have a very robust, beautiful integrated social media presence or an organization with a budget over 10 million can use the same tool.

Regardless of all the features you offer, many organizations might have trouble getting the rights to use pieces of music or convincing players that a mobile app is somewhere the music should be. What are some of the arguments you can make to convince them that this is something worth doing? I come from the musician/recording side of things, am a current member of the AFM and as a child was a member of AFTRA and SAG, so I certainly appreciate where the musicians are coming from. What I think is important is that, fortunately or unfortunately (however you want to perceive it) we’re in a new period that is such disruptive change, brought on in part by technology and change is always painful. There are new great opportunities but the existing models and the days of residuals in the way that we used to know it, at least right now, are not there. I think it’s really important to actually be very upfront about this. Artists are highly intelligent people and creative people and have wonderful ideas to bring to the marketing and development side of the business. I think it is so important that everyone be a stakeholder charting this new world together.

I think part of this is taking a holistic approach and saying that there are certainly a group of people that are going to want a physical CD’s and there’s people that are going to want to download content. But there’s a certain group of people know who just want what they want, when they want it. We see from our stats that music or videos are increasingly having a very short shelf life. People will often want to hear a new live recording over and over and then they move on and that is it. I often say to my friends and colleagues that you have to have some faith and work together to try things. Every community and every audience is different, but if you don’t have the tools to even explore or try things, that’s very challenging. I think mobile is so essential. People will be accessing the internet via their mobile devices more than from a computer within the next few years. If you don’t have a good user experience to access your content via mobile or any content in it, in my opinion, I think it will be very, very challenging.

You mentioned that you are focusing your research efforts on return on investment in digital media and how orgs can get sponsorship for their technology initiatives. Can you tell me more? Why should anyone care about social media and an integrated digital strategy? Why should anyone care about having a mobile app? I’m a very “nuts and bolts”, frugal person. When I look at some of the organizations that I work with and see how much they’re spending on print material and yet they don’t want to spend a few thousand on a mobile presence, or want to wait a few years to see where this mobile thing goes, I think it is quite alarming. Showing people the return on investment is critical for them to care.

Many arts organizations' apps are sponsored by corporations or foundations.

Many organizations we are working with have packaged a digital or mobile sponsorship package and in many cases, new donors or people who were not previously interested in sponsorship at a significant level, are excited to be part of new technology, sponsoring live video streaming on websites and mobile platforms and much more.

It’s built into our platforms—ways for people to feature sponsors, going through their audience to create new audiences, etc. What I’ve found is that I’ve been pushing organizations to think outside of their printed program with a printed logo and think, how can we think outside of the box and take the old ways that we used to monetize and seek corporate sponsors and take this into the digital space? Some organizations have come up with some fabulous things. Whether it’s embedded streaming announcements featuring sponsors, ad spots with sponsors, getting grants to cover new educational and audience development initiatives—various things. They’ve been able to monetize this in new ways, from new sponsors (not cutting out of other things).

I am very excited that Telstra, the telecom company in Australia, hired us to create a premium custom app for the Sydney Symphony that will include live video streaming of ten concerts in their mobile apps and website. This is a win-win for everyone.

How should arts organizations approach technology? There are so many platforms out there, there is so much information and everything’s changing so rapidly. I remember launching the app with the New York Phil and at that time, apps were still—people thought “what the heck is that?” So, the most important thing is to integrate digital media strategy into organizational strategy—into every aspect of decision-making across all departments. It involves PR, it involves marketing, it involves development, it involves operations, the audio recording department, musicians—absolutely everyone. I think that buy-in is essential. I think one of the most important things is just to start small and get permission to try something. If it doesn’t work, don’t let it validate that “this is never going to work” and if it’s a home-run, that doesn’t mean that it’s going to work the next time. It’s important to look at just getting started but with a clear, integrated strategy approach. And really, actually, there’s a lot of fear sometimes when it comes to technology and I think part of what’s been fun to see is that oftentimes, it’s a lot more fun than people necessarily anticipate and I think that that’s been a really rewarding part of what we’re doing.

When Thomas Hampson made his recital available for download and Performance Today, Minnesota Public Radio, and European Public Broadcasting Union announced it on air. We read the comments from people from around the world—from United Arab Emirates, to Germany to California to Wisconsin—about how that music impacted them and how grateful they were for that. I was very honored and proud that some of our tools could be helping to connect the power of his voice and his artistry globally in a way that’s never happened before. I think that’s really what it’s about and instead of feeling more protectionist, as scary as it feels, to embrace democratized access in this engaged age.

More info on Maestro API:

Changing The Way You See With Augmented Reality

When looking ahead at what will be the most exciting technology for the arts in the coming year, augmented reality is bound to pop up in the conversation. You may be familiar with augmented reality already. The National Football League has taken advantage of this technology for years to project lines of scrimmage and game time information onto the field in real-time during television broadcasts. So what exactly is augmented reality (AR)? According to Wikipedia:

Augmented reality (AR) is a term for a live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound or graphics.

AR layers digital elements on top of our view of the everyday world. This overlay can be done in a number of ways: through the use of handheld devices like smartphones, through desktop computers with a webcam, by wearing specialized headsets, or by projecting a digital images/animations onto a real world location.

Since AR has the possibility to create such a unique visual experience, it naturally has attracted adopters from the creative community. Here are just a few cool projects taking advantage of AR:

DIY Day MoMA – Augmented Reality Art Invasion! On October 9^th, 2010, Sander Veenhof and Mark Skwarek decided they would circumvent the traditional art world and host their own exhibition at the Museum of Modern Art using AR. The artists took advantage of the augmented reality viewer app Layar, currently available only on iPhone and Android devices, to create a virtual exhibition of digital works. By downloading the Layar app and loading the “AR Exhibition” layer, anyone with a camera-enabled smartphone or mobile device can view numerous digital artworks throughout MoMA. This includes both 2-D and 3-D images and animations and an additional 7^th floor that only exists in the world of AR. The digital artworks have continued to be on display since DIY Day wrapped up in October.

The Virtual Public Art Project Also utilizing the Layar app, the Virtual Public Art Project (VPAP) takes the idea of AR artworks one step further by placing pieces around the globe. The artworks can be viewed in the round and from multiple perspectives, just as you would be able to with a real piece of public art. Check out some of the current works on display on VPAP’s website . If you’re interested in creating your own piece of AR artwork for the public, VPAP puts out multiple calls for submissions.               

The Macula Project – Mapping 600 years of history The Macula Project is comprised of a group of artists exploring the relationship between image, sound and the viewer. The city of Prague was searching for a way to celebrate the 600^th anniversary of the astrological clock tower situated in the center of the city and turned to the Macula Project for a creative solution. The artists at the Macula Project turned to AR for the project and projected this stunning work that took the audience through the 600-year history of the clock tower.  Macula was able to achieve this by digitally mapping the building beforehand and tailoring their animation so when it ran through a digital projector, the perspective lined up perfectly with the real life clock tower.               

The Getty Museum – Exploring the Augsburg Display Cabinet in 3-D If there is one frustrating roadblock shared across the gamut of art lovers, it’s the frustration over not being able to personally handle and explore a piece of art in a museum. The Getty Museum understood this burning desire and took advantage of AR technology to let their visitors have a more in-depth exploration of one of the pieces from their decorative arts collection. Using an online program launched within a web browser, a computer’s built-in webcam, and a printed out “AR tag”, an art lover can handle a 3-D model of a 17^th century collector’s cabinet. By rotating and tilting the AR tag, the cabinet will spin 360 degrees and various doors can be opened and explored.               

The University of Gronigen – Giving students X-ray vision Part of a permanent exhibit in their Hall of the Bernouilliborg, the University of Gronigen teamed up with Science LinX and Nanopodium to create a unique science experience for their students. The exhibit is comprised of multiple boxes covered with AR tags. By wearing a special set of glasses containing display screens and a camera, the boxes’ walls suddenly disappear for the user. The user can then tilt and turn the now ‘hollow’ box that contains various types of 3-D science-related models, such as a model of a chain of molecules.

The types of projects that are now possible with augmented reality are rapidly growing and the technology is some pretty exciting stuff to play with. But beyond the wow factor that comes with AR, what are some of the ways that arts organizations can use this technology in worthwhile and innovative ways? Can AR extend to projects beyond the visual arts? Are there ways that this technology can be used to engage with large audiences as well as the individual? I don’t think it is too far a stretch of the imagination to envision actors on stage reacting to digital props or musicians playing digitally created instruments. As with any new tech, it will be exciting to see how AR develops and continues to merge our reality with a digital one.

New Poll: What's Your Favorite App for the Arts?

As 2010 rolls to a close, Technology in the Arts wants to know: What were your favorite arts-related mobile apps of the past year? Which apps this past year were the best? The wackiest? The most engaging? Take a second and click the link below to let us know. Enter the name of the app, the platform you use it on (iPhone/Android/etc.), and why you think it's the best of the best. We'll be taking the most popular app choices and including your comments in our year-end app wrap.