Maximizing Concert Experiences Through AR & VR

Application of VR & AR in music. Infographic by author.

The application of Virtual and Augmented reality in music has been around for a while. From collaborative creation and interactive music videos to its incorporation in art museums, it seems as if the arts industry is competing with itself to enhance the audience experience. As concerts are a highly user-oriented experience, these technologies have started gaining more traction in the live music space. The demand for interactive and immersive experiences has called for innovative models to be incorporated within music. Thus, the application of such models is added to the concert experience with the implementation of VR and AR technologies to help maximize its interactivity. Data shows that the VR/AR market is projected to increase by more than 7 times between 2018 and 2022. The incorporation of these technologies in live entertainment not only contributes to consumer satisfaction but also acts as an additional source of income that will be taken advantage of by the industry.

METHODOLOGY

This paper comprises both primary as well as secondary research. The primary research contains a survey that was conducted to understand how different demographics think when it comes to attending concerts and what their thoughts are on VR technology. The survey was distributed on social media platforms and resulted in 56 responses.

I also conducted an ethnography where I entered a musical metaverse using VR technology. I will draw comparisons in the use of this technology to deliver an experience to actually being physically present at a live concert. Moreover, I have come across some extraordinary secondary research that helped me go into the depths of what has been going on in the VR/AR sphere and what the future holds.

VR Concerts: Case Studies

The main goal of these virtual concert experiences is to make the end-user feel completely immersed in an experience. However, in the VR concert setting, there is a difference with how people approach others or interact with them. According to an experiment conducted by students from UC Davis and the University of Maryland, people in live concert settings would willingly move towards others to strike up a conversation in real life. In a VR experience, people directly initiate specific conversation with others - perhaps to compensate for the social element of attending live events. 

Currently, VR creates an environment where people interact as avatars and have the ability to live chat with other avatars. An online review from a Billie Eilish Concert gave insight into how such concerts operate. It confirmed one of the biggest motivations behind attending a concert is social interaction. The Billie Eilish VR concert experience allowed for the option to attend in a social or a solo mode. The virtual experience also allowed users to mute people in their environment. These features were added to possibly enhance the experience of attending a concert from home.

Image of Virtual Reality concerts. Source: Rolling Stones.

Since maximizing the experience becomes the main goal for delivering any event, a number of studies have attempted to find a way to accomplish that in VR. This has been tested is through the user’s ability to manipulate movement in the VR sphere. One study talks about four approaches that aid movement - self movements, copying other avatars’ movements, synchronizing movements to the beats, and using machine learning to synthesize movements. 

Another study talks about the use of Haptic Technology to enhance the experience of using VR tech. Haptic feedback refers to sense of touch felt by the user via vibrations, motions, or other forces. The user’s muscles are activated in order to make them feel more present in the virtual space, thus more immersed.

These studies in aiding movement are part of a larger goal in increasing user engagement and providing maximum utilization of technology. For instance, AR can help locate friends, concession stands, and restrooms in live venues. This proves to be useful in extremely crowded arenas. A different use of user engagement in AR is the audience’s ability to individually interact with the artist. One such example of this is the Maroon 5 concert which has a sing-along karaoke on screen if one pointed the phone to the stage. Another concert depicted spaceships flying around the stage if the device was pointed towards it. 
Other sensory elements - scent, for instance, are also measures that are being considered to maximize the user experience. These features allow users to attend a concert with friends in the same room using VR headsets. The concert VR experience allows users to enjoy the music and interact with friends in a familiar environment, altered to look and smell like a concert arena.

Ethnography: VR Concert Experience

The author used ethnography as a research methodology, thus this portion of the written account will be conducted in first person.

I wanted to get a first-hand experience of attending a concert virtually. The ethnographic study left me with mixed feelings. Through this research, I used the platform Soundscape VR. One important point to raise from this entire experience is that it doesn’t matter if the experience is live, but not being physically present in a concert arena brought about a sense of laziness within me when it came to attending the concert, almost creating a sense of “it’s not real anyway.” Additionally, the pre-concert excitement that I have always felt while getting ready for live concerts was largely diminished.
This ethnographic study I conducted and comparing that with collating data from online reviews both involved two applications – one with music one without. Through this comparison, the following figure below was developed to rate the experience, with a score of 1 being the lowest, and a score of 5 being the highest.

Figure 1: A comparison of VR concert Experiences. Infographic by author.

SOUNDSCAPE VR

When it comes to making concerts as authentic as possible, the graphics did not cater to this purpose. When I attempted the VR concert experience, it was reminiscent of a video game. Different applications need different graphics to deliver an experience. The Soundscape graphics could be influenced by the fact it was an application launched in 2017, but once I surpassed the moment of awe of being in a VR space, I began to miss the realism component. While photorealism has been achieved in AR, VR, and in video games, it is still far from being a completely immersive experience. 

When I talk about the logistics associated with attending concerts using a VR headset, the pure weight of it on my head started making me nauseous and irritable and I wasn’t really able to appreciate the music. According to the survey I had conducted,  if something is blocking the ability for concertgoers to fully experience the music, the quality of the experience goes down. 

Soundscape VR was lonely. The ability to look at my friends, dance with them, and sing our hearts out disappeared – it was just me in a digital arena (a bit too futuristic, in my opinion) listening to music.

Due to time constraints, I wasn’t able to partake in a live VR concert experience, I had to watch a concert that was recorded in VR. Thus, my opinions about this experience weren’t as positive as I was expecting them to be. 

TILT BRUSH

Tilt brush, a painting app for creative expression, had basic graphics. Despite this, the developers were able to succeed in what they intended to deliver. I do not believe that extremely rich graphics would have made a drastic difference, regardless, as I found myself thoroughly enjoying the painting process.

Tilt brush as an application did not require me to socially interact, thus I never felt lonely or a lack of social presence. This made me feel more detached from reality, contributing to my immersion in the experience.

BILLIE EILISH - ONLINE REVIEW

The aforementioned Billie Eilish concert case study was deemed a success to the user, but they noted that its immersion could have been increased with certain features. 

The Billie Eilish concert had an option to chat with other avatars and interact with people all around the world. The author titled their review “Billie Eilish In Oculus Venues Was Good Social VR, But Not A Great Concert.” They noted that the graphics of the Billie Eilish concert were decent enough, but the depth perception was off and some of the camera angles of the concert didn't make sense.  Other online reviews praised the graphics of such concerts, especially the recent ones, which exemplify the advancements in the implementation of VR and AR in live music.

ADOPTION OF VR & AR

ARTISTS

The pandemic provided an avenue for AR and VR to be more frequently used. Artists did not have the opportunity to engage with their fans, and the usage of such technologies allowed them to once again do the same. The interactive elements provided through these virtual experiences act as additional income for the artists, as well as create a way to use the money for good through charity donation. Inexpensive tickets for such experiences allow concertgoers to be open about donating part of the residual to charity. 

BUSINESSES

While the development of VR began in the 1800s, big names such as Google, Facebook (or Meta), and other tech companies will be known for pioneering this technology, as they have have acquired or backed the startups that are responsible for the development of VR and AR headsets.

Currently, in Big Tech news, Apple is in the process of acquiring NextVR for $100 million dollars. It is entering the VR and AR space, attempting to make it the next big thing. It is predicted that Apple glasses will replace the iPhone in the next decade or so. AR has projected growth to $78 billion by 2024. Big Tech is investing heavily in VR and AR in order to dominate tech sphere and suppress potential disruptions in the market.  

CONSUMERS  

VR and AR prove to be a good option to have for live music, especially with the factors of convenience and safety in consideration. According to the survey I conducted, 76.8% of the people would prefer a safer alternative to attending concerts, which is where the world of VR and AR comes in. Moreover, the one-time investment will lead to cheaper concert tickets and wider accessibility to shows around the world that one could not have seen otherwise via the comfort of their homes.

Figure 2: Survey of users' opinions on concert safety after the pandemic. Source: Author.

Aside from the pandemic, an important point to highlight in safety is crowd control. A Travis Scott concert in November 2021 led to a crowd surge that killed 8 people. Beyond the Astroworld tragedy, there have been multiple incidents in concert history where crowd surges have have led to people losing their lives. Thus, if there is an alternative mechanism for people to attend concerts, there is a possibility that it could save lives. Adoption of VR concerts could prove to be an instrument in preventing such accidents.  However, based on the survey I conducted, there is a clear sense of hesitancy in people’s perception about VR and AR technology.  

Figure 3: User Perception of VR and AR. Source: Author.

No technology is without its limitations. Over the years, VR and AR have faced backlash. This is a result of:

1. Graphics Rendering VR: Real-time graphics are still far from perfect. While VR is close to producing photorealism elements, it isn’t enough to make a user feel completely immersed and in a different world. While the future will have faster and more advanced GPUs, the current technological climate is not there yet.

2. Field of View: FOV proves to be one of the biggest obstacles in the acceptance of VR tech. Human vision enables people to see 190 degrees horizontally and 120 degrees vertically. In comparison, VR only allows users to see a mere 90 degrees. This limits how immersive a user can feel in a VR experience. 

3. Feeling of Nausea: A concert duration on an average is about 3 hours. People generally want to enjoy an experience of this length. However, with a heavy piece of equipment on your head, users often experience motion sickness, especially when camera angles change. This is something I also experienced while performing the ethnography. For me, 15 minutes was my benchmark in handling the application. If a VR application has a frame rate with lower frames per second, it can induce extreme disorientation. However, app developers target 90 FPS to ensure that this doesn’t happen.

4. Expensive and Bulky Headsets: VR headsets are extremely expensive. When the Oculus came out, it was about $600. Now, with growing popularity in the mainstream, VR headsets cost about $300. However, until this price becomes worthy of investment for the masses with respect to the graphics and content availability, users may not be happy with their experience compared to its cost. Moreover, the weight of the VR headset causes inconvenience in terms of portability and weight.

5. Ethical Issues: This is perhaps one of the biggest conundrums regarding VR and AR. With such an immersive experience that is not truly real, boundaries get blurred. Since live concerts allow you to socially interact with people, even though you’re physically in your environment, people might take advantage of the situation and use it as an opportunity to abuse or harass people. This concern can also be attributed to the morality of the kind of content that gets delivered through VR and AR. 

6. Inconvenience in AR: The concert experience is diminished with the task of taking out a phone to point towards the stage. This causes audiences to watch concerts through their phones, detracting from truly enjoying the live experience. Until a wearable AR tech is developed, this inconvenience will continue to be an obstacle for AR users.

7. Graphics for AR: In order for AR tech to work, it must to fit its graphics into the perspective of the device that is being used. With the limited processing power of phones, the conversion takes away the resolution, thereby reducing the quality, and thus the virtual immersion. 

The Future of Entertainment in VR & AR

While the mass adoption of VR and AR has yet to take place, the introduction of 5G makes the future of VR and AR seem extremely bright. Graphics are now divided between the VR or AR device and the edge cloud. This prevents a lag and enables easy processing. As people begin to own a more reliable connection, the adoption of VR and AR will happen sooner rather than later.  The graphic rendering obstacle that stand in the way of enhancing the authenticity of concerts would also improve as more people own 5G tech. In addition to 5G being an accelerator in the advancements of graphics, LiDAR depth sensors will also lead to progress in photorealism and contribute to the immersive aspect. These developments in tech will work in tandem with the reality that VR headsets will become less and less of an inconvenience – bulky headsets will likely soon become a thing of the past. Wearable tech, such as Apple Glasses, will dominate the sphere. This combination of enhanced tech will soon produce a future of mixed reality headsets. 

It is predicted that 40% of interactions on our smartphones would be through smart agents in the near future. Thus, the incorporation of AI with AR and VR should not be a surprise. Machine Learning and AI will contribute to the optimization of AR and VR tech in delivering immersive experiences to the users. Music Festivals will largely start taking place online. With Facebook (Meta), the establishment of a virtual world will enable all live and social events to take place in the metaverse.
While there are obstacles in the mass adoption of VR and AR as concert substitutes, they can be certainly seen as an addition to one. People are still reluctant that current technology can match up to their expectations of the live concert experience. However, future trends in VR and AR would definitely help in making virtual concerts an experience comparable to the physical ones. Thus, while there is hesitancy associated with live music delivery via AR and VR, the countless technological changes every day will result in maximizing the end-user experience, increasing consumer satisfaction.

+ Resources

Andrew Hanson. “What Metaverses Mean for the Music Industry.” CULTR, November 19, 2021, https://www.cultr.com/news/what-metaverses-mean-for-the-music-industry/.

Baker, Harry. “Impressions: Billie Eilish In Oculus Venues Was Good Social VR, But Not A Great Concert”, September 09, 2019, https://uploadvr.com/billie-eilish-oculus-venues-vr-impressions/.

Baer, Drake - “The Real Reason Facebook Bought Oculus: 'Virtual Reality Will Be The On-Ramp To Optical Computing” Nov 20, 2014, https://www.businessinsider.com/why-facebook-bought-oculus-2014-11.

Bardi, Joe - “What is Virtual Reality” - 26 March, 2019, https://www.marxentlabs.com/what-is-virtual-reality/.

Barnard, Dom. “History of VR - Timeline of Events and Tech Development.” VirtualSpeech. VirtualSpeech, August 6, 2019, https://virtualspeech.com/blog/history-of-vr.

Bludov, Sergey - “Augmented & Virtual Reality Coming of Age in the Music Industry” Jan 24, 2019, https://virtualrealitypop.com/augmented-virtual-reality-coming-of-age-in-the-music-industry-cb80fff35008.

Brodsky, Sascha. “How the Next Concert You Attend Could Be in VR”, May 20, 2021, https://www.lifewire.com/how-the-next-concert-you-attend-could-be-in-vr-5185477.

Chang, Victoria. Fang, Jingchao. Gao, Ge. Wang, Hao-Chuan. “Social Interactions in Virtual Reality: What Cues Do People Use Most and How” (October 2021), https://dl.acm.org/doi/fullHtml/10.1145/3462204.3481772.

Chiland, Elijah. ‘Are Virtual Concerts the Future of Live Entertainment?’, March 1, 2021, https://labusinessjournal.com/news/2021/mar/01/are-virtual-concerts-future-live-entertainment.

Charron, Jean - Phillipe - “ Music Audiences 3.0: Concert-Goers’ Psychological Motivations at the Dawn of Virtual Reality,” 2017, https://www.researchgate.net/publication/317389859_Music_Audiences_30_Concert-Goers%27_Psychological_Motivations_at_the_Dawn_of_Virtual_Reality.

Connellan, Shannon. “Facebook's Oculus Venues streams its first VR concert. Was it any good?”, May 30, 2018, https://mashable.com/article/oculus-venues-vr-concert-vance-joy.

Courtney, Megan, Lindenwood University, Bailey King, Oscarjack, Christian Tobias, Alexandra Carther, Maeve Barton, et al. “10 Reasons Why I Love Attending Concerts so Much.” The Odyssey Online, October 16, 2019, https://www.theodysseyonline.com/10-reasons-love-attending-concerts.

Ding, Mandy. “Augmented Reality in Museums.” AMT Lab Reports. May, 2017,https://static1.squarespace.com/static/51d98be2e4b05a25fc200cbc/t/5908d019f5e2314ab790c269/1493749785593/Augmented+Reality+in+Museums.pdf.

DeCesare, Matt & Wang, Homer. ‘The Use of AR and VR to Change the Future of Music - Wharton Neuroscience Initiative,’ April 23, 2020, https://neuro.wharton.upenn.edu/community/winss_scholar_article4/.

Emrich, Tom. “21 Augmented Reality Trends to Keep an Eye on for 2021.” LinkedIn. LinkedIn, April 13, 2021, https://www.linkedin.com/pulse/21-augmented-reality-trends-keep-eye-2021-tom-emrich/.

Farthing, Rob, and Chris Hull. “XR Trends to Watch in 2022.” ARPost, November 24, 2021, https://arpost.co/2021/11/24/xr-trends-to-watch-in-2022/.

Fox, Will. “Virtual Reality – Future Trends.” Future Timeline. Accessed December 7, 2021, https://www.futuretimeline.net/data-trends/19-virtual-reality-future-trends.htm.

Guzman, Joseph. “Beyond Astroworld, 4 Other Infamous Deadly Concert Disasters in U.S. History.” TheHill, November 9, 2021, https://thehill.com/changing-america/well-being/580614-beyond-astroworld-4-other-infamous-deadly-concert-disasters-in-us.

Hall, Stefan & Takahashi Ryo - “Augmented and virtual reality: the promise and peril of immersive technologies” September 8, 2017, https://www.weforum.org/agenda/2017/09/augmented-and-virtual-reality-will-change-how-we-create-and-consume-and-bring-new-risks/.

Jack, By: Gravity. “How Apple's Lidar Is Advancing Augmented Reality.” Gravity Jack, April 20, 2021. https://gravityjack.com/news/apple-lidar-ar.

Jayraj. “9 Major Limitations of Virtual Reality: Limitations of VR.” Appy Pie. Appy pie, November 27, 2020, https://www.appypie.com/virtual-reality-limitations.

Jones, Damian, “Virtual reality gigs are coming – but do we want them?” October 5, 2018, https://www.nme.com/blogs/nme-blogs/virtual-reality-can-make-you-feel-like-youre-at-an-actual-concert-onstage-with-the-band-how-vr-is-bringing-live-concerts-into-our-homes-2381721.

Knorr-Evans, Maite. “What Is a Crowd Surge? What Happened at Travis Scott's Concert at the Astroworld Festival?” AS.com. AS En, November 9, 2021, https://en.as.com/en/2021/11/09/latest_news/1636412818_728896.html.

Leavens, Justin. “5 Challenges for VR Concerts.” LinkedIn. LinkedIn, March 9, 2018, https://www.linkedin.com/pulse/5-challenges-vr-concerts-justin-leavens/.

Lee, Leonard. “ SXSW 2021: 5G Opportunities to Reimagine Live Music.” 13 May, 2021, https://next-curve.com/2021/05/13/sxsw-2021-5g-opportunities-to-reimagine-l.

Marr, Bernard. “5 Problems and Solutions of Adopting Extended Reality Technologies like VR and Ar.” Forbes. Forbes Magazine, June 18, 2021, https://www.forbes.com/sites/bernardmarr/2021/06/18/5-problems-and-solutions-of-adopting-extended-reality-technologies-like-vr-and-ar/?sh=fc8e1e23f239.

Marr, Bernard. “Future Predictions of How Virtual Reality and Augmented Reality Will Reshape Our Lives.” Forbes. Forbes Magazine, June 4, 2021, https://www.forbes.com/sites/bernardmarr/2021/06/04/future-predictions-of-how-virtual-reality-and-augmented-reality-will-reshape-our-lives/?sh=3bcf5bcc68b4+https%3A%2F%2F5ginsider.com%2Ftechnology.

Makarov, Andrew. “10 Augmented Reality Trends in 2021: The Future Is Here.” MobiDev. MobiDev, November 16, 2021, https://mobidev.biz/blog/augmented-reality-future-trends-2018-2020.

Matney, Lucas. “7 investors discuss augmented reality and VR startup opportunities in 2020”, October 20, 2020, https://techcrunch.com/2020/10/20/7-investors-discuss-augmented-reality-and-vr-startup-opportu nities-in-2020/.

Mileva, Gergana. “Using Augmented Reality to Elevate the Concert Experience.” February 4, 2020, https://arpost.co/2020/02/04/augmented-reality-elevate-concert-experience/.

Miller., April. “4 Ways XR Is Changing the Future of Music.” AR Insider, October 15, 2021. https://arinsider.co/2021/10/15/4-ways-xr-is-changing-the-future-of-music/.

Morozova, Anastazia. "Virtual Reality for the Entertainment Market: All You Need to Know to Make Investment Decisions." https://jasoren.com/virtual-reality-for-the-entertainment/.

Onderdijk Kelsey E., Swarbrick Dana, Van Kerrebroeck Bavo, Mantei Maximillian, Vuoskoski Jonna K., Maes Pieter-Jan, Leman Marc. “Livestream Experiments: The Role of COVID-19, Agency, Presence, and Social Context in Facilitating Social Connectedness”, 2021, Frontiers in Psychology." https://www.frontiersin.org/articles/10.3389/fpsyg.2021.647929/full.

Rockhill, Tom. “To Engage Remote Audiences, AR Is More Vital than Ever.” Digital Arts. Accessed December 7, 2021. https://www.digitalartsonline.co.uk/features/hacking-maker/engage-remote-audiences-ar-is-more-vital-than-ever/.

Roettgers, Janko, “VR wants to save the live music business, one avatar at a time”, May 26, 2021, https://www.protocol.com/vr-music-concerts-festivals.

Sahota, Raji. “What is the future of live events?” February 27, 2021, https://www.engineering.com/story/what-is-the-future-of-live-events.

Sambol, Anita. “10 Virtual Reality Trends to Look out for in 2021 and Beyond.” ARPost, April 8, 2021, https://arpost.co/2021/04/07/10-virtual-reality-trends-2021-beyond/.

Tang, Yuk Ming, and H.L. Ho. “3D Modeling and Computer Graphics in Virtual Reality.” IntechOpen. IntechOpen, April 6, 2020, https://www.intechopen.com/chapters/71263.

Tladi, Modisha. “What is Extended Reality and How does it work” May 02, 2021, https://www.makeuseof.com/what-is-extended-reality-xr-how-does-it-work/.

Tracy Watson. “Top 5 Virtual Reality Challenges - from Pricing to Ethical Issues and Health Effects.” Skywell Software, April 22, 2021, https://skywell.software/blog/virtual-reality-challenges-ethical-issues-and-health-effects/.

VK, Sai Krishna, and Community. “What Big Tech Companies Investing in AR Means for Your Business.” Inc42 Media, August 19, 2021, https://inc42.com/resources/what-big-tech-companies-investing-in-ar-means-for-your-business/.

Yakura, Hiromu & Goto, Masataka “Enhancing Participation Experience in VR Live Concerts by Improving Motions of Virtual Audience Avatars.” November 2020, https://www.researchgate.net/ publication/345630178_Enhancing_Participation_Experience_in_VR_Live_Concerts_by_Improving_Motions_of_Virtual_Audience_Avatars.

“Augmented Reality (AR) vs Virtual Reality (VR).” Splunk. Accessed December 7, 2021, https://www.splunk.com/en_us/data-insider/what-are-augmented-reality-and-virtual-reality.html.

Expert Panel, Forbes Agency Council, “11 Experts Share Top Impacts From AR And VR”, Aug 07, 2019, https://www.forbes.com/sites/forbesagencycouncil/2019/08/07/11-experts-share-top-impacts-from-ar-and-vr/?sh=74386a564292.

Fortune Business Insights - “Virtual Reality (VR) in Gaming Market to Reach USD 53.44 Billion by 2028; Ubisoft Entertainment SA Includes Splinter Cell and Assassin's Creed in its Portfolio: Fortune Business Insights™.” September 22, 2021, https://www.globenewswire.com/news-release/2021/09/22/2301045/0/en/Virtual-Reality-VR-in-Gaming-Market-to-Reach-USD-53-44-Billion-by-2028-Ubisoft-Entertainment-SA-Includes-Splinter-Cell-and-Assassin-s-Creed-in-its-Portfolio-Fortune-Business-Insigh.html.

“Hybrid Concerts and the Future of Music Entertainment: Musicians Institute.” Musicians Institute Hollywood, February 18, 2021, https://www.mi.edu/in-the-know/hybrid-concerts-future-music-entertainment/.

“Limitations of Ar.” Limitations of AR. Accessed December 7, 2021, http://aboutaugmentedreality.blogspot.com/p/limitations-of-ar.html.

“The Importance of Frame Rates – IRISVR.” Accessed December 7, 2021, https://help.irisvr.com/hc/en-us/articles/215884547-The-Importance-of-Frame-Rates.

“The Future of Graphics Technology and VR.” A Real New World, February 10, 2018, https://realnewworld.com/vr-graphics/.

“Three Trends Supercharging AR Technology: Jabil.” Jabil.com. Accessed December 7, 2021, https://www.jabil.com/blog/technology-trends-to-supercharge-augmented-reality.html.

“VR vs. Ar: What Will Dominate the Future?” VR vs. AR: What will dominate the future? Accessed December 7, 2021, https://www.electronicspecifier.com/products/vr-ar/vr-vs-ar-what-will-dominate-the-future-1.

“Virtual Reality v Augmented Reality: Which Is the Future?” Wareable. July 12, 2017, https://www.wareable.com/vr/virtual-reality-vs-augmented-reality-which-is-the-future.

“What You Need to Know about AR and VR Concerts." Tickpick. Accessed December 7, 2021, https://www.tickpick.com/blog/what-you-need-to-know-about-ar-and-vr-concerts/.