Recently I’ve been thinking a lot about gaming, despite the fact that I’ve never been much of a gamer. This is probably because live events in gaming have ballooned in recent months, in contrast to the cancellations and low revenues in the live event space that the rest of the entertainment industry has seen during the same time. The trailer for last weekend’s Pokemon Day Post Malone concert is was a revealing example:
At the end of the trailer, Post Malone changes into an in-game avatar version of himself, creating an almost poignant message of just how much live performance has come to rely on the gaming industry within the past year. Since global lockdowns began in March of last year, there have been countless virtual concerts for gaming, from Epic Games’ blockbuster Travis Scott Fortnite concert in April to Lil Nas X’s equally impressive concert with Roblox in November. The reason tens of millions of gamers and non-gamers attended these events was because the escapism that many relished in virtual worlds now became one of the only avenues through which they could access what they missed most in the pre-pandemic real world, including live music performances.
Post Malone’s Pokemon Day performance was one of the first virtual concerts in Pokemon’s P25 Music Series this year to help celebrate Pokemon’s 25th anniversary. While live viewers complained about the brevity of Post Malone’s 13-minute show, they were delighted by the Pokemon world tour - during which a Gyrados jumped in the seas, Charizard brooded in a lava cave, and a herd of Rapidashes stampede through the desert - that the singer’s avatar gave. The investment that Pokemon and other game makers have made in quality graphics for these concerts has increased with the demand for these types of virtual experiences. And while this demand has been expedited because of the pandemic, it’s hard to say that the live business wouldn’t have gotten here regardless. More than a year before Travis Scott, there was the Fortnite in-game Marshmello concert, which was hailed as the vision for the future and became Epic Games’ biggest event at the time. With or without coronavirus, the world was already trending towards bringing more of people’s favorite real-world experiences online.
This year will be particularly interesting for the performance industry, given people’s dual demand for both in-game concerts and physical performances. With a vaccine out and a third manufacturer’s production just approved, much of the music industry is eager to bring back their biggest revenue stream, physical tours and concerts. Just in the past few weeks Live Nation and one of its subsidiaries have begun selling out live festivals planned for August 2021. The fact that these festivals sold out in just a few days and that refund rates are very low point to fans’ predictable eagerness to see their favorite artists in person.
That real-world concert performances were coming back, however, has never been news - it was just a matter of when and how. A part of the “how,” which has become more obvious to the gaming and music industries since in-game concerts blew up, is that live entertainment should increasingly reflect the best of both a viewer’s in-person and online experiences. And, if we look at areas in which the music and gaming industries are making the most investment, the trending answer to this best-of-both experience is augmented reality (AR).
The AR Future of Live Performance
When I think about why AR has been popularized and more accepted in live performance in recent years, I think about what SUGA, a member of K-pop supergroup BTS, said in a Billboard interview last year about music (begin at 1:24):
Music is not just about listening anymore. You can see it nowadays, so I think the visuals are very important
SUGA was speaking in the context of music and fashion, but equally important to live music is performance setting and stage visuals. As has been trending in gaming, the stage visuals of a performer included not just interesting fashion, but the digitalization of the real performer. The inverse has also trended to be true; as much as gaming companies have been experimenting with digitizing real-world musicians in virtual concerts, they have also been exploring projecting virtual characters into real-world performances.
In terms of real-world shows starring completely virtual artists, the most successful case study here of course is Hatsune Miku, a fully virtual idol artist created in 2007 who performs on a real stage in front of real people through an animated projection and a virtual voice created using a vocaloid. Her wild success throughout the world sparked an entire set of vocaloid characters by her creators, Crypton Future Media, that have become part of Hatsune Miku’s cultural universe.
While Hatsune Miku was originally meant to appeal to the professional music community, the idea of bringing virtual stories and characters into the real world has never been a far reach for the gaming community. Successful games almost always expand their orbits by bringing their digital experiences into the real world, whether through amusement parks, live events, television series, or even cosplay. And in the past few years, gaming companies have extended this to include AR performances that would bring live events to their users. Since 2018, Riot Games has created a collection of music skins for its hugely popular League of Legends game, of which K/DA, an AR K-pop group, has seen smashing success. Their live concert performance in Seoul 2018, during which the projected animations of the K/DA members stood shoulder to shoulder with the real-life musicians who voiced them, received more than 20 millions views. K/DA has continued to see mainstream music success since then, including a musical comeback this past November through their single “MORE” that received high rankings on real-time music charts.
AR live performance was the natural next step in bridging the gap between the digital and the real world, increasing engagement through the familiarity that game players already have with their favorite characters. Given the success of virtual concerts and the ever-increasing integrations between music and gaming, it’s no wonder that music industry giants have also begun to see the potential in incorporating AR into live performance experiences. In January, Sony announced it was creating a subsidiary focused on immersive media that uses Unreal Engines, the gaming engine from Epic Games. This comes on the heels of Epic Games’ wrap-up to its Spotlight event in the fall, yet another well-attended musical concert series featuring well-known artists.
The popularity of virtual artists has also influenced the way real musicians have thought of incorporating AR onto their stages as well. Particularly in K-pop, where visuals are of critical importance, companies are experimenting with a future in which virtual avatars stand on a real-world stage to an adoring crowd. In September of last year, major K-pop music label S.M. Entertainment debuted Aespa, a new K-pop girl group with both real and virtual members that would be the first K-pop group to have hybrid virtual and live interactive experiences. Each member has an avatar version of themselves that is expected to stand on stage during these performances. While it has yet to be proven exactly how successful Aespa will be, the fact that a leading K-pop label was willing to acquiesce and invest in an AR-inclusive idol format caused much fanfare upon the group’s debut.
Indeed, I suspect that the appetite for AR performances will be here to stay even after the vaccine brings live performance back. Even outside the scope of live concerts, AR is increasingly affecting the way the industry thinks about artists' image and how they connect to fans. For example, a new insurgence of holographic performances of deceased K-pop stars using AI technology will allow fans to experience a connection with their favorite artists even after they have passed. In China, where concert venues have been open since late last year, AR performances and virtual artists remain popular. A discovery and survival competition for strictly virtual artists, Dimension Nova, was hosted by a handful of high-profile Chinese celebrities and took off when it aired in September. A new era of AR performance in which technology, and not necessarily the music, creates an immersive experience, is upon us.