Dolby House at SXSW 2023 runs from March 10-19th at Brazos Hall, Floor 1, 204 E 4th Street.
[ TITLE ]
'KOKOVERSE' AT SXSW DOLBY HOUSE
[ DATE ]
15/03/23
[ ENTRY ]
#047
[ A DEMO FOR FUTURE LIVE VR PERFORMANCES ]
At SXSW (the confusing acronym for Austin’s South By South West event) Draw & Code have revealed a tech demo that harnesses Dolby.io and Move.ai technology to experiment with future performance capture pipelines for the music industry. This working proof-of-concept playfully dubbed ‘KOKOVerse’ takes an onstage performance by rising UK soul star Jake Isaac at London’s spectacular KOKO venue and transfers it into a trial VR performance.
Dolby House arrived at SXSW 2023 and packed a punch with a stunning array of tech showcases and industry speakers. What’s more, it is not only for SXSW badge holders but the general public also have the chance to experience the next generation of visionary technology for TV, movies, games and music.
One of several showcase projects at Dolby House is ‘KOKOVerse’, Draw & Code’s tech demo that utilises Dolby.io’s real-time streaming technology and Move.ai’s movement capture system to stream a live music performance. Music and the metaverse are often cited as ideal bedfellows but we feel there is room for a different take on this concept – one that majors on the capture of music performance in a natural way that is geared towards the needs of the musician and their fans rather than the restrictions of the digital delivery platform.
While there are some exciting moves in music and the metaverse projects, the production pipeline appears to be at the whim of the tech and games industry, not the artist. There is also a disconnect between the real-world gig experience and its virtual cousin, albeit sometimes that means pushing the envelope of creativity which does have its benefits. This experimental project aims to turn this emerging model for music performance rendered within game environments on its head.
Recording in a real music venue means there are heaps of benefits to the artist and ultimately the audience. Musicians often have exacting requirements in terms of sound and staging within music venues. By operating the capture in a world-class venue such as KOKO, we do not have to duplicate a sound engineer’s complex set-up or find a studio space with room for the artists to work in. KOKO is fastidiously built to be a brilliant place for a musician to ply their trade.
[ USING NEXT-GENERATION TECHNOLOGIES ]
While other motion capture music performances typically use specialist suits to track performance, using Move.ai’s software meant no suits were required. The artist is free to express themselves with the clothes they want to wear yet their every move is transferred accurately onto the virtual stage during the live performance. All the Move.ai team required at the shoot was a few iPhones, nothing more. The promise of this technology has not gone unnoticed, with Move.ai sweeping the Innovation Award at SXSW 2023 for their mocap software and how it’s changing how we use performance capture.
If we are going to make such efforts to connect the artist with the digital world, we also want to do the same for fans. Draw & Code Labs wanted to experiment with fans being able to see themselves on the virtual stage, even if the only tech they had to do this was a webcam. For this feature we required robust streaming technology – that’s where Dolby.io came in. We integrated Dolby.io’s streaming tech to enable live real-world images to be fed into the virtual world. The whole thing – the webcam and the virtual view – can then be streamed back into the real world via the web. Dolby.io’s philosophy is that we should use technology to feel connected to each other and to the art and entertainment we love.
Why would a music venue, such as KOKO, be interested in a digital twin and such an innovative capture and streaming pipeline? The ultimate aim of the project would be to give music lovers a comprehensive and credible virtual music venue – something that has its roots in a real-world location with heaps of history and a proven track record. To do that you need the right technology and the right team, something Draw & Code found when working in alliance with Dolby.io and Move.ai.
