Dear Immersive Filmmakers: please stop making "Interactive Experiences"
Video is the only format that will sustain your career
This article is intended as an open letter to the creative community of XR professionals that gathers at festivals like Cannes, Venice and SXSW: you are a hotbed of innovation and artistic initiatives - and the source of most XR production in the world - but please be mindful about which technological formats you invest in, as they will not all be able to sustain your career.
TL:DR;
The XR community (esp. in public-funded Europe) shifted from making linear stories to interactive experiences because of financial incentives from Meta
But XR interactivity requires to use real-time 3d game engines for production; this inherently limits the audience, partners and commercial viability of a project
I argue that Immersive Video has matured as a format, and is much more likely to sustain the development of a healthy ecosystem for XR creatives & professionals
I’ve been thinking about this article since I came back from SXSW, where I gave a talk about the XR market for general media producers in the music, film, and advertising industries. My talk focused on Immersive Video and the unique opportunity it represents: demand for the format is growing fast; it is easy to produce and platform-agnostic (you can distribute it on any headset). At Acute we think Immersive Video is the format that will finally grow the XR market - this is why we started the platform.
But at SXSW I also spent time interacting with XR creators and producers, and I tried as many pieces in the official selection as possible. Like in most festivals, the majority of XR experiences shown at SXSW were location-based experiences (LBE): physical installations made for one or more users, that are very difficult to scale and monetize.
Festivals like SXSW, Cannes or Venice are amazing: they are structural to the industry and everyone working to make them happen is passionate. They are also a very privileged way to experience Immersive media, and unfortunately many of these LBE experiences will never be shown outside of festivals. Since I want artists to have the widest possible reach, I thought I should explain in more detail why I think the XR industry should pay renewed attention to Immersive Video.
👉 My goal here is to help XR creatives and professionals find a sustainable business model, by maximizing the earnings they can make from their work
Before I start, some quick definitions. Note that I am writing this for a general audience of creative professionals, so my goal is not to be hyper-accurate about technical terms:
XR: anything to do with a headset. Whether it be location-based (with a headset), at-home entertainment (with a headset), augmented, virtual or mixed - as long as it’s in a headset, I call it XR.
Immersive Video: XR content made with a camera. The Blackmagic URSA Cine Immersive, with its high-end specs and affordable price, will make the bulk of production for years to come, but there are other systems out there - some cheaper, some not. Note: video is a file that can be played on any headset.
Real-time 3D: XR content made with game engines like Unity and Unreal. The people who are proficient to use these engines usually come from the videogame industry. The output of these tools is an executable program that needs to be installed, updated and adapted for each headset out there - at great cost.
Now you may be starting to see where my argument is going. Let’s dive into it.
1. The creative industry is built on filmmaking
Cinema is almost 150 years old. For a long time, it was the most popular form of entertainment. An entire industry was developed to serve the needs of filmmakers and their audience: the roles across the value chain have been defined, from the macro-level of companies (producers → distributors → exhibitors) to the micro-level of people (Screenwriter, Director, DOP, Gaffer, DIT, Editor, Colorist, Sound Engineer…).
Even today, with streaming and digital filmmaking, the overall production chain remains the same: you make a storyboard, you find a location or a soundstage, you run a casting, go to a rental house, hire a crew, bring the film to a post-production suite etc.
These practices have been refined over generations, and they form the basis of most of creative production throughout the world: even though there will be variants for each of their specialties, a live TV producer, an advertising executive and a film director will use the same tools and processes. They all speak the same language.
👉 It’s a huge leap to ask people who have built their entire career on filmmaking to suddenly use real-time 3D engines!
My theory is that what held back XR over the years is the choice to use 3D game engines, when most of the decision-power in the creative industry is based on filmmaking. Asking people who have built their entire career and expertise on given processes (storyboard, previz, blocking, table reads, rough cuts etc) to use entirely new tools (3D game engines) that are controlled by a completely new generation of professionals (game developers, 3D artists) will inevitably reduce the number of projects that can be commissioned, produced and distributed to the general public.
Anyone who has ever pitched an XR project to a “traditional” film institution - a production company, a brand buyer or a TV commissioner - will know that they simply don’t get it. It’s not against them: it’s just not their expertise, and they can’t project themselves or make a confident decision. As a result there are fewer projects made and the audience has nothing to watch - thereby limiting the appeal of the format in a self-fulfilling prophecy: “why would I buy a headset, if there is nothing to watch in it?”
2. Meta is the reason XR pursued interactivity
In 2019, Meta (then Facebook) launched the Quest: it quickly became the best selling headset around the world, and for a while Meta was the only company funding XR production. They commissioned a few narrative experiences, but very quickly Meta decided to focus on gaming - or at least, interactive experiences.
The Quest came with hand controllers you see, and the data showed that users spent a lot more time engaging with interactive content compared to linear (non-interactive) content. As a result, Meta started to require projects to have some interactive elements if these projects were to be funded by the company.
It took a few years, but by 2022 the CNC in France (at the time the most active public institution in the world funding Immersive creators) renamed its grants program to incorporate “interactivity” in its description. Public film funds often require that a portion of any project’s budget comes from private sources, and since Meta was the only private source of funding available, the CNC (originally a film institution) had to adapt its criteria, if it was to keep supporting the emerging generation of Immersive creators and help them get funding from Meta.
Slowly but surely (these projects take years to make, so industry-level changes take time to materialize) Immersive creators - many of them coming from the film world - started learning the language of interactivity. Primarily, they learned to use Unity and Unreal - or at least they learned to talk with people who knew how to use these tools, since videogame 3D engines are the only way you can produce interactive XR experiences. “Linear” video (i.e., films) became out of fashion - not because of the format itself, but because it was impossible to fund.
Don’t get me wrong: I am immensely grateful and appreciative of both the CNC and Meta for providing Immersive creators and producers (including me!) with the support required to bring their projects to life. I am simply here trying to contextualize why we’ve seen a shift in XR over the years from linear to interactive content, and so from live-action filmmaking to 3D engines.
3. Video is the most sustainable format for XR
Now here is the issue: 3D content is extremely expensive to maintain. One of the things that XR creators have learned, as they started using 3D engines, is that you are never done updating your project. The main difference between a film and a game, from a commercial standpoint, is that a film is done once and for all, whereas a game requires constant patches - either for security purposes, or because the headset manufacturer updated its OS, or because you are pursuing a “live-service” monetization model (sending updates to players so that they come back and stay engaged with your game).
In short: XR professionals were forced to make games, but without the required resources. Production budgets had to be shifted to distribution, updates and technical ports. As a result, we ended up with game-like experiences that are very limited in scope (often < 15min) and interactivity (Filmmakers are not Game Designers). None of these 3D experiences have any chance of making money for their creators. This is not sustainable, and it really makes me sad to see artists investing a dead-end format.
Again, don’t get me wrong: some of these experiences are absolutely mind-blowing. At SXSW, I cried watching A Long Goodbye (directed by Kate Voet and Victor Maes) which won the Agog Immersive Impact Award. These are truly unique masterpieces.
There is also a growing market, starting from Europe, for Location-based experiences: companies like Excurio, UNIVRSE, and Unframed, are finding business models to produce and distribute ticketed, collective physical installations. The Infinite has proven that it is possible to generate significant revenue from such experiences.
But often times, interactive pieces in festivals end up being distributed as location-based because there is no other way to monetize them: LBE has become the default mode that creatives get stuck in, since the only they could get funding was for an interactive piece, even though their experience is more of an individual one.
These real-time 3D experiences, whether they be at-home or on-location, all lack what is needed for the emergence of a truly sustainable economy: a standardized, inter-operable format. If you use your hands, your feet or your eyes as an input; if you can run on Quest, Pico or Apple headsets: each of these decisions require a unique technical adaptation - and finding the budget that goes with it.
In comparison, Immersive Video is a simple file, that requires minimal adaptation in order to be played on any of these headsets; audiences do not need to learn your input method - these are linear experiences, and we all know how to sit and watch a movie.
180 and 360 video production has also been abandoned because at the time, the resolution of XR headsets wasn’t good enough: you could still see the grid of pixels, and it’s not fun to watch a concert (say) through a screen door.
But Apple, with its Vision Pro, has changed all that: the headset is designed to make virtual content indistinguishible from reality - and for the most part, it succeeds in its goal. Sure, it’s expensive: but the next generation of headsets (Pico 5) has a similar resolution, and the price of hardware will go down as the content available will go up.
Most importantly: video is guaranteed a durability through time that is simply unrivaled. We can all watch Lawrence of Arabia (1962) and still enjoy its magnificence, whereas game preservation is a real issue beyond even XR: how many games from your childhood are you still able to play today? I would bet much fewer than movies.
→ Conclusion: my plea to Immersive Filmmakers
I write this as the Cannes Film Festival is about to start, and with it its Immersive competition. This letter is directly intended to the artists that will attend the festival, but also representatives from the instutions that help and support the XR industry: governmental funding bodies, cultural institutes, other festivals and magazines etc.
I know that we have all been burned by the “VR winter” of 2020-2022: funding dried out, private industry (brands, advertising agencies) did not commit, and we had to find new ways to fund the media that we all believe in. That is all very understandable.
But I can see coming, from here in California, a new wave of interest in content made for headsets - triggered in large part by the (slow but steady) arrival of Apple to XR. Apple is in my mind, the only company that has any chance to bring Immersive to the mainstream. Meta, Pico and Google will just always be limited to niche enthusiasts.
So dear artists, festival organizers and TV commissioners: thank you for all your work, but please pay attention to Immersive Video. Dust the decks and scripts off from the shelves and drawers you put them in. Even Immersive documentaries have a chance: the best-selling app on Apple Vision Pro is a library of simple travel videos shot by one single person. Imagine what you could achieve with your resources and expertise!
I can’t wait to see more Immersive Films, made by career professionals and inspiring artists. If you think I can help you make a project in any way, please get in touch!
Paul
PS: I know it can feel like going back in time: “180 Video, really? But we were doing this in 2016!” Don’t forget that in this industry, we all have been incredibly early and the general public is only just now catching up with the possibility of the medium. We need to stay humble and go back to the basics!
PPS: of course, it’s totally OK to create a Location-based experience using 3D engines if you are clear-minded about the economic outcomes of this business model. If you are fully intentioned to make a piece with this distribution in mind, congratulations! I just wouldn’t want people to assume that this is the default (or only) business model.
PPPS: I guess what is missing to assuage the fears of people who are working with 3D engines to make XR content is a consideration of the following:
How can I exploit my XR content made with a 3D engine as a video file format?
How can I show Immersive Video as a location-based (group) experience?
How can I make money off of my Immersive Film?
These will all be topics that I want to explore in further posts. Let me know what you think!
Thank you for writing this! You put into words my excited and frustration with XR. And I believe too that things are changing!