Filmmaking in Augmented Reality: An application to help filmmakers

By Jesse Vander Does / 03 Jul 2017

Topics: augmented reality | arkit | blog | lab

The film industry isn’t new to mixed reality -- the visual effects industry has been mixing real and computer-generated (CG) elements for decades, and the complexity of those effects has grown incredibly.

As an agency that specializes in developing mixed reality applications that enable creative work, we’re planning to prototype a tool to assist filmmakers.

Here’s our concept: We’d like filmmakers to be able to look through their phone’s camera and see virtual elements composited in real time. They should be able to block out elements in their scene, frame shots with a virtual lens, then record takes for reference. We think by blocking on location before the shoot -- possibly during a walk-through -- we can cut down on the setup times during the day.

Of course, it’s easier to move virtual items than real times.

We’re still nailing down what would go into our very first prototype, but I have a few ideas and a few questions, which is a good start:

On placing virtual elements throughout the scene: What types of objects should you be able to place in our MVP?
When setting the virtual lens: Should we have preset camera models and focal length, or something more open-ended, like sensor size and focal length?
Recording a take and saving it somewhere: What does that look like?

We’re working with Jyothi Kalyan Sura, an MFA at the University of Southern California, to build a proof of concept and get it into the hands of filmmakers. Luckily, for the filmmakers all around us in our home base of Los Angeles, they can test the application in its earliest stages.

Jesse Vander Does

Culver City, CA

Jesse worked for 5 years as a VFX artist and software developer. He now is a developer at HTML Fusion building web apps, bullet time photo rigs and holographic user interfaces.