Finding Their ‘Calling’

*** SPOILER ALERT: This article contains plot information related to the VR feature film Calling. You’ve been warned. ***

On November 2nd, 2018, I was invited to the Beijing Film Academy to witness the premiere of something remarkable: a VR feature film. Directed by Beijing Film Academy alumnus and Griffith University PhD candidate Charles Zhang, in collaboration with writer/producer Jonathan Cui, Calling tells the story of a middle-aged Chinese real estate tycoon under extreme personal and professional pressures, who is driven to the breaking point by a mysterious stalker who may or may not be a figment of his imagination.

Calling is an impressive undertaking with relatable characters, an engaging story, and an approach to cinematic VR that - while not without issues - produced a feature-length VR experience that was surprisingly comfortable. I had the opportunity to speak with writer/producer Jonathan Cui and director Charles Zhang the week after the premiere, and pose a few questions about the experience behind the experience...

KG: I thought the story of Calling was fantastic. People are discussing the importance of creating stories that need to be told in VR, as opposed to in the forms of traditional films or TV series. Why did you feel that this story was perfect for VR? Did you develop the story first, and then think to express it in VR? Or was your intention from the beginning to create something fresh for VR? What was the order of your process?

JC: We wanted to do something in VR, and we wanted to create a story that was suitable. So, the first step in our process was getting that story, and our original thoughts were in three different genres: action movie, slasher movie and suspense movie. Our initial goal was to shoot a 90-minute VR film. Of course after editing, that came down to 70 minutes, but it’s still feature-length. Since we didn’t want to make the audience dizzy or ill, that ruled out the action and slasher genres, so in the end we settled on a suspense movie and started thinking about the story. The writing process was very different from a traditional film script, because we had to constantly think about how the audience feels. The visual impact of VR is very different from a regular movie. For example, on the big screen, you need to have that big explosion or action sequence or other intense scene to engage the audience. But in VR, everything is right there, up close. You’re in it. If a person walks past you in VR, you’ll feel as though that character is right in your face. And this can have the same intensity as an explosion.

KG: The feeling of presence makes it more powerful.

JC: Exactly. There are different ways of creating those powerful moments in VR. But then, if you put that same story on TV or the big screen, it might be extremely dull.

KG: Plus, in VR you may choose to look back at that person who walked past you, whereas in a traditional film they’d just walk out of frame.

JC: Yes, and that’s another important consideration, which relates to the challenge of having an audience sit through a feature-length VR film without getting tired. While our development and production focused on the content issues related to that, we found during exhibition that most VR headsets are too heavy for a feature-length experience.

KG: (laughing) Yes. In fact, I should congratulate you, because while watching Calling, the only discomfort I felt was from the headset, not the film itself. After about an hour, I was thinking, “OK, get this thing off my face.” But that’s not you, that’s the hardware manufacturer. (laughing)

JC: (laughing) I’m happy to hear that it wasn’t because of our story.

KG: Not at all. In terms of the story, the staging and the shooting, you guys did a good job of applying VR as comfortably as possible for the audience. I didn’t feel any sickness. I asked a few people around me afterwards, and they didn’t feel sick either - but they all complained about the headset weight. (laughing)

JC: Within the spectrum of the VR space - with the staging, the blocking and the direction - we established cues right away to let the audience know that this is not a traditional film: you’re inside an environment. We tried to make that clear but also keep it minimal so that we wouldn’t exhaust the audience.

KG: Right. You don’t want the viewer feeling compelled to spin their head around like an owl while watching.

JC: (laughing) No, you don’t. And this is how we made it possible for an audience to sit through a long-form VR experience.

KG: Yes, I think you were quite successful in that regard, which leads to my next point of discussion regarding your use of the VR camera. As people navigate the real world, they generally focus on their frontal 120-degree to 180-degree field of view. They’re aware of things around them, and may turn around occasionally if they hear or sense something, but people generally don’t walk down the street with their heads craning around, so we shouldn’t expect that of them in VR. In the early days of computer graphic animation, people were flying the virtual camera all over the place just because they could, but eventually they calmed down and began incorporating tried-and-true cinematic practices. And we see something similar happening now in VR. I remember halfway through Calling, I was thinking to myself, “Hmm. They’re using the VR camera fairly conventionally, which is comfortable, but I wonder how it would feel if the camera was from the point of view of the unseen antagonist - the stalker.” Then at the end of the film, when it becomes apparent that the “stalker” is probably a facet of the protagonist’s split personality, it then makes sense why the VR camera was used as more of a “God view.” Did you guys discuss your approach to the VR camera in advance of shooting, or did you arrive at it during the process of filming?

JC: We definitely considered it during story development.

CZ: Yes, before shooting, we had extensive discussions about the point-of-view of our VR camera. We wanted to find the appropriate perspective from which to express our idea, and found that the “God view” was the best way to focus the audience’s attention - especially in complex scenes. In simple dialogue scenes, we could place the VR camera down among the actors.

JC: When we were writing the story, we broke it down with storyboards to figure out staging and camera position. But then while we were on set, there were some surprises.

KG: (laughing) I’ll bet.

JC: Many of them non-VR-related, such as our access to a location being cancelled, in which case we had to adjust on the spot. From a story point of view, the VR camera represents the state of mind of our main character -- it might be used in a “God view” during flashbacks or intense situations, and then more intimately in conversation scenes. In certain respects, this is similar to the ways you might apply a traditional camera, but it’s also very different.

KG: Yes, and speaking of your main character, one thing that impressed me is that he’s a very relatable protagonist - especially for contemporary Chinese. I’m viewing it as a foreigner, of course, but I understand the social pressures and the need to achieve. It was quite heart wrenching to see the hero and his wife struggle to pursue a better life, but in the process, she is pressured to have an abortion, which causes their marriage to dissolve. This is something that people - even if they hopefully haven’t been through it - can relate to as “real” life in virtual reality. In particular, Chinese people are now facing the choices and ironies of what happens when the good life you’re chasing turns bad. Is it worth it or not?

JC: Exactly.

KG: And the plot point of the real estate market boom, and people jumping into stocks hoping for quick profits but then getting burned, is also something that everyone can relate to. (laughing)

JC: (laughing) You know, I was educated in the West and then came back to China. And I wanted to show how some Chinese people became rich in a very short amount of time, and the price that many of them paid for that, which is something most foreigners don’t know or understand - especially in the context of the larger history of China. This aspect of modern China, and the mentality of the Chinese people, is very interesting.

KG: Yes, and the flashbacks at the end of the film were bittersweet, where you have scenes of the hero, his wife and his friends in more innocent times, dreaming about a better life together. The original thoughts were very pure, but then it all goes really bad.

JC: Yes.

KG: So, that brings us to my final question: What was your biggest learning in making a VR feature film? Was there anything you weren’t able to do in this one that you’d like to be able to do in your next one?

CZ: The next one will expand upon our experiences making Calling, and involve collaboration with other filmmakers. Perhaps you’d like to work with us?

KG: (laughing) Happy to.

CZ: We’d also like to create VR series and other immersive entertainment formats, in order to explore the possibilities and expand our experience. VR is fairly new compared to other media. We need to make more VR content and better VR content in order to push the medium and the industry forward.

KG: Was there something in particular, creatively or technically, that you weren’t able to do in Calling that you hope to achieve in a subsequent project?

CZ: (laughing) Many things. There are new possibilities based on VR camera improvements - especially in the area of real-time monitoring on set. We’d also like to explore the application of head-mounted VR cameras...maybe even put one on a dog. (laughing)

KG: (laughing) That’d be a crazy VR film. I strapped a GoPro to a dog’s butt once, and it was just nuts. Watching the regular video almost made me sick, to say nothing of if we had shot it in VR.

CZ: (laughing) We’d also like to use greenscreen to incorporate more special effects into our VR films. Most importantly, we’d like to expand our understanding and application of virtual spatial narrative, transitions and montage, and to apply new theories and research to our next VR film.

JC: Between the uncertainties of the technology and the constraints of our budgets, creating stories for VR is like jumping off a cliff with your hands and feet tied. (laughing)

KG: (laughing) That’s a pretty vivid metaphor. As a writer/producer, do you try to write freely and then train your cold, hard producer eye on it later, or do you have your producer hat on while you’re writing?

JC: I try to create the story with my writer and producer hats on at the same time. I don’t want to go into a creative zone that can’t be shot and needs to be redone. That’s a waste of time for the project, and devastating for me personally. (laughing) I try to avoid that.

KG: And of course, constraints are often opportunities to exercise our creativity in different ways, and perhaps discover new approaches and solutions. Spielberg’s use of POV shots and a compelling musical cue in place of the broken mechanical shark from JAWS is one of the most famous examples. I could sense places in Calling where you were probably working around some issue.

JC: Yes, but you have to remember that it takes dozens of “beautiful mistakes” to get that one memorable breakthrough. (laughing)

KG: (laughing) I’m sure. But all things considered, creating a feature-length VR film is a pretty remarkable breakthrough.

CZ: Thank you. It’s just the beginning.

Following are comments that director Charles Zhang made on VR production following the premiere of Calling at the Beijing Film Academy on November 2nd, 2018...

Introduction

The history of cinema spans more than 100 years since its invention. Science and technology drive the evolution of the film. When the Lumiere brothers presented their first film, Arrival of a Train, in France in 1895, the audience was shocked by the images projected on the screen for the first time. VR film is not only a combination of computer technology and art but also a change in the way of watching movies and the language of movies. Audiences can fully immerse themselves in a virtual reality scene, experience stories, explore plots, communicate online and even conduct cross-regional exhibitions through VR technology. On the other hand, the new cinematic language and shooting methods will promote the development of film technology.

VR breaks the fourth wall

The rectangular screen has been presented to the audience as a picture frame for more than 100 years. Both paintings and videos are presented in this picture frame, also on phone screens and computer screens. Breaking the frame seemed impossible until virtual reality came along and changed everything. VR combines computer technology, visual art and multimedia technology to create a new mode of media communication. VR recreates a virtual space where viewers can interact with the scene or even the characters. Viewers need to wear HMD devices to immerse themselves in the space as if living in a virtual world. In this space, the picture frame no longer exists and the relationship has changed between actors and viewers: the audience is part of the space.

Recently, a film by Steven Spielberg, Ready Player One, showed not only the change in the way of watching movies in the future world but also the concerns of VR as a new generation of media. Regardless, the VR era has set off a boom in the film industry around the world. Many experts believe that VR film breaks the “fourth wall” theory in drama: the audience is no longer a passive viewer, is free to choose what they want to view, and the boundary between the audience and the performer has disappeared. Therefore, many scholars believe that VR film will bring a revolution in filmmaking and cinematic language. Our research is focused on how to develop a new cinematic language to help VR filmmakers.

Immersive theater

Immersive theater is a participatory theater based on experience. These experiences often depend on a series of sensory stimuli and some viewing locations, which are usually partly determined by the movements of the audience in space. For example, Punchdrunk’s immersive drama Sleep No More further developed the theory of environmental theater. The characteristic of immersive drama is that the viewer watches the performance while wearing a mask onstage, but the number of viewers is limited.

Introduction of VR feature film

On Calling, our purpose was to explore the possibility of shooting a VR feature film, and to explore the development and application of VR cinematic language. The film is a 720-degree panoramic video with a total length of 70 minutes. The story is about a middle-aged man’s emotional entanglements between reality and illusion.

Shooting method

A VR shot presents a 720-degree real environment, so it is entirely different from the traditional shooting method.

Firstly, the production process is different. VR film employs a panoramic camera with six lenses. Each lens records images from different angles separately, and each image is distorted. Only after stitching the six images can the director view the VR shot, which makes real-time monitoring almost impossible. Although some panoramic cameras offer wireless surveillance technology, it does not work in certain environments. For example, in our film, there is a scene in the underground garage where many barriers make the camera unable to transmit a signal. Without a monitor, it is a very big challenge for us to direct the actors’ performances.

The second challenge is how to hide the crew. Because the panoramic camera may see the crew, it requires each of us to find a place to hide before turning on the camera.

The third challenge is hiding the lights and sound recording equipment. These devices are likely to be exposed to the lens, so choosing real lighting equipment on the scenes or changing lighting equipment is a feasible approach. For the sound recording, we decided to use both live recording and post recording, which requires a lot of time and cost.

When all the filming is done, the more difficult part is the post-production. The challenge at this stage is unimaginable. The main issue is stitching. We need to stitch each VR lens - six separate videos - which extends the time of post-production. However, the current stitching technology is not ideal enough. Although we use the most advanced optical stitching technology, some ghosting still exists due to light intensity and angle problems.

Focus area

Because there is no monitoring, the director must design and rehearse the actors’ blocking carefully before shooting. According to the focus area principle I proposed, we could plan the placement of the VR camera very well. We designed the position of lighting and the camera in panoramic mode. However, one day when we were ready to shoot, the site was unexpectedly unavailable. So, we had to replace the planned site with a new site, forcing us to revise the previous design.

360-degree principle

A circum-viewing shot places the camera within a 360-degree circular region. The placement of the camera establishes a spatial boundary, while the circum-viewing shot and the center shot jointly establish an immersive experience for the participants.

The center shot can also be called the participation shot. The camera is positioned in the center of the shooting area, in the center of the performing area or the center of the event rather than the center of physical space.

Virtual frame and spatial composition

In our shot design, we applied the virtual frame and spatial composition. The virtual frame is an invisible frame that can help viewers focus on specific areas in VR environments. Spatial composition rearranges objects in space in order to effectively convey visual information to the audience.

POV

POV is usually associated with eyeline-matching shots, immersing viewers in the space. The aim is to confirm the viewer’s identity and position in VR space through the first-person POV in VR film.

VR long takes

The long shot records the whole process of an event, which can immerse the viewer as an onlooker in a VR environment. The movement of the characters is the point of interest (POI) for a long take of narrative. The POI in a VR shot, and the related character performances, need to be designed and rehearsed in advance because the long take cannot be cut. Otherwise, continuity will get lost. The POIs should also be organized in a definite order, and this order should be reasonable. For example, if the actress is immediately murdered by a man when she enters a room, the tension will disappear. However, if for narrative purposes, when the actress enters the bathroom, there is time for viewers to observe the whole room, then the viewers will discover the man hiding under the bed. However, this process is not enough to provoke the tension of the viewers, so we have the man move to the bathroom first. As he approaches the bathroom, the actress suddenly screams at a splash of hot water. The man is afraid she will find him, and is forced to hide behind the curtains. The course of action creates a reasonable path for the actress to walk to the window. When the actress approached the window, the man attacks her from behind. Thus, this long take requires a reasonable arrangement for blocking. Although the audience may miss some details based on where they choose to look, they can reconstruct the plot and the logic.

Spatial transition

We use the corner of the scene to realize spatial transitions. In addition, the fade-in and fade-out technique of traditional films can be used for transitions. At the same time, intensity of the light can draw the audience’s attention, allowing the viewer to shift focus from one point to another.

Flashback / flash-forward

I also experimented with time change and color contrast. Flashbacks are reconstructions of memory. The colors represent different fragments of memories and emotions. Flash-forward is a form of imaginative fantasy about the future situation.

Spatial montage

Next is the editing. I have tested several editing softwares, and finally used the latest cuts of Premiere Pro and After Effects. They already possess some VR editing functions and special effects, which can solve fundamental technical problems. However, the challenge in VR is not editing software but editing theory. Some question whether traditional editing and montage techniques can work on VR movies. We did some experiments with VR editing, and spatial montage was used in this movie. Spatial montage came from the new media language of Manovich, which proposed that spatial montage combine multiple images simultaneously in one space. We experimented with his ideas and developed them in VR.

Spatial narrative

VR film is more inclined towards spatial narrative, which comes from environmental theatre. We can find similarities in immersive theatre works such as Sleep No More. However, the difference is that the VR audience does not require as much story structure and character interaction as in an immersive theatre drama, nor does it have restrictions on venue and staging. During our VR shoot, we adopted a micro narrative method, and integrated fragments of plot into the film.

Duration

Finally, I would like to talk about the viewing time. Normally, a VR movie is from 5 minutes to 40 minutes in length. Many filmmakers believe that it is difficult to make a feature-length VR film. We admit that this was indeed a challenge. Through research, experimentation and filming, we arrived at ideas such as focus area, virtual framing, spatial continuity, spatial montage and so forth, which can assist directors of VR films.

Kevin is the author of AWN's Reality Bites blog, his musings on the art, technology and business of immersive media (AR, VR, MR) and AI. You can find Kevin's website at www.kevingeiger.com and he can be reached at holler@kevingeiger.com.