Patterns are what lie at the heart of artificial intelligence (AI), which is why businesses love it. Patterns help to predict the future, and predicting the future is a great way to mint money. Although at times it’s ambiguous how these things work together, film studios like 20th Century Fox are using AI to predict what films audiences want to see by partnering with AI tech giants like Google.
Now, to be successful in the movie industry, movie studios need to attract moviegoers, but attracting huge audiences is no easy task. The movie-going audience is a diverse group of people with a wide range of tastes and interests. Conventionally, studios rely on intuition and experience when deciding to invest in a particular script. This is a risky approach, especially when the story is unique or unprecedented. Movie studios have relied in the past on high-level data processes to map out customer segments and make forecasts for the future film’s box office earnings. Also, until recently, granular predictions at the segment level have remained elusive due to the dearth of requisite technological tools and algorithms. However, with the advent of machine learning, these barriers have been overcome. Artificial intelligence (AI) is such a tool helping in this iterative and complex process of matching scripts and audiences much more accurately.
20th Century Fox, one of the most reputable movie studios in the world, is using advanced machine vision systems powered by sophisticated AI algorithms to examine trailer footage and predict the likelihood of audiences watching their movie. They do this investigation frame by frame, labeling important objects and events, and then comparing it to data generated from other film trailers. The core idea is that movies with similar sets of labels/scenes will attract similar sets of audiences. 20th Century Fox has partnered with Google to launch project Merlin—a codename for the movie attendance and prediction system.
How Does It Work?
The first step for the team working on this project was identifying the right technology to power the system. The Cloud Machine Learning Engine (Cloud ML Engine), in conjunction with the TensorFlow deep learning framework, appeared to be the best choice after brainstorming by the team members. Most of the experts on the team voted for it because Cloud ML Engine is a managed service, and can automate all resource provisioning and monitoring. This implies that the team just needs to focus on building the deep learning model for Merlin, rather than devoting time in the arduous process of configuring its infrastructure.
Scanning movie trailer
The first step of project Merlin involves scanning the movie trailer. Consider the example of the movie Logan. Merlin scans the trailer and labels the objects, such as “tree”, “facial hair”, “car”, “forest” etc.
While the first stage involves only scanning for labels and recording the frequency of these labels, the final data generated is much more complex, because the final data also considers how long these objects appear onscreen and specifically when they show up the video.
Labelling the scanned data
The temporal information obtained by the process of scanning and labeling is particularly rich, as it correlates with the film’s genre. For example, a trailer with a long close-up shot of a protagonist is more likely for a drama/romantic movie than for an action blockbuster. Similarly, a trailer with quick and furious shots is more common for an action movie. This surely holds true for Logan, with its trailer featuring plenty of slow shots of Hugh Jackman drenched in blood.
Comparing with other labelled dataset
After completing the label analysis of a movie trailer (Logan in the considered example), the next stage is to compare that new analysis with the labels generated from other movie trailers to identify ‘similar movies’. There’s a presumption that there must be some overlap between the audiences for Logan and other action movies. In other words, moviegoers who have previously seen an action-packed film with a “rugged” male protagonist are inherently more likely to see Logan.
What were the Results?
By comparing this data with analyses of other movie trailers, Merlin tries to predict what films might interest those moviegoers who saw Logan.
The table below shows the top 20 films that moviegoers who went to see Logan also watched. The right column shows Merlin’s predictions, while the left column shows the actual data (collected, based on the survey, i.e., literally asking the people directly).
|Actual Data||Prediction by Merlin||Data Generated by Merlin|
|X-Men: Apocalypse||True||The Magnificent Seven|
|John Wick: Chapter 2||True||Jason Bourne|
|Doctor Strange||True||John Wick: Chapter 2|
|Batman Vs. Superman: Dawn Of Justice||True||Terminator Genisys|
|Suicide Squad||True||The Legend Of Tarzan|
|Dead Pool||False||Mad Max Fury Road|
|Terminator Genisys||True||The Revenant|
|Mad Max: Fury Road||True||Independence Day: Resurgence|
|Captain America: Civil War||False||Rogue One: A Star Wars Story|
|Star Trek Beyond||True||The Hunger Games: Mockingjay-Part 1|
|Independence Day: Resurgence||True||The Accountant|
|The Magnificent Seven||True||Star Trek Beyond|
|Avengers: Age of Ultron||False||Suicide Squad|
|Kingsman: The Secret Service||False||The Martian|
|Split||False||Batman Vs. Superman: Dawn Of Justice|
|Rogue One: A Star Wars Story||True||San Andreas|
|Fantastic Beasts And Where To Find Them||False||Doctor Strange|
|Furious 7||False||Mission Impossible: Rogue Nation|
As seen in the table, Merlin guesses quite a few movies correctly, including other superhero movies like Batman Vs. Superman: Dawn of Justice, X Men: Apocalypse, and Doctor Strange. It even aptly recognizes John Wick: Chapter 2 as a bedfellow of Logan. That’s an impressive result, as John Wick is certainly not a typical superhero movie. However, it does feature a similarly weary protagonist with a rugged look. Merlin identifies all five of the top choices, albeit not in the same order of importance. Overall, it was successful in identifying more than half of the films.
That being said, there were places where Merlin erred. Merlin predicted that The Legend of Tarzan would be a hit with ‘Logan-liking’ audiences, for example. Interestingly, neither Google nor 20th Century Fox offered any explanation for this. Now, my conjecture is that it could it have something to do with the “tree,” “forest,” and “light” found in Logan—elements that the Tarzan trailer also shows in a high concentration.
Movie studios are keen on adopting AI for these sorts of analytics, and several companies are striving hard to design systems that can predict the likelihood of a movie’s success through data mining and processing. With more data and further advances in the field of machine learning, we can expect these systems to get more accurate/efficient, since the accuracy remains a challenge… for now!