Netflix uses AI to push visual effects beyond the green screen

Researchers at Netflix say they may have made the ubiquitous green screen obsolete. Described as an innovative advancement in the application of AI in the film and television industry, the Magenta Green Screen (MGS) harnesses the power of artificial intelligence to enhance visual effects, making them more realistic and accurate in real time.

The streaming giant’s new technology is an impressive addition to its diverse arsenal, which already uses deep learning to tailor the user experience. Using AI, Netflix has made a name for itself by successfully profiling its users and suggesting content to viewers. AI has also helped Netflix create engaging highlights, recaps, and trailers to increase viewership.

Greenscreen technology is a common process in which actors are filmed against a bright green background, which can later be digitally removed and replaced. The process can be automated, but problems arise when green elements appear in the foreground or when fine details such as hair strands or transparent objects are involved. To address these issues, operators often manually adjust settings or paint over each frame, a time-consuming process.

Netflix says MGS uses AI to get better results than old green screens in a fraction of the time it normally takes to edit a scene.

How Netflix uses AI to up its VFX game

The MGS method requires filming actors against bright green LEDs, with red and blue LEDs illuminating them from the front, resulting in a magenta glow. This unique lighting configuration creates separate red, green and blue channels. The green channel captures only the background, making the foreground appear black, while the red and blue channels capture only the foreground, making the background appear black.

The introduction of AI into this system allows the green channel to be replaced in real time, meaning that actors can be placed realistically and directly in the foreground of another scene. What’s even more remarkable is that this AI technology can work accurately with transparent objects and intricate details such as individual strands of hair, which has been challenging with traditional methods.

The application of AI does not stop with filming. Since the MGS technique can result in magenta-toned actors and objects, Netflix also uses AI to restore the full color spectrum to the foreground. This is achieved by using a reference photo of the normally lit actors.

Netflix is ​​just one of many companies riding the AI ​​wave. Other startups and unicorns are using AI technology to better profile users, provide better customer experiences and optimize their products.

If Decrypt recently reported, Zoom has developed a GPT-powered meeting digest. Also, Meta is using generative AI to make better ads, Google is improving its office suite thanks to its own LLM, and even the US Department of Defense is trying to come up with interesting use cases for AI as a strategic tool.

Despite the complexity associated with the MGS technique, Netflix believes in the potential of its AI-powered GS system. However, others in the industry have reservations.

Drew Lahat of video production company Geiger Post expressed concern about the technique’s usefulness in fast-paced shooting environments and its competitiveness with other emerging methods. “It can work well in a fully controlled space,” he said, “but it would have to compete with other new techniques like virtual production stages, and convince producers in real-world scenarios.”

Nevertheless, as Netflix researcher Paul Debevec told New Scientist, “Computers have already provided such powerful tools to make a lot of things easier. [This is] another thing we can make easier so that the talented artists we have can focus on being an artist, actually making things look better.

Netflix is ​​betting on AI to not only improve viewer experiences, but also to streamline things behind the scenes. Writers may hate AI creeping into their businesses, but do VFX artists feel the same way?

Adblock test (Why?)