Meta's new video AI model for dummies: 101 on if it could take on OpenAI's Sora

This week, Meta unveiled a new AI model called 'Movie Gen'. It joins the growing list of AI video platforms that marketers can use in brand films, assets and more such as OpenAI's Sora.

Curious what Meta's Movie Gen is and how it stacks up against other AI video platforms when it comes to the marketing industry? Read on. 

What is Movie Gen?

The new technology can produce custom videos and sounds, edit existing videos, and transform personal images into a unique video through simple text inputs. 

Given a prompt text, Movie Gen can create high-quality and high-definition images and videos. The model also has the ability to generate videos of up to 16 seconds at a rate of 16 frames per second.

Meta expanded the above foundation model to support personalised video generation where it can take a person's image and combine it with a text prompt to generate a video. The generated video will contain the referenced person and visuals informed by the text prompt. 

Don't miss: How AI will impact our planet and how we can embrace green AI

In addition, the editing variant of Movie Gen combines video generation with image editing and will allow users to add, remove or replace elements in the video. It can also change the background or style of the video. 

Finally, the new AI model can take a video and optional text prompts and generate audio up to 45 seconds. This includes ambient sound, sound effects and instrumental background music. 

Furthermore, Movie Gen has an audio extension technique that can generate coherent audio for videos. 

What is Sora?

Sora is OpenAI's solution to getting AI to understand and simulate the physical world in motion, with the goal of training models that help people solve problems that require real-world interaction, it said in a statement when the platform was announced in February this year. 

As such, Sora is a text-to-video model that can generate videos up to a minute long while maintaining visual quality and adherence to the user’s prompt.

Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.

The model has a deep understanding of language which then enables it to accurately interpret prompts and generate compelling characters that express vibrant emotions. Sora can also create multiple shots within a single generated video that accurately depict characters and visual style.

"Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios. We leverage a transformer architecture that operates on spacetime patches of video and image latent codes," said OpenAI.

Could Movie Gen go head-to-head with Sora?

According to experts that MARKETING-INTERACTIVE reached out to, Meta's Movie Gen is positioned as a direct competitor to OpenAI's Sora as both models aim to revolutionise AI video generation.

However, since Sora is not yet publicly available it is hard to do a one-to-one comparison, said Milind, an AI Scientist from Mercedes, who was expressing independent views. 

"Movie Gen appears well-equipped to compete with Sora. It has been many months since Sora was announced and months in the field of AI is years in terms of normal time," he explained. "I am sure that the team working on Sora has continued to refine the model, but it is hard to do a direct comparison as Sora model is not publicly available."

Saying that, based on his experience, Milind said that going on the features, marketers might find Movie Gen more advantageous due to its comprehensive capabilities in video generation, editing, and audio synchronisation.

"Its focus on realism and personalisation aligns well with current marketing trends that prioritise engaging and tailored content," he said. 

Meanwhile, Don Anderson, CEO of Kaddadle Consultancy explained that he personally would not exactly interpret Meta’s announcement of Movie Gen as an intended direct challenge to Sora.

"There’s no question that Sora yielded more immediate awareness and attention when it was announced earlier this year, but marketers should anticipate that every social media platform provider will offer an AI text-to-video solution in order to enhance those relationships and secure more share of wallet and ad spend," he said. 

He added that Movie Gen will enable marketers to produce videos targeted to Meta’s many social platforms – that’s their key advantage over Sora – they have Facebook, Instagram, Reels, Threads and WhatsApp at their disposal, which OpenAI "cannot match in terms of scope".

Anderson explained that depending on the monetisation model Meta opts for and how brands staff or train up internally to manage these tools, it could lead to increased content creation efficiencies for brand marketers, particularly for real time content, and potentially decrease their reliance on outside vendors for production support.

Saying that, it is still too early to tell how Movie Gen and Sora will stack up against each other as workable tools for marketers, as neither has a clear commercial and public release date.

"Based on demo videos released by Meta and OpenAI, and some initial brand tests from the likes of Toys R Us in the US using Sora, there’s no question the two will compete head on," said Anderson.

Anderson was referencing how Toys "R" Us saw its sentiments plummet from 12.2% positive, 13.5% negative and 74.3% neutral to 3.4% positive, 53.4% negative and 43.2% neutral after it released an ad in June that was entirely created by Sora, according to media intelligence firm CARMA at the time

According to CARMA, many netizens expressed "disappointment" and "frustration" with the ad, calling it "soulless" and "cynical", according to its world cloud after the incident.

"As much as OpenAI’s ChatGPT has become one of the most widely used and accepted AI tools for text-based natural language outputs, Meta has an established, global network of brands and agency relationships which its video solution will be well positioned to serve and drive tool adoption," Anderson said. 

Anderson added that he would give Meta’s Movie Gen the advantage over Sora in terms of potential use base, and features notably audio integration and video editing capabilities, particularly the latter where users will be able to modify existing footage with greater ease, including adding or removing elements.

He said:

For marketers and content producers, Movie Gen ought to provide more creative flexibility

What marketers need to know

When it comes to what marketers need to know with regards to the usage or either Sora or Movie Gen, Milind explained that marketers should consider the following when using Movie Gen:

• Quality and realism: Movie Gen produces high-quality videos that are highly realistic and compare well with the offerings of competitors. This tool can help enhance brand storytelling.

• Personalisation: The ability to create personalised content using user-provided images can be leveraged for targeted marketing campaigns.

• Editing capabilities: Marketers can utilise its editing features to tailor content dynamically based on audience feedback or trends.

• Audio integration: The synchronised audio generation can enhance the overall viewer experience, making it suitable for various marketing formats

Meanwhile, Anderson explained that marketers need to take note of copyright compliance as they navigate these new tools. 

"These AI tools are being trained on everyone else’s content. Marketers need to ensure they have their legal teams across anything they produce using Movie Gen or Sora to avoid any potential rights infringement issues," said Anderson. "They should consider employing third-party verification tools to review and valid any AI-supported video outputs and clarify rights across images and audio."

He added that there are also ethical considerations of using these tools, namely brand appropriateness and how the public will respond to the content, noting recent backlash received by Toys R Us with its initial Sora experiment.

"While brands may be excited by possible content cost efficiencies these AI-solutions may yield, do they really want to be associated with the growing concern of widespread creative layoffs in the industry?" he said. 

"These tools may sound great on the surface, but no amount of generative AI is going to replace the human aspect necessary in content creation to ensure that brand messaging connects with consumers in truly meaningful ways," added Anderson.

Artificial intelligence simply cannot do it all – Movie Gen, Sora and others should be viewed as tools to complement the creative process, rather than replacing creators.

True enough, 50% of consumers are able to spot AI-generated copies. In a study by Bynder, it was revealed that millennials were the most successful at spotting non-human content which comes as no surprise as the demographic is also the most likely to use AI when creating content.

Interestingly, the survey also revealed that 56% of participants said that they preferred the AI version over the human-made work. 52% of consumers cited that they would become less engaged if they suspect a copy is AI-generated.

In contrast, participants aged 16 to 24 were the only age group to find the content created by a human more engaging than the AI-generated version (55%).

Anderson added that it’s in the best interest of marketers and video production teams worried about AI affecting their jobs to train up on these tools. "As it has been said, AI isn’t going to take their jobs, it’s those who know how to use AI that will," he said. 

Related articles:
Claude for dummies: 101 on the new AI platform that might challenge ChatGPT
Cracking the code: How women can get ahead in an AI-driven workplace
AI, data, and social commerce set to shape SEA marketing landscape in 2024