– Stability AI is previewing a new generative AI called Stable Video Diffusion that creates short-form videos with a text prompt.
– Stable Video Diffusion consists of two AI models, SVD and SVD-XT, capable of creating videos at a resolution of 576 x 1,024 pixels.
– Users can customize the frame rate speed between three and 30 FPS.
– The length of the videos depends on the model chosen, with SVD playing for 14 frames and SVD-XT playing for 25 frames.
– Rendered clips will play for about four seconds before ending.
– The quality of the videos is high, with impressive details and visuals demonstrated in the Ice Dragon demo.
– Stable Video Diffusion has limitations, including difficulty achieving perfect photorealism, generating legible text, and rendering faces accurately.
– The project is in the early stages and not intended for real-world or commercial applications at this time.
– Interested users can join a waitlist to try out Stable Video Diffusion and access a Text-To-Video interface.
– The project’s white paper mentions using publicly accessible video datasets for training, potentially raising concerns about data scraping.
– No specific launch date for Stable Video Diffusion has been announced.
– Other AI video makers are available as alternatives.
Stability AI, the developer behind the Stable Diffusion, is previewing a new generative AI that can create short-form videos with a text prompt.
Aptly called Stable Video Diffusion, it consists of two AI models (known as SVD and SVD-XT) and is capable of creating clips at a 576 x 1,024 pixel resolution. Users will be able to customize the frame rate speed to run between three and 30 FPS. The length of the videos depends on which of the twin models is chosen. If you select SVD, the content will play for 14 frames while SVD-XT extends that a bit to 25 frames. The length doesn’t matter too much as rendered clips will only play for about four seconds before ending, according to the official listing on Hugging Face.
The company posted a video on its YouTube channel showing off what Stable Video Diffusion is capable of and the content is surprisingly high quality. They’re certainly not the nightmare fuel you see on other AI like Meta’s Make-A-Video. The most impressive, in our opinion, has to be the Ice Dragon demo. You can see a high amount of detail in the dragon’s scales plus the mountains in the back look like something out of a painting. Animation, as you can imagine, is rather limited as the subject can only slowly bob its head. The same can be seen in other demos. It’s either a stiff walking cycle or a slow panning shot.
In the early stages
Limitations don’t stop there. Stable Video Diffusion reportedly cannot “achieve perfect photorealism”, it can’t generate “legible text”, plus it has a tough time with faces. Another demonstration on Stability AI’s website does show its model is able to render a man’s face without any weird flaws so it could be on a case-by-case basis.
Keep in mind that this project is still in the early stages. It’s obvious the model is not ready for a wide release nor are there any plans to do so. Stability AI emphasizes that Stable Video Diffusion is not meant “for real-world or commercial applications” at this time. In fact, it is currently “intended for research purposes only.” We’re not surprised the developer is being very cautious with its tech. There was an incident last year where Stability Diffusion’s model leaked online, leading to bad actors using it to create deep fake images.
If you’re interested in trying out Stable Video Diffusion, you can enter a waitlist by filling out a form on the company website. It’s unknown when people will be allowed in, but the preview will include a Text-To-Video interface. In the meantime, you can check out the AI’s white paper and read up on all the nitty gritty behind the project.
One thing we found interesting after digging through the document is it mentions using “publicly accessible video datasets” as some of the training material. Again, it’s not surprising to hear this considering that Getty Images sued Stability AI over data scraping allegations earlier this year. It looks like the team is striving to be more careful so it doesn’t make any more enemies.
No word on when Stable Video Diffusion will launch. Luckily, there are other options. Be sure to check out TechRadar’s list of the best AI video makers for 2023.
You might also like
AI Eclipse TLDR:
Stability AI, the developer behind Stable Diffusion, has unveiled a new generative AI called Stable Video Diffusion. This AI can create short-form videos based on a text prompt. The system consists of two AI models, SVD and SVD-XT, and can generate clips at a resolution of 576 x 1,024 pixels. Users can customize the frame rate speed, ranging from three to 30 frames per second. The length of the videos depends on the selected model, with SVD playing for 14 frames and SVD-XT for 25 frames. However, the rendered clips only play for about four seconds before ending.
Stability AI has showcased the capabilities of Stable Video Diffusion through a video on its YouTube channel. The content produced by the AI is of high quality, unlike the disturbing results from other AI models like Meta’s Make-A-Video. One standout demo is the Ice Dragon, which exhibits impressive detail in the dragon’s scales and picturesque mountains in the backdrop. However, the animations are limited, often featuring slow movements like a bobbing head or a panning shot.
The AI has some limitations, including an inability to achieve perfect photorealism or generate legible text. It also struggles with rendering faces, although a demo on Stability AI’s website shows that it can render a man’s face without any noticeable flaws, suggesting that performance may vary depending on the case.
It is important to note that Stable Video Diffusion is still in the early stages and not ready for wide release. Stability AI has stated that the AI is intended for research purposes only and not for real-world or commercial applications. This caution stems from a previous incident where the Stability Diffusion model leaked online and was misused for creating deep fake images.
Interested users can join a waitlist on the company’s website to try out Stable Video Diffusion. The preview will include a Text-To-Video interface, although the release date for public availability is currently unknown. Stability AI’s white paper provides more detailed information about the project. The document mentions using publicly accessible video datasets for training, which may raise concerns considering Getty Images’ lawsuit against Stability AI for data scraping allegations earlier this year.
While there is no specific launch date for Stable Video Diffusion, there are alternative AI video makers available. TechRadar has compiled a list of the best AI video makers for 2023.