directly image-to-video generation

[Chendaiping]

Thank you very much for your excellent work. I wanted to ask if it would be possible for you to provide code that directly generates videos from images, bypassing the first stage of text-to-image generation. Specifically, can we omit the text-to-image step and proceed directly with image-to-video generation? What would be the differences between this approach and the original method, which first generates an image from text and then generates a video from that image?

I look forward to your response!

[Chendaiping]

Can we consider using both text and images together as inputs to generate videos? Why do we need to use the first stage of text-to-image to generate an image?

[Hiccupwzy]

Same question as you. In my experiment, the process of generating images (first stage) and generating video with the similar motion (second stage) are decoupled. And you can use any prompt in video generating process once you have specify the input images.