ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos

💾 VRAM < 10GB             🚀 Infinitely Long Videos            ⭐️ Tuning-free

     

📽️ See more video samples in our project page!

"Impressionist style, a yellow rubber duck floating on the wave on the sunset, 4k resolution.",

VideoCrafter2, 100 frames, 320 X 512 resolution

Clone our repository

git clone https://github.com/yanghlll/ScalingNoise.git
cd ScalingNoise

☀️ Start with VideoCrafter

1. Environment Setup ⚙️ (python==3.9.21 recommended)

conda create -n ScalingNoise python=3.9.21 
pip install -r requirements.txt

2.1 Download the models from Hugging Face🤗

Model	Resolution	Checkpoint
VideoCrafter2 (Text2Video)	320x512	Hugging Face

2.2 Set file structure

Store them as following structure:

cd FIFO-Diffusion_public
    .
    └── videocrafter_models
        └── base_512_v2
            └── model.ckpt      # VideoCrafter2 checkpoint

3.1. Run with VideoCrafter2 (Single GPU)

bash scalenoise.sh

Generic Text-to-video Generation

"A spectacular fireworks display over Sydney Harbour, 4K, high resolution."	"A cute raccoon playing guitar in a boat on the ocean, 4k resolution."

"A cozy, low-poly cabin in the woods surrounded by tall pine trees, with a warm light glowing from the windows and smoke curling from the chimney, 4k resolution."	"Cinematic closeup and detailed portrait of a reindeer in a snowy forest at sunset."

🤓 Acknowledgements

Our codebase builds on VideoCrafter and FIFO-Diffusion. Thanks to the authors for sharing their awesome codebases!