How to run Image to Video using AnimateDiff?
1. If you haven’t installed AnimateDiff, go to install AnimateDiff from github.
2. Go to hugging face animatediff, download “v3_sd15_adapter.ckpt” and “v3_sd15_mm.ckpt”, and put them at “models\MotionLoRA.” Download “v3_sd15_sparsectrl_rgb.ckpt” and put it at “models\SparseCtrl.”
3. Prepare an image file to animate. The image size can be 256×384. You can also use 256×256 or 512×512. Note the smaller the size, the faster it runs. Put the image file in “_assets_\demos\image.”
4. Under the “AnimatedDiff” directory, go to “configs\prompts\v3”. Open “v3-1-T2V.yaml.” In the file, there are a few sections for different demos.
5. It is easier to run one section at a time. To run Text to Video, use the first section “# 1-animation” and remove others. Save the file with a different name “v3-1-T2V-animation.yaml.” Now open the file and edit it.
6. In the field of controlnet_images, change the path of the image to your file.
7. Change the prompt to describe the scene.
8. Open an Anaconda Prompt. Run command:
>conda activate animatediff
9. In the Anaconda prompt, navigate to the directory “AnimateDiff,” run the yaml file:
>python -m scripts.animate –config configs/prompts/v3/v3-1-T2V-animation.yaml
10. Monitor the progress in the Anaconda console. When it finishes, the result can be found at “samples” directory.
11. Modify your prompts and run again until you find good results.