Face animation using an Ai model – GitHub code

Thin-Plate Spline Motion Mode is an Ai model to add movement to a still image based on the driving video. The paper is here. One of its usage is to do face animation using Ai technology.

1. First download the code from Thin-Plate-Spline-Motion-Model GitHub repository. Open a dos prompt, in the directory where you want to install, type the command:
>git clone https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model.git
2. Still in the dos prompt, run the following commands
>cd Thin-Plate-Spline-Motion-Model
>mkdir checkpoints
>pip3 install wldhx.yadisk-direct
>curl -L $(yadisk-direct https://disk.yandex.com/d/i08z-kCuDGLuYA) -o checkpoints/vox.pth.tar
This will download vox.path.tar in the “checkpoints” directory.
3. Now it is time to configure a virtual environment to run the code. If you haven’t installed Anaconda3, go to install Anaconda3.
4. Then you setup a conda environment by the instruction. If you run into problems when you setup, you can download a pre-configured pytorch_env at Gumroad (compatible with CUDA 11.8). Unzip it and put it at your anaconda3 installation under “envs” directory.
5. Prepare your input files. The first one is a short video mp4 file with a face movement. The dimension is 512 x 512. Name it as “driving.mp4.” Put it at “assets” directory.
6. The second file is an image with a face that you want to animate. Note the mouth of the character should be open to show a little bit upper teeth. The dimension is 512 x 512. Name it as “source.jpg” and put it at “assets” directory.
7. Download main.py and put it at “Thin-Plate-Spline-Motion-Model” directory.
8. Open an Anaconda Prompt. Run command:
>conda activate pytorch_env
9. Still in the Anaconda prompt, go to the directory “Thin-Plate-Spline-Motion-Model” and run command:
>python main.py
10. When it finishes, the new video is called “generated.mp4” under “Thin-Plate-Spline-Motion-Model” directory.
11. (Optional) You can enhance resolution of the animation using Real-ESRGan.