How to Use Prompt Travel

Hello everyone, 2023 is almost over.
I wish you all a happy and healthy 2024, and today, I want to introduce Prompt Travel!

Prompt Travel

Prompt traveling is a technique that fine-tunes your positive prompt to control specific details at designated time frames within the final GIF or Video creation. With AnimateDiff, we can influence the visual aesthetics of our GIFs or videos, including style, background, clothing choices, and many other elements.

Prompt Travel Banner

1. Requirements

  • Stable Diffusion / Automatic1111
  • AnimateDiff Extension
  • AnimateDiff Model

1) Installing Stable Diffusion

First, to start today’s Prompt Travel, you need to download Stable Diffusion’s Automatic1111. If you’ve used AI Art Generate programs before, this might be familiar, but for new users, check the link below for installation instructions.

–>Go to Post (How to Install Stable Diffusion)

2) Installing AnimateDiff Extension

If you’ve installed Stable Diffusion, as shown below, enter the Git URL to install the AnimateDiff extension.

–> Github Link: https://github.com/continue-revolution/sd-webui-animatediff

Installing Extension by Url
Installing Extension by Url

Once the extension is installed, click the Reload UI button and restart the Automatic1111 window.
You should see an AnimateDiff tab in the lower left corner of the Txt2Img tab.

3) Installing AnimateDiff Model

To fully utilize the AnimateDiff extension, you need a dedicated model for it. Download the model from the link below and place it in the
Stable Diffusion > Extensions> AnimateDiff > Models folder.

–> Go to Link (Download AnimateDiff Model)

AnimateDiff Module
AnimateDiff Module

2. How to Use Prompt Travel

Now that you’re ready, enter the familiar Txt2Img tab and write your desired Positive Prompt.
If you find this difficult, feel free to follow along with my example.

ex) women, looking at viewer, red dress, upper body, golden earring

After writing the prompt, press Enter and write what you want for specific frames.

Frame refers to the number of images that make up 1 second of a video. For example, setting 30 Frames per second means 30 images per second. High-speed cameras use thousands of frames per second, enabling slow-motion in broadcasts.

Set Frame and video length in the AnimateDiff tab. Following my settings:

  • Positive Prompt
    women, looking at viewer, red dress, upper body, golden earring
    0: (smile:1.2)
    15: (angry:1.2)
    30: (sad:1.2)
  • Negative Prompt
    worst quality, low quality, multiple views, watermark, letterbox

Other basic settings are shared below:

  • Checkpoint: Dreamshaper
  • Sampling: DPM++2M Karras
  • Sampling Steps: 30
  • Upscaler: 4x-Anime6B
  • Upscale by: 2
  • High-res steps: 14
  • Denoising: 0.35
  • Size (Width & Height): 352 & 624
  • CFG Scale: 7
  • Seed: -1
Settings
Settings

Finally, open the AnimateDiff tab and set up as shown below.

Basic UI from AnimateDiff
Basic UI from AnimateDiff

First, select the Motion Module on the top left and choose the downloaded model(mm_sd_v15_v2.ckpt)
Then set the total Frame count to 40 and FPS to 10.
An FPS of 10 means 10 images per second, and with 40 total Frames, the video will be 4 seconds long.

Choose ‘A’ in the Closed Loop setting to make the first and last frames flow naturally.

Finally, select Enable AnimateDiff and go back up to press the Generate button.

Settings for AnimateDiff
Settings for AnimateDiff

After setting all options as shown, make sure to check the Enable button.

By default, the GIF creation is checked, so it’s fine, but if you want to export as a video, you can also check the MP4 option.
After clicking the Generate button, it will take some time depending on your hardware specifications. Take a break while you wait~!

3. Output

AnimateDiff Prompt Travel
AnimateDiff Prompt Travel

Ta-da~!! Finally, the results are out. Isn’t it incredible?
We can see that the smiling face at 0 Frame, the angry expression at 15 Frame, and the sad look at 30 Frame, all mentioned in our Prompt, have been well reflected. The ability to create our own animation through a series of continuous Prompts is truly fascinating.

We used facial expressions for variations, but more complex scenarios, like a running dog, are also possible. So try it out now! Next time, I’ll share more interesting information. You’ve all worked hard this year.

See you next year~!!”

1 thought on “How to Use Prompt Travel”

Leave a Comment

Your email address will not be published. Required fields are marked *

Shopping Cart