Hello everyone, are you having a pleasant day today? Today, I want to introduce the advanced Inpaint Upload technology of Stable Diffusion.
Inpaint Upload
Inpaint Upload is one of the inpainting features, which allows you to use existing separate masking data instead of drawing masks directly with a mouse or tablet in the Web UI. For those who have done masking work even once, you might have already realized the key point of this feature β the ability to achieve precise masking.
Alright then, shall we dive right into the practical application today?
1. Preparation
- Stable Diffusion’s Automatic1111
- Photoshop
- Tablet or Pencil
1) Install Stable Diffusion’s Automatic1111
If you haven’t installed Stable Diffusion’s Automatic yet, you can follow the installation guide at the link below.
–> Go to Post (How to install Web UI Stable Diffusion)
2) Photoshop
The reason we need Photoshop today is to perform precise masking on the produced images. Of course, any platform that allows you to draw is suitable, so please prepare accordingly!
3) Tablet or Pencil
Finally, you’ll need a tablet or a pencil to draw a new picture. Those who are used to digital art on computers or using tablets can continue using their tablets. For those without tablets, you can also print out the image and draw on it with a pencil, so please prepare this as well!
2. Creating the Desired Image
As regular readers of our articles, you are likely familiar with creating images using prompts and models. Try creating a fantastic result first using your favorite base model, the combination of LoRA and prompts, and separate upscale settings. For those who are new or still find it challenging to produce results, you can follow the same process using the settings I provide below.
Settings
- Base Model : hyperV5Anime_hyperModelV6Detail.safetensors [f833ed57a1]
- Positive Prompt : epic realistic, 1man , blond hair, a beard, a tattoo on his neck, in an white t-shirt, upper body, looking at viewer, smiling, black background
- Negative Prompt : worst quality, low quality, badhandv4, watermark, letter box
- Sampling Method : DPM++ SDE Karras
- Sampling Steps : 30
- Upscaler : 4x-UltraSharp
- Upscale by : 2
- Denoising Strength : 0.2
- Size : 512 X 512
- Seed : -1
ControlNet
- Openpose : dw_openpose_full / control_v11p_sd15
- Depth : midas / pytorch_model
For your reference, when using ControlNet, make sure to set the reference photo to the same size as the image generation size you have chosen for Generating. If you apply a mismatched size here, ControlNet might not function properly, so please keep this in mind!
For clear masking and for the tutorial, I used the image of a boxer with a clenched fist in front of their face as a reference for ControlNet.
Well, it turns out I didn’t even need to touch it… the result is already too good, isn’t it?π
Did everyone manage to create results they are satisfied with? Since today we’ll be working on dressing the best image you think in different clothing, please choose the highest quality result you can imagine!
If everyone has achieved satisfactory results, now click on the Img2Img tab and then press the Upload Inpaint tab to drag in the created image file.
3. Preliminary Settings for Inpaint Work
After sending the image to the Img2Img tab, locate it in Stable Diffusion>Output>Txt2Img and then import it into Photoshop. One of the most crucial aspects of today’s task is accurate masking. You can think of the white area as the region where changes are desired, and the black area as the region to be left as is in the original image. We will also now use the pen tool in Photoshop to mark the area and create an accurate masking image.
As you can see, in the original image, I’ve filled only the arm and body parts, excluding the fist, face, and background, with white. This is to recreate these areas with a new prompt. The reason for using a tool like Photoshop for masking is to achieve precise masking, which helps avoid distortion in unwanted areas of the image. It’s important to keep this in mind.
The masking image I just created will be applied in the window below the Upload Inpaint, which reflects the previously created image. Next, you need to draw the outline of the shape you want. This is why I advised preparing a tablet or pencil. Those confident in drawing can apply details finely, and even if you’re not skilled at drawing, a bit of sketching can produce great results through specific functions of ControlNet.
As shown in the picture above, sketch the shape you want to express on the prepared image. What we need to focus on is differentiating between surfaces, so it’s more important to clearly distinguish these surfaces, using colors if necessary, rather than having a precise sketch.
You can draw the picture in any form that pleases you. In my case, I colored in the body’s partitioned sections, attempting to give a feeling of armor. I may not have drawn it well, but I believe the prompt will make many enhancements based on these lines and surfaces.
A tip here: since the reprocessing is based on the lines of the masking image you created earlier, if you want to dress the figure in an outfit that significantly deviates from the body shape, it’s important to ensure enough white area is available.
4. Generate Image with Inpaint Upload
Now that all the preparations are done, shall we apply these images to create a new one?
As shown in the image above, sequentially drag in the existing processed image and the masking image. Modify the prompt slightly so that the desired object appears in the masking area. The settings below can be similar to the initial ones you set.
Now open the ControlNet tab, apply the previously referenced boxer’s image as a Joints for the image, and set the painted image using soft edges, as shown above. Set the depth settings similarly as shown above, and then proceed to reprocess the image.
Ta-da~! Can you see it, everyone? Isn’t it amazing?
As you can see, the ‘golden armour’ prompt has been accurately applied based on the precisely masked white area, and we can see partition lines forming based on our painted picture. What kind of results did you get? I’m confident that with just a little more effort in drawing, you can achieve even more incredible results.
Try it out right now, and start creating various outfits and accessories for your desired characters!