If you're using ComfyUI for AI image generation, you've probably run into this problem:

You carefully craft your prompt.
You switch models multiple times.
Yet the result still feels… slightly off.

Want the exact color style from another image?
Want your generated character to actually resemble a reference person?
Or maybe you want to achieve accurate face swapping inside ComfyUI?

That’s where IPAdapter becomes a game changer.

In this guide, we’ll walk through:

• How to install IPAdapter in ComfyUI
• How different IPAdapter models work
• How to build a working face swap workflow
• How to optimize style transfer results
• Common mistakes and practical tips

By the end, you’ll be able to run IPAdapter confidently on your own.

What Is IPAdapter? Why Is It More Precise Than Prompts?

Let’s explain it in simple terms:

Prompts tell the model what to draw with text.
IPAdapter lets an image directly influence how the model draws.

IPAdapter is an image-guided conditioning model. It extracts visual features from a reference image and injects them into the sampling process.

That means it doesn't just “overlay style” — it actively influences generation during sampling.

What Can IPAdapter Do?

• Style transfer
• Multi-image style blending
• Precise face swapping
• Combine with ControlNet for structure + style control

If you frequently generate portraits, the face version of IPAdapter is practically essential.

How to Install IPAdapter in ComfyUI

SEO keyword: IPAdapter ComfyUI installation tutorial

Installation consists of two parts:

1. Node extension
2. Model files

Step 1: Install the IPAdapter Node

Inside ComfyUI:

Manager → Install Custom Nodes → Search “IPAdapter”

Look for:

ComfyUI_IPAdapter_plus

Install it and restart ComfyUI.

If the node doesn’t appear after restart, close and reopen ComfyUI again (cache issue).

Step 2: Download Required Model Files

The node is just the framework — the model files do the real work.

You’ll need two types of files:

1️⃣ IPAdapter Model Files

Place them in:

ComfyUI/models/ipadapter

(Create the folder if it doesn’t exist.)

2️⃣ CLIP Vision Model

Place it in:

ComfyUI/models/clip_vision

Many users miss this step and encounter loading errors.

How IPAdapter Works Inside ComfyUI

The core node is:

IPAdapter Apply

It:

• Receives the base model (checkpoint)
• Receives CLIP Vision
• Receives IPAdapter model
• Receives reference image
• Outputs a modified model to KSampler

Essentially, it injects visual features during sampling.

How to Adjust IPAdapter Parameters

This is what most people care about.

Weight

Controls how strongly the reference image influences the result.

• 0.5 – 0.8 → Natural blending
• Above 1 → Very close to reference, possible distortion

Beginner recommendation: start at 0.6

Noise

Higher values reduce reference influence.

If you want maximum feature retention:

Set it close to 0.01

Weight Type

Common options:

• original
• linear
• channel penalty (more natural for face swap)

For face swapping, channel penalty is usually the best choice.

Which IPAdapter Model Should You Choose?

Common SEO question:

“What’s the difference between IPAdapter models?”

For SD1.5:

• ip-adapter_sd15 → Basic style transfer
• ip-adapter-plus_sd15 → Stronger reference similarity
• ip-adapter-plus-face_sd15 → Optimized for faces
• ip-adapter-full-face_sd15 → Extracts more detailed head features

If your goal is accurate face swapping, go with plus-face.

Face Swapping in ComfyUI Using IPAdapter (Full Workflow)

SEO keyword: ComfyUI face swap tutorial

Here’s the practical logic.

1. Basic Node Setup

You’ll need:

• IPAdapter Apply
• IPAdapter Model Loader
• CLIP Vision Loader
• Checkpoint Loader
• Load Image Node

Connection logic:

Model → IPAdapter → KSampler

2. Choose a Good Reference Face

Best practices:

• Front-facing headshot
• Clear lighting
• No obstruction
• Neutral expression

Higher-quality reference = higher success rate.

3. Add Face Detection (Highly Recommended)

Without masking, the entire image will be influenced.

Proper workflow:

• Use YOLOv8 face detection
• Use SAM for segmentation
• Feed mask into VAE encode

This ensures only the face area is replaced.

4. Recommended Sampling Settings

Stable combination:

• Steps: 30
• CFG: 6
• Sampler: ddim
• Denoise: 0.85

If facial results look unnatural, slightly reduce denoise strength.

Style Transfer with IPAdapter

SEO keyword: ComfyUI style transfer tutorial

IPAdapter is also powerful for artistic style blending.

Suggested approach:

• Generate base structure with a realistic model
• Use an oil painting or illustration reference
• Set weight around 0.7
• Reduce noise below 0.1

You’ll notice:

Composition remains stable, but style shifts clearly.

For better control, combine with ControlNet for pose stability.

Frequently Asked Questions (FAQ)

Q1: IPAdapter node not showing after installation?

Restart ComfyUI.

Q2: Face doesn’t resemble reference?

Increase weight or switch to plus-face model.

Q3: Severe distortion in output?

Lower weight or increase noise.

Practical Tips from Experience

• Generate multiple times and compare seeds
• Reference image quality matters more than parameters
• Different checkpoints produce very different results
• Always use face models for portraits

Why IPAdapter Is a Must-Learn Tool in ComfyUI

If you only rely on prompts, you’re still limited to text-based control.

IPAdapter upgrades image generation from:

Text-driven → Image-driven

For creators working on:

• Portrait generation
• Style recreation
• AI face swaps
• Commercial visual production

IPAdapter is not optional — it’s essential.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:👉 https://market.cephalon.ai/aigc
Join our global creator community:👉 https://discord.gg/KeRrXtDfjt
Collaborate with creators worldwide & get real-time admin support.

ComfyUI is flexible but gets cluttered with repetitive node connections. The Efficiency-nodes extension fixes this by packing multiple functions into 2 core node types (loaders + samplers), cutting workflow complexity and saving time. It also adds features like HD restoration and XY parameter comparison charts.

Key Nodes in Efficiency-nodes

Efficiency Loaders

• What it does: Combines 6+ standard nodes (Checkpoint loader, CLIP encoder, Latent, VAE, prompts) into 1 node.
• Variants: "Efficiency Loader" (basic) + "Efficiency Loader (SDXL)" (for SDXL models).
• Extras: Supports LoRA/ControlNet stacking (easily add multiple LoRAs/ControlNets without messy wiring).

Efficiency Samplers

• What it does: An upgraded K Sampler with built-in VAE decoding + real-time preview (no extra "Preview Image" node needed).
• Variants: "K Sampler (Efficiency)" (basic), "Advanced Efficiency" (more controls), "SDXL Efficiency" (for SDXL).
• Extras: Works with scripts (e.g., HD upscaling) via a "Script" input port.

Quick Guide: Use Efficiency’s XY Chart

The XY Chart helps test 2 parameters (e.g., steps + samplers) at once. Here’s how:

Set up the base workflow

1. Add "Efficiency Loader" (set model/VAE/prompts) + "K Sampler (Efficiency)" (keep default).
2. Connect their ports.

Add the XY Chart node

1. Link "Efficiency Loader"’s "Dependency" output to XY Chart’s "Dependency" input.
2. Link XY Chart’s "Script" output to the sampler’s "Script" input.
3. Set "Spacing" (image gap) to 5; set "Image Output" to "Plot".

Add parameter nodes

1. Add "Steps" node: Set "Count=4", "Start Step=5", "End Step=20" (test 5/10/15/20 steps).
2. Add "Sampler Scheduler" node: Set "Input Count=4", pick 4 samplers (e.g., euler, dpm_sde, dpmpp_2m, lcm).
3. Link "Steps" to XY Chart’s X input; link "Sampler Scheduler" to XY Chart’s Y input.

Generate & compare

1. Click "Add Prompt to Queue" — the XY chart will show results for all parameter combinations.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:👉 https://market.cephalon.ai/aigc
Join our global creator community:👉 https://discord.gg/KeRrXtDfjt
Collaborate with creators worldwide & get real-time admin support.

For anyone using ComfyUI to create AI images, managing and optimizing complex workflows can be a real challenge. Enter a powerful solution: ComfyUI-Impact-Pack. This tool automates many advanced yet tedious image processing tasks, making it perfect for beginners looking to boost both efficiency and output quality.

Overview & Installation

The ComfyUI-Impact-Pack is a free, custom node package built specifically for ComfyUI. Its main appeal is offering plug-and-play intelligent modules. You can achieve sophisticated results without building complex logic from scratch, including:

• Automatic Face Detection & Restoration: Precisely locates faces in images and enhances their details.
• Advanced Mask Generation: Intelligently identifies object outlines to create perfect masks for inpainting.
• Detail Refinement: Performs high-quality, iterative redraws on specific small areas like faces or hands.

Installation via Manager
The process is straightforward and happens entirely within ComfyUI:

1. In the main ComfyUI interface, click the "Manager" button at the bottom right.
2. In the pop-up window, navigate to the "Install Nodes" tab.
3. Type "Impact" into the search box at the top right and press Enter.
4. Find "ComfyUI Impact Pack" in the results list and click the "Install" button on its right.
5. Once installed, restart ComfyUI. The new nodes will be available under the "Impact" category in the node menu.

Core Features: The Three Detectors

The intelligence of Impact-Pack comes from its built-in detectors. Understanding them helps you use the toolkit more effectively.

BBOX Detector (Draws a Box)

• What it does: Quickly identifies the location of specific targets (like a face or hand) by drawing a rectangular bounding box around them.
• Common use: Employ the bbox/face_yolov8m.pt model to swiftly find all rectangular face regions in a picture.

Segm Detector (Traces the Outline)

• What it does: More precise than BBOX, it traces the target's complete contour to generate a shape-fitting mask.
• Common use: Use the segm/person_yolov8n-seg.pt model to get an accurate outline of a person in the scene, not just a box.

SAM Detector (Adds Fine Detail)

• What it does: This is a powerful segmentation AI capable of generating masks with extremely rich detail. It's typically used in tandem with the BBOX detector: first, BBOX quickly locates the general target area, then SAM performs refined segmentation within that area to produce a perfect, sharp-edged mask.

Practical Application: Automatic Face Detection & Refinement

This is one of Impact-Pack's most popular features, fully automating the enhancement of facial quality in portraits.

We'll use the "Face Detailer" node as the core. This powerful node integrates a complete pipeline—detection, cropping, redrawing, and compositing—within itself. You simply connect it like you would a regular sampler.

Simple Workflow Setup:

Place the Core Nodes:

• Add the "Face Detailer" node from the menu.
• Add a "Load Image" node and upload a portrait.
• Add a "Checkpoint Loader" and choose a suitable portrait model (e.g., majicmixRealistic_v7.safetensors).
• Add two "CLIP Text Encoder" nodes. It's good practice to rename them "Positive Prompt" and "Negative Prompt". Input simple quality prompts, e.g., Positive: "masterpiece, best quality, portrait"; Negative: "blurry, deformed".

Connect the Detection Module:

• Add a "Detector Loader" node. In its "model_name" dropdown, select bbox/face_yolov8m.pt (the face detection model).
• Add a "SAM Loader" node, leaving all parameters at their defaults.

Now, connect the outputs from all the above nodes to the corresponding inputs on the "Face Detailer" node:

• Connect image, model, CLIP, VAE, positive/negative as you normally would.
• Connect the output of the "Detector Loader" to the bbox_detector input.
• Connect the output of the "SAM Loader" to the sam_model_opt input.

Key Parameter Settings:
Adjust these few key parameters in the "Face Detailer" node for good results:

• guide_size: Set to 384. This means detail enhancement triggers automatically when a detected face region is smaller than 384 pixels.
• denoise: Set to 0.5. This controls the strength of redrawing; 0.5 balances detail restoration and preserving the original look well.
• feather: Set to 5. This creates a more natural transition between the repaired area and the original image, avoiding hard seams.
• Check the boxes for "Generate Mask Only" and "Force Inpaint" to ensure processing is focused only on the facial area.

Generate & Preview:
Finally, connect the output image from the "Face Detailer" node to a "Preview Image" node. Click "Add Prompt to Queue", and after a moment, you'll get a refined image where facial details (like skin texture and eye clarity) are noticeably enhanced.

Conclusion

Think of ComfyUI-Impact-Pack as a smart assistant added to your ComfyUI workflow. By packaging complex algorithms, it simplifies professional-level image processing tasks—like face refinement and intelligent masking—that would otherwise require multiple steps.

For beginners, starting with the "Face Detailer" function is recommended, as it most directly demonstrates the extension's value. As you grow more comfortable, you can explore its other detectors and tools to gradually build more automated and powerful image processing pipelines.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:👉 https://market.cephalon.ai/aigc
Join our global creator community:👉 https://discord.gg/KeRrXtDfjt
Collaborate with creators worldwide & get real-time admin support.

If you're already using Stable Diffusion to generate images and suddenly find yourself thinking, “What if I could make these characters move?” — then you’re ready to explore AnimateDiff.

Despite its technical-sounding name, AnimateDiff is surprisingly easy to understand:
It turns static images into animations.
No extra training, no model conversion — simply plug it into your ComfyUI workflow and it starts working immediately.

This guide walks you through installation, setup, workflow structure, recommended parameters, and troubleshooting tips.

1. What Is AnimateDiff and Why Is It Popular?

Stable Diffusion has evolved rapidly over the past few years. With tools like LoRA and DreamBooth, generating high-quality static images has become simple.
The problem is that those images don’t move.

That’s where AnimateDiff comes in.

🔥 Its core function: Make your model “move.”

Key Advantages:

Model-agnostic: Most T2I models can be animated directly

Natural temporal consistency: No “every frame looks different” issue

Frame interpolation and unlimited sequence length

Compatible with ControlNet and standard sampling workflows

Creator-friendly: Keep using your existing LoRAs, prompts, and checkpoints

AnimateDiff adds motion to your current models without changing how you work.

2. Installing AnimateDiff in ComfyUI

AnimateDiff works as a node extension. Follow this sequence to install it correctly.

2.1 Install AnimateDiff Node Extension

Open ComfyUI

Open the Manager panel (bottom-right corner)

Click Install Node

Search for AnimateDiff

Select AnimateDiff-Evolved and install it

AnimateDiff-Evolved is the most actively maintained and recommended version.

2.2 Download at Least One Motion Model

AnimateDiff requires a motion model .ckpt file to function.

Popular models include:

mm_sd_v14

mm_sd_v15

mm_sd_v15_v2 ← most recommended

v3_sd15_mm

Place the files inside the proper models folder under the AnimateDiff extension directory.

3. The Core Node: Dynamic Diffusion Loader

All AnimateDiff workflows rely on:

Dynamic Diffusion Loader
Path: New Node → AnimateDiff → Gel → Dynamic Diffusion Loader

This node injects motion logic into the generation pipeline.

3.1 Input Ports (Simplified Explanation)

Model

Must use SD1.5 models

SDXL is currently unsupported

Context Settings

Required if generating beyond default frame length

Without it, V2 motion models enforce a 32-frame limit

Dynamic LoRA

Optional; adds extra style or motion characteristics.

AD Settings

Advanced parameters; safe to ignore for beginners.

Sampling Settings

Controls interaction with the sampler.

AD Keyframes

Used for advanced keyframe animation workflows.

3.2 Node Properties

Model Name: Select a motion model such as mm_sd_v15_v2.ckpt

Scheduler: Recommended → slerp or slerp_linear

Dynamic Scale: Motion intensity

<1: Smoother

1: Stronger motion

Use V2 Model: Enable only if using a V2 motion model

The node’s Model Output connects directly to KSampler.

4. Required Components for Exporting Video

AnimateDiff outputs image frames. To convert those frames into video, install:

4.1 videoHelperSuite

Install via Manager.
Adds a “Merge to Video” node supporting common formats.

4.2 FFmpeg (System-Level)

FFmpeg is required for video encoding.

Installation steps:

Download FFmpeg

Extract the files

Add the bin directory to your system’s PATH

Ensure no Chinese characters exist in the file path

Without FFmpeg, ComfyUI cannot render animations into video.

5. Example Workflow (Anime-Style Demonstration)

If you understand text-to-image workflows, AnimateDiff setup is simple.
You’re essentially inserting a few additional nodes.

Recommended Workflow Structure:

Start with a standard T2I workflow

Add Dynamic Diffusion Loader

Add Dynamic Diffusion Context Option

Connect Checkpoint Loader → Loader Model Input

Connect Context Option → Loader

Loader Model Output → KSampler

Add VAE Decoder → Merge to Video

Feed frames into the video node

This produces a complete animation.

6. Recommended Parameters for High-Quality Results

6.1 Checkpoint & Prompts

Checkpoint: counterfeitV30 or anime-style equivalents

Positive prompts: Character + scene description

Negative prompts: Remove blurriness, deformities, etc.

6.2 Dynamic Diffusion Loader

Motion Model: mm_sd_v15_v2.ckpt

Scheduler: slerp_linear

Dynamic Scale: 1

Use V2 Model: Enabled

6.3 Context Options

Length: 16

Step: 1

Overlap: 4

Loop Context: Off

6.4 KSampler

Steps: 20

CFG: 7

Sampler: dpmpp_2m

6.5 Video Settings

Batch Size: Depends on frame count
(Example: 48 frames ≈ 6 seconds @ 8fps)

Format: mp4mpeg4

Resolution: 512×512

7. Generate Your First Animation

After confirming your settings, click Generate.
AnimateDiff will begin producing frames one by one.

When finished, you’ll notice:

Natural blinking

Subtle body movement

Gentle tilting

Smooth motion transitions

The difference from a static image is dramatic.

8. Practical Tips for Better Animations

Start small: Long clips consume a lot of VRAM

Try different motion models: Each has a unique motion signature

Mind your hardware: Low VRAM = more crash risk

Learn from the community: Users often share LoRAs and workflows

9. Troubleshooting Checklist

If your workflow isn’t running:

Did the Loader detect your motion model?

Is the file path correct?

Is videoHelperSuite installed?

Is FFmpeg properly added to PATH?

Are you using an SD1.5 checkpoint?

Most issues come from these points.

Conclusion

AnimateDiff brings a new dimension to ComfyUI:
From “creating static images” to “making characters move.”

While the nodes may seem complex at first, once you produce your first animation, it becomes clear why AnimateDiff is so popular.
From there, you can explore longer scenes, cinematic shots, and multi-character interactions.


Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:👉 https://market.cephalon.ai/aigc
Join our global creator community:👉 https://discord.gg/KeRrXtDfjt
Collaborate with creators worldwide & get real-time admin support.

High-definition upscaling is a frequent requirement in image processing. While ComfyUI offers various methods for enlargement, many require complex node setups or yield limited results. The SD UpScale extension provides a more efficient solution by achieving high-quality upscaling through tiled processing, which saves both time and storage space. This guide will cover the working principles of SD UpScale, its installation process, a detailed explanation of its node parameters, and a practical case study.

How SD UpScale Works: Why It's Ideal for HD Upscaling

The core concept behind SD UpScale is to segment the image into multiple smaller blocks, upscale each block independently, and then seamlessly stitch them back together to form the complete, enlarged image. This method offers two main advantages:

• Preserves Image Quality: Local optimization prevents overall image distortion.
• Efficient Processing: Tiled processing reduces memory consumption and speeds up the generation time.It's important to note that tiling and re-painting blocks can sometimes lead to incoordination between adjacent areas. Therefore, adjusting the tile size and parameter settings is crucial to ensure global consistency.

Installing the SD UpScale Extension

SD UpScale is an extension node for ComfyUI. Follow these steps to install it:

• Open the ComfyUI interface and click the "Manager" button in the lower right corner.
• In the pop-up window, click "Install Custom Nodes," and then enter "Upscale" in the search bar on the top right.
• Locate the "SD Upscale" extension in the list and click the "Install" button.
• After the installation is complete, restart ComfyUI to start using it.Once installed, you can find the node under New Node → Image → Upscale → SD UpScale.

Case Study: Using SD UpScale to Enlarge a Realistic Portrait

The following steps demonstrate how to set up an SD UpScale workflow to enlarge a realistic-style portrait, assuming you have a base image ready.

Step 1: Set Up the Base Workflow
Load a Text-to-Image workflow and create a new "SD UpScale" node. Connect the following nodes to the SD UpScale node:

• The "image" output port of the "VAE Decode" node $\rightarrow$ SD UpScale node's "image" input port.
• The "model" output port of the "Checkpoint Loader" node $\rightarrow$ SD UpScale node's "model" input port.
• The "conditioning" output port of the positive and negative prompt nodes $\rightarrow$ SD UpScale node's "positive" and "negative" input ports.
• The "VAE" output port of the "Checkpoint Loader" node $\rightarrow$ SD UpScale node's "VAE" input port.

Step 2: Add Upscale Model and Preview
Create an "Upscale Model Loader" node and connect its "upscale_model" output port to the SD UpScale node's "upscale_model" input port. Then, create a "Preview Image" node and connect the SD UpScale node's "image" output port to the Preview node's "image" input port.

Step 3: Select Model and Prompts
Since we are processing a realistic portrait, select a realistic-style model in the Checkpoint Loader (e.g., realisticVisionV50).

• In the Positive Prompt box, input quality descriptions such as "best quality, masterpiece, white dress, lake, upper body."
• In the Negative Prompt box, input phrases to avoid low quality, such as "lowres, text, error, blurry."
  

Step 4: Set Base Parameters
In the "Empty Latent" node, set the initial size of the generated image (e.g., $512 \times 768$) and set the batch size to 1. In the "Upscale Model Loader" node, select an upscale model (e.g., ESRGAN_4x.pth).

Step 5: Configure the K Sampler
In the K Sampler, set the seed to 0, the operation after running to "randomize," steps to 25, CFG to 7, sampler to "dpmpp_2m," scheduler to "karras," and denoise to 1.

Step 6: Adjust SD UpScale Node Parameters
In the SD UpScale node, set the following parameters:

• Upscale Factor: 2
• Seed: 0
• Steps: 25
• CFG: 7
• Sampler: dpmpp_2m
• Scheduler: karras
• Denoise: 0.2
• Mode Type: chess
• Tile Width: 512
• Tile Height: 768
• Bluring: 12
• Tile Padding: 32
• Seam Fix Mode: None (Keep other seam parameters at default)

Step 7: Generate the Image
Click the "Queue Prompt" button and wait for the processing to finish. The generated image will be displayed in the preview node, featuring clear person details without any visible seams or artifacts.

Summary: Advantages and Applicable Scenarios of SD UpScale
SD UpScale is a powerful upscaling tool in ComfyUI, particularly suited for processing high-definition portraits, anime, or landscape images. By using tiled processing, it maintains detail richness while increasing resolution. For new users, the key is to adjust the tiling parameters and prompts based on the image type. As ComfyUI continues to update, SD UpScale may receive further optimizations; it is advisable to follow the official extension repository for the latest features.With this guide, you can quickly get started with SD UpScale and apply it to your daily image processing to achieve high-quality upscaling results.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

In the process of image generation and manipulation, a common challenge arises: when using the "Img2Img" function and increasing the Denoising Strength to enhance details, the image content can often change unpredictably. The Tile Model is precisely the tool designed to solve this problem. It is capable of optimizing image details while maintaining the stability of the overall composition, making it ideal for high-definition image upscaling and detail restoration. This tutorial will introduce the fundamental principles of the Tile Model and provide a step-by-step operational guide to help beginner users easily set up a Tile workflow in ComfyUI.

Tile Model Overview: Why It's Perfect for Detail Restoration

The core advantage of the Tile Model lies in its block-processing mechanism. It avoids the destruction of the overall structure by segmenting the image into multiple small blocks and optimizing the details within each block sequentially. This means that even when upscaling an image to a significantly higher resolution, the Tile Model can enhance details while ensuring the coherence of the original content remains intact.For instance, if you have a blurry image and use the Tile Model to boost its resolution from a base value to a higher one, you will notice a significant enhancement in image details (such as textures and edges), yet the main structure remains unchanged. This characteristic makes the Tile Model an ideal choice for processing anime, photographs, or artistic images.

Deep Dive into the Tile Preprocessor: Key Parameter Explanation

The Tile Model utilizes only one preprocessor: the "Tile Preprocessor" (Tile Tiled Preprocessor). This node contains two main components:

Iterations: Controls the degree of blur applied to the image. A higher value results in a blurrier image. It is generally recommended to set this to 1 to prevent over-processing.

Resolution: Sets the reference resolution for the processing, which is typically set to match the height of the uploaded image.Tip for Usage: Adjust the Iterations based on the image type. For scenarios requiring slight blurring, you might try a higher value, but usually, keeping it at 1 is sufficient.

Setting Up the Tile Workflow: A Step-by-Step Guide

The following detailed steps for setting up a Tile workflow in ComfyUI are suitable for beginner users. We will assume you are familiar with the basic interface and node operations of ComfyUI.

Step 1: Load the Base Workflow and Add the Preprocessor
Navigate to the ComfyUI interface and load a Text-to-Image workflow. Create a new "Tile Preprocessor" node and connect it to the "Load Image" and "Preview Image" nodes. Click the "choose file to upload" button in the "Load Image" node to upload the blurry image you intend to process.

Step 2: Connect the ControlNet Nodes
Create the "ControlNet Apply" and "ControlNet Loader" nodes and connect them. In the "ControlNet Loader," select the Tile Model (e.g., control_v1f_sd15_tile.pth). Then, connect the "image" input port of the "ControlNet Apply" node to the "image" output port of the "Tile Preprocessor" node.

Step 3: Set Up the Conditional Guidance
The ControlNet Apply node serves as a positive condition for image generation. Therefore, its "conditioning" port needs to be connected in series between the "CLIP Text Encoder" and the "K Sampler" nodes. This allows the Tile Model to guide the detail optimization during the generation process.

Step 4: Select the Model and Prompts
Choose an appropriate Checkpoint Model based on the image type. For instance, when working with anime images, you might use counterfeitV30.130.safetensors.In the Positive Prompt box, enter phrases describing a high-quality image (e.g., "best quality, masterpiece, smile, outdoors"). In the Negative Prompt box, input phrases to avoid low quality (e.g., "lowres, error, blurry").

Step 5: Configure Parameters
In the "Tile Preprocessor" node, set Iterations to 1 and the Resolution to match the uploaded image's height (e.g., 320). In the "ControlNet Apply" node, set the Strength to 0.8. In the "Empty Latent" node, set the desired output image dimensions (e.g., $1024 \times 1536$) and adjust the batch size.

Step 6: Adjust K Sampler Settings
In the "K Sampler" node, set the seed to 0, the operation after running to "randomize," steps to 25, CFG to 7, sampler to "dpmpp_2m," scheduler to "karras," and denoise to 1.

Step 7: Generate the Image
Click the "Queue Prompt" button and wait for the processing to finish. You will obtain a high-definition, upscaled image with visibly enhanced details, while the overall composition remains stable.

Conclusion: Practical Applications of the Tile Model

The Tile Model is not just suitable for image upscaling; it is also a powerful tool for restoring details in blurry or low-quality images. With this tutorial, you can quickly get started with the Tile workflow in ComfyUI and apply it to process anime, photos, or other image types. Remember, the key is to adjust the preprocessor parameters and prompts to suit the needs of different scenarios. If you are interested in the latest features of ComfyUI, keep an eye on official updates, such as optimizations to the Tile Model or support for new nodes.Through practice, you will find the Tile Model to be a simple yet powerful tool that helps you easily achieve detail enhancement while maintaining image integrity.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

The function and usage of Inpaint are similar to local redrawing, except that Inpaint essentially replaces the algorithm for local redrawing in the native text-to-image function. By means of a deep learning model, it analyzes the missing areas in the image and the information of surrounding pixels, intelligently predicts and fills in pixels that match the surrounding environment, thereby achieving natural restoration of the image.

Inpaint Workflow Construction

What differs from other ControlNet preprocessors is that Inpaint has only one preprocessor and no additional components. Meanwhile, the construction of the Inpaint workflow is quite different from other models—since it requires a reference image, its workflow is based on the image generation workflow. The specific setup is as follows:

(1) Enter the ComfyUI interface, load the image generation workflow, delete the "VAE Encoder" node, create a "VAE decoder" node, and connect the "Latent" output port of the "VAE decoder" node to the "Latent" input port of the "K Sampler" node. Then connect the output port of the "Checkpoint Loader" node to the "VAE" input port of the "VAE" node.

(2) Create and connect the "ControlNet Apply" and "ControlNet Loader" nodes. In the "ControlNet Loader," select the Inpaint model "control_v1p_sd15_inpaint_fp16.safetensors." Connect the "image" output port of the "Inpaint Preprocessor" node to the "image" input port of the "ControlNet Apply" node.

(3) In the workflow, the "ControlNet Apply" node acts as a positive condition to guide the generation, so connect the "condition" port of the "ControlNet Apply" node in series between the "CLIP Text Encoder" node and the "K Sampler" node, as shown in the figure below.With this, the Inpaint workflow is fully set up.

Practical Operation

Although the Inpaint workflow has been set up, it still requires creating a mask image for use. Here, we’ll walk through the Inpaint workflow step-by-step using a product scene case, and explain the case settings in detail. The specific operation steps are as follows:

(1) Enter the ComfyUI interface. In the "Load Image" node, click the "choose file to upload" button to upload the prepared product material image. Right-click on the "Load Image" node, and select the "Open in mask editor" option from the pop-up menu to open the mask editor window.

(2) In the mask editor window, use the brush to paint over the white areas of the image—completely cover all white parts except the perfume bottle. Click the "Save to node" button in the bottom right corner of the window. The masked image will then be displayed in the "Load Image" node.

(3) Since the image to be partially redrawn is a photorealistic e-commerce-style image: select the photorealistic Checkpoint model “majicmixRealistic_v7.safetensors”.

(4) In the positive prompt box, enter a description of the new background. Here, we input: "still life, indoors, spot backdrop, pink flower, best quality, masterpiece, bottle, solo". In the negative prompt box, enter prompts for undesirable image quality: "lowres, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg". Set the "Strength" of the "ControlNet Apply" node to 1, as shown in the figure below.

(5) In the "K Sampler" node:
• Set "Random Seed" to 0
• Set "Post-run Action" to "randomize"
• Set "Steps" to 25
• Set "CFG" to 7
• Set "Sampler" to "dpmpp_2m"
• Set "Scheduler" to "karras"
• Set "Denoise" to 0.8
As shown in the figure below. Note that the "Denoise" value should not be set below 0.5, otherwise the redrawing effect will be very unnoticeable.

(6) Click the "Add Prompt Queue" button, and an image of the product with a new background will be generated.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

OpenPose is a key model for controlling human poses. It can detect key points of the human body structure—such as the position of the head, shoulders, elbows, and knees—while ignoring detailed elements like the person’s clothing, hairstyle, and background. By capturing the position of the human structure in the frame, it restores the person’s pose and expression.

OpenPose Preprocessors

OpenPose has 5 preprocessor nodes: "Dense Pose Preprocessor", "DW Pose Preprocessor", "MediaPipe Facial Mesh Preprocessor", "OpenPose Pose Preprocessor", and "AnimalPose Animal Pose Preprocessor".

In the "DW Pose Preprocessor" and "OpenPose Pose Preprocessor" nodes, you can control which body parts are included in the skeleton map. The "Dense Pose Preprocessor" node differs from others: it uses different colors to distinguish body parts to achieve pose control. The "MediaPipe Facial Mesh Preprocessor" node can detect and track human faces in real time from input images or videos, then generate a dense mesh containing 468 key points. The "AnimalPose Animal Pose Preprocessor" node detects key points of animal body structures and generates corresponding skeleton maps.

These five nodes vary significantly in function, application scenarios, and features:

1. Dense and DW Pose Preprocessors focus on full-body pose analysis and recognition.
2. MediaPipe Facial Mesh Preprocessor specializes in extracting facial features.
3. OpenPose Pose Preprocessor provides full-body pose and key point detection.
4. AnimalPose Animal Pose Preprocessor is dedicated to animals.

Practical Operation

(1) Enter the ComfyUI interface, load the text-to-image workflow, create a new "DW Pose Preprocessor" node, and connect it to the "Load Image" and "Preview Image" nodes, as shown in the figure below.

(2) Create new "ControlNet Apply" and "ControlNet Loader" nodes and connect them. In the "ControlNet Loader", select the "control_v11p_sd15_openpose.pth" OpenPose model.

(3) In the workflow, the "ControlNet Apply" node acts as a positive condition to guide the generation. Therefore, connect the "Condition" port of the "ControlNet Apply" node between the "CLIP Text Encoder" node and the "K Sampler", as shown in the figure below.

(4) Since we’re generating an IP character image, select the architectural-style model "IP DESIGN_3D Cute Style Model_V3.1.safetensors" as the Checkpoint model.

(5) Enter the description of the refined style in the positive prompt box—here we input: "1girl, Sailor Moon, solo, earrings, upper body, portrait, looking at viewer". In the negative prompt box, enter prompts for poor image quality—here we input: "lowres, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg".

(6) For the "DW Pose Preprocessor" node, set all detection options to "Enable", set the "Resolution" to 768 (this is the height of the uploaded image, so you can keep the default size of the image here). Set the image size in the "Empty Latent" node to 512×768, and set the batch size for image generation to 1, as shown in the figure below.

(7) In the "K Sampler" node:
Set "Randomize" to 0
Set "Post-run Action" to "Randomize"
Set "Steps" to 25
Set "CFG" to 7
Set "Sampler" to "dpmpp_2m"
Set "Scheduler" to "karras"
Set "Noise Reduction" to 1

(8) Click the "Add Prompt Queue" button, and the IP character image with the same action as the uploaded image will be generated.

Openpose skeleton image

In addition to uploading real human images, the Openpose Preprocessor node can also upload simple human action images or directly upload pre-extracted human skeleton images. You can download human action images directly from this website: www.posemaniacs.com/zh-Hans.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

In ComfyUI, ControlNet models help users fine-tune and control how images are generated. Among them, Scribble and Lineart are two of the most commonly used edge-based sketch extraction models. They’re ideal for hand-drawn or line-art–style control.
This guide walks you through the basic concepts, how to choose the right preprocessor, and a simple workflow example to help beginners get started quickly.

1. Scribble Model Intro: Sketch-Style Line Extraction

Scribble is an edge detection model that produces results resembling loose hand-drawn sketches.
Its lines are thicker and less precise, making it great for situations where you only need a rough outline and want Stable Diffusion to freely fill in the details.
For example, you can use it to turn a photo into a casual sketch and then generate a stylized image based on that outline.

2. Scribble Preprocessors: Four Types and How to Choose

Scribble provides four different preprocessor nodes, each with unique features and use cases:

• FakeScribble Preprocessor – Simulates a doodle-like effect using an approximate algorithm rather than real edge detection. Perfect for quick, rough sketch generation.
• Scribble Preprocessor – Converts an image into a simplified and abstract sketch with cleaner lines.
• ScribbleXDoG Preprocessor – This one uses the Extended Difference of Gaussian (XDoG) method for edge detection. The threshold setting controls how detailed the lines are — a lower value picks up more edges (even the faint ones), while a higher value keeps things clean and minimal.
• ScribblePiDiNet Preprocessor – Based on the Pixel Difference Network (Pidinet), this one is good at detecting both curves and straight edges for more precise line control.

Recommendation: If you’re new to ComfyUI, start with FakeScribble for its simplicity. For finer control over detail, try ScribbleXDoG.

3. What Is the Lineart Model: Detailed Line Extraction

Lineart is another edge detection model, but it focuses on capturing clean, artistic lines.
Unlike Canny, which produces hard, computer-perfect outlines with uniform width, Lineart lines feel more hand-drawn — you can see subtle variations in thickness and brush-like texture.
It includes two main styles: realistic and anime, allowing you to choose based on your source image type.

4. Lineart Preprocessors: Four Variants and When to Use Them

Lineart provides four preprocessor nodes, each suited for different artistic effects:

• LineArt Preprocessor – Includes a Roughness option to simulate the irregular texture of hand-drawn lines.
• LineArtStandard Preprocessor – Controls Gaussian blur intensity via the guassian_sigma parameter. A smaller value results in sharper lines; a larger value makes lines smoother.
• AnimeLineArt Preprocessor – Designed specifically for anime-style images, capturing clean and stylized linework.
• MangaAnime Preprocessor – Emphasizes sharp outlines, ideal for manga-like results.

Recommendation:
Use LineArt or LineArtStandard for real-world photos, and AnimeLineArt or MangaAnime for anime or comic images.

5. Building a Lineart Workflow: Simple Steps and Example

Lineart workflows are built the same way as Scribble ones — simply swap in a Lineart preprocessor.
Here’s a basic example using anime-style character re-styling:

1. Set up a text-to-image workflow in ComfyUI and add your chosen Lineart preprocessor node (e.g., AnimeLineArt).
2. Connect the ControlNet Loader and Apply nodes, and select the corresponding Lineart model.
3. Connect the preprocessor output to ControlNet Apply and integrate the condition port between the Text Encoder and Sampler.
4. Set prompts and parameters (like image size and sampling steps), then generate the image.

For instance, you can upload an anime character, extract its line art using AnimeLineArt, and then change its visual style — say, from casual attire to a fantasy outfit. Lineart helps preserve structure while allowing the model to freely reinterpret the style.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

In ComfyUI, preprocessors play a key role in controlling how your images are generated. This article introduces two commonly used ones—MLSD and Depth—and walks you through real examples to help you get started quickly.
Whether you’re new to ComfyUI or looking to optimize your workflow, these tips will help you create more efficiently and with greater control.

1. MLSD Preprocessor: The Straight-Line Extraction Specialist

The MLSD (M-LSD) Preprocessor is designed to extract straight-line edges from an image.
It identifies and preserves linear features while ignoring curves—perfect for tasks that require precise geometric boundaries, such as architectural design, interior decoration, and engineering drawings.

2. Understanding the MLSD Preprocessor Node

ComfyUI includes one MLSD-related node:
“M-LSD Line Segment Preprocessor.”
This node provides two main parameters:

• Score Threshold (0–2): Controls the strength of line detection. The higher the value, the fewer lines will be kept.
• Distance Threshold (0–20): Filters out lines that are too short, keeping only longer segments.

3. Practical Example: From Empty Room to Finished Interior

Here’s a simple walkthrough to help you use MLSD in practice:

1. Open ComfyUI and load a text-to-image workflow. Add the “M-LSD Line Segment Preprocessor” node. Connect it to “Load Image” and “Preview Image,” then upload a raw interior photo.
2. Create “ControlNet Apply” and “ControlNet Loader” nodes. In the loader, select the model:control_v11p_sd15_mlsd_fp16.safetensors, and link the image output to the MLSD node.
3. Connect the “Condition” port from ControlNet Apply between the CLIP Text Encoder and the K Sampler.
4. Load a suitable checkpoint model, for example:
   Interior_ModernStyle_Fine_2.0.safetensors.
5. Enter prompts:

• Positive prompt: Ceramic tiles, ceiling lights, doors, windows, sofas, fine decoration
• Negative prompt: lowres, text, error, extra digit, cropped, low quality, jpeg

6.   Set parameters:

• MLSD Score Threshold and Distance Threshold = 0.1
• Resolution = 512
• ControlNet Strength = 1

7.   In the Empty Latent node, set output size to 512×512 and batch = 1.

8.   Configure K Sampler: seed = 0, steps = 25, CFG = 7, sampler = dpmpp_2m, scheduler = karras, denoise = 1.

9.    Click “Add Prompt Queue” to generate the image.

After completing these steps, you’ll see how MLSD extracts clean structural lines from the raw photo, providing a solid base for refined interior design generation.

4. Depth Preprocessor: Controlling 3D Depth and Perspective

The Depth Preprocessor generates a depth map from an image, showing the distance of objects in grayscale—the closer an object is, the lighter it appears; the farther it is, the darker.
It’s particularly useful for managing foreground and background relationships and enhancing the spatial depth of generated images.

5. Choosing a Depth Preprocessor Node

ComfyUI offers three Depth preprocessors, each suited to different needs:

• LeReS Depth Preprocessor: Allows foreground/background removal. The “Enhance” option strengthens edge details and mid-range object separation.
• MiDaS Depth Preprocessor: The default option. Adjust the “Angle” for better depth interpretation from different viewpoints; “Background Threshold” separates foreground and background.
• Zoe Depth Preprocessor: Balances detail and stability, sitting between LeReS and MiDaS—ideal for general use.

Conclusion

The MLSD and Depth preprocessors are powerful tools within ComfyUI that give you greater control over geometric structure and 3D composition.
By fine-tuning their parameters and combining them with the right models, you can easily move from basic outlines to complex visual designs.
Experiment with these tools in different projects—you’ll find endless creative possibilities waiting to be explored

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

When using ComfyUI for AI image generation, ControlNet plays a key role in enabling structured and controllable image outputs.
Among its many tools, Canny and SoftEdge are two of the most widely used edge detection models. They extract outlines in different ways, allowing creators to precisely guide the structure and details of their generated images.

This guide explains the principles, parameters, and workflow setups for both models, helping beginners quickly master their use in ComfyUI.

1. Introduction to the Canny Model

Canny is one of the most widely adopted and essential models in ControlNet.
Based on a classic edge detection algorithm, it captures image contours with great precision and uses them to guide the generation of new images.

The preprocessed output from Canny looks like a finely drawn sketch with clean outlines — perfect for scenes that require clear structure and sharp edges.
For example, when converting a realistic portrait into anime style, Canny helps preserve facial proportions and overall structure.

📌 Canny Preprocessor Node

Canny uses a single preprocessing node called the “Canny Fine Edge Preprocessor”, which consists of three main components:

• Low Threshold
• High Threshold
• Resolution

Threshold Settings

Thresholds control how detailed the sketch lines will appear, with values ranging from 1 to 255:

• Lower values → more complex lines, capturing finer details.
• Higher values → simpler lines, keeping only major outlines.

Canny’s “dual-threshold” logic works as follows:

• Above high threshold → strong edges, always kept.
• Between thresholds → weak edges, kept only if connected to strong edges.
• Below low threshold → ignored as noise.

You can fine-tune the thresholds to achieve your preferred sketch complexity:
too detailed may lead to noisy results, while too simple can reduce precision.
A typical range is 100–200, keeping resolution consistent with the original image.

🧩 Example Workflow: Realistic Portrait → Anime Style

Here’s how to build a Canny-based workflow in ComfyUI to turn a real portrait into anime-style artwork:

1.Create Nodes
Open ComfyUI, load your text-to-image workflow, and add a “Canny Fine Edge Preprocessor” node.
Connect it to “Load Image” and “Preview Image” nodes, then upload your source photo.

2.Load ControlNet Model
Add “ControlNet Loader” and “ControlNet Apply” nodes.
In the loader, select control_v11p_sd15_canny, and connect it to the preprocessor output.

3.Integrate into Main Flow
Connect “ControlNet Apply” between the “CLIP Text Encoder” and the “K Sampler.”

4.Select Checkpoint Model
Use an anime-style model, e.g., counterfeitV20_v30.safetensors.

5.Enter Prompts

• Positive prompt:
  anime, 1girl, outdoors, short hair, green shirt, depth of field...
• Negative prompt:
  lowres, error, cropped, low quality...

6.Adjust Parameters

• Threshold: 50–80
• Resolution: same as source (e.g. 512)
• Control Strength: 1
• Size: 512×512
• Steps: 20 | CFG: 8
• Sampler: dpmpp_2m | Scheduler: karras

7.Generate
Click “Add Prompt Queue” to generate the anime-style result.

2.Introduction to the SoftEdge Model

Unlike Canny, SoftEdge extracts smoother, more natural edge transitions — ideal for illustration, oil painting, or 3D anime-style works.
It generates blurred yet coherent outlines, giving the image a soft and artistic feel.

⚙️SoftEdge Preprocessor Nodes

SoftEdge provides two preprocessor options:

• HED Soft Edge Preprocessor
• PidiNet Soft Edge Preprocessor

Both include:

• Stabilize — enhances contrast and reduces excessive blur.
• Resolution — controls detail level in the output sketch.
  

Algorithm Comparison

🧩 Example Workflow: Stylized Portrait Generation

The setup is similar to Canny — just replace the preprocessing node with SoftEdge.
Here’s how to create an “oil painting portrait” workflow:

1.Create Nodes
Add an “HED Soft Edge Preprocessor,” connect it to “Load Image” and “Preview Image,” and upload your material.

2.Load Model
Choose control_v11p_sd15_softedge_ip in the “ControlNet Loader” and link it to the preprocessor.

3.Integrate into Main Flow
Connect “ControlNet Apply” between “CLIP Text Encoder” and “K Sampler.”

4.Select Checkpoint Model
Use an oil-painting model like SHIMILV_OilPainting_V2.1.safetensors.

5.Enter Prompts

• Positive:
  1girl, jewelry, hanfu, outdoors, lake, red lips, upper body...
• Negative:lowres, error, cropped, low quality...

Set Parameters

• Stabilize: enabled
• Resolution: match source (e.g., 768)
• Control strength: 1
• Output: 512×768, Steps: 25, CFG: 7, Sampler: dpmpp_2m

Generate
Click “Add Prompt Queue” to create an oil-painting-style portrait.

Summary

• Canny: great for clear, structured images like realistic-to-anime conversions.
• SoftEdge: ideal for softer artistic transitions like illustrations or oil paintings.

Both models give creators precise, controllable results in ControlNet.
Whether you’re a beginner or an experienced artist, tuning parameters and combining models will help you find your ideal creative balance.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

When working with Stable Diffusion (SD), you may have noticed how unpredictable the results can be — generating the same prompt several times often gives entirely different images.
ControlNet was created to fix exactly that problem.

By using Conditional Generative Networks, ControlNet allows you to provide structured visual guidance — such as poses, sketches, or depth maps — so the model can generate results that follow your intended structure instead of leaving everything to randomness.

This tutorial walks you through how to install and use ControlNet in ComfyUI, step by step:

1. Install ControlNet preprocessors
2. Install ControlNet models
3. Set up and connect ControlNet nodes in ComfyUI

On the Cephalon Cloud platform, skip the complex installation process—simply create a ComfyUI application to directly experience the powerful image control capabilities of the ControlNet plugin.

The platform offers multiple GPU configurations and abundant model resources to meet various creative needs. You can easily access high-performance RTX 5090 graphics cards in the cloud, benefiting from faster rendering and greater stability—all for just $0.712 per hour. This makes creation more efficient and economical.

What Is ControlNet?

ControlNet is an extension for Stable Diffusion that adds extra control inputs to the generation process.
With it, you can guide your image using visual references such as:

• Human pose or body position
• Object contours or edges
• Depth maps, segmentation maps, sketches, or line art

In other words, ControlNet helps the AI understand your intended composition and structure, giving you much more control over what the model generates.

Installation Steps

To use ControlNet in ComfyUI, you’ll need to install two parts:

1. ControlNet Preprocessors – Convert reference images (e.g., sketches or poses) into structured data that the model can understand.
2. ControlNet Models – The actual models that process the structured data to influence image generation.

Let’s go through these one by one.

1. Installing ControlNet Preprocessors

1. Open ComfyUI and click the Manager button on the right sidebar.
2. In the ComfyUI Manager window, select Install Custom Nodes.
3. Use the search bar at the top right and type “ControlNet”.
4. Find the package named ControlNet-Aux and click Install.

⚠️ Important Notes:

• Do not install the old comfy_controlnet_preprocessors plugin — it’s deprecated.
• The new auxiliary preprocessors are actively maintained.
• If you already installed the old one, uninstall it before installing the new version.

After installation, restart ComfyUI.
You’ll now see a “ControlNet Preprocessors” category when right-clicking to add new nodes, including popular options like Canny, Depth, and Pose.

2. Installing ControlNet Models

1. Go back to the ComfyUI Manager window and click Models Manager.
2. Search for “ControlNet” in the top-right bar.
3. Choose the model version that matches your base Stable Diffusion version (e.g., SD1.5 or SDXL).

You can also download the models manually:

Official model repository (Hugging Face):
👉 https://huggingface.co/lllyasviel/ControlNet-v1-1/

Download the model you need (e.g. control_v11p_sd15_depth.pth)

Place the file in:

ComfyUI/models/controlnet/

Once placed, create a Load ControlNet Model node in ComfyUI — your new models should appear automatically in the dropdown.

💡 Tip:
If you’ve previously installed ControlNet models in Automatic1111 WebUI, ComfyUI can use those same files directly, as long as they’re in the same directory path.

ControlNet 1.1 Model Version Overview

In ControlNet 1.1, the number of models has increased from the previous 1.0 version to 14, with a more standardized naming convention.
The naming rule is:
“Version + Model Status + Stable Diffusion Version + Model Type”
For example: control_v11p_sd15_canny.

Model Status Description

ControlNet 1.1 introduces three model statuses to help users distinguish between stability and use cases:

Naming Examples

Here are some common examples to help you quickly understand version differences:

• control_v11p_sd15_canny → Version 1.1 Production, based on SD1.5, using the Canny edge detection model
• control_v11e_sd15_depth → Version 1.1 Experimental, based on SD1.5, using the Depth model
• control_v11u_sd15_pose → Version 1.1 Unfinished, based on SD1.5, used for pose control

Using ControlNet in ComfyUI

In ComfyUI, ControlNet works through a few interconnected nodes. Here’s how they function and connect.

1. ControlNet Loader

Path:Loaders > ControlNet Loader

Purpose: Loads a ControlNet model file.

Output: ControlNet

Connects to: The ControlNet input of the ControlNet Apply node.

2. ControlNet Apply

Path: Conditioning > ControlNet Apply

Purpose: Combines your text prompt, ControlNet model, and preprocessed image into a conditioning signal that guides image generation.

Typical setup:CLIP Text Encoder → ControlNet Apply → KSampler

Inputs:

Condition (from CLIP encoder)

ControlNet (from ControlNet Loader)

Image (from ControlNet Preprocessor)

Output:

Condition (connect this to the positive conditioning input of your sampler)

You can chain multiple ControlNet Apply nodes to combine multiple conditions — for example, Pose + Edges + Depth — giving you multi-layered control over the generation process.

3. ControlNet Preprocessor

Path: ControlNet Preprocessors

Purpose: Converts an image into a structural guide (e.g., edge map, pose, depth map).

Input: Image

Output: Image

Connects to: The Image input of the ControlNet Apply node.

You can preview the preprocessed result by connecting it to a “Preview Image” node.

What’s Next?

ControlNet includes a wide range of model types — such as pose, line art, segmentation, normal map, and depth.
In upcoming tutorials, we’ll explore each of these with practical ComfyUI workflow examples so you can learn how to build advanced, controllable generation pipelines step by step.

✅ Summary

ControlNet transforms Stable Diffusion from a random generator into a controllable creative tool.
Whether you want to lock character poses, maintain scene structure, or refine image consistency, ControlNet is one of the most essential tools to learn.

With ComfyUI’s visual, node-based workflow, using ControlNet becomes much easier and intuitive — perfect for both beginners and advanced creators alike.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

When starting your AI art journey in ComfyUI, the first and most crucial choice you'll encounter is selecting the Base Model (often called the "checkpoint" or "large model"). Positioned at the very beginning of a typical workflow, its placement alone hints at its fundamental role.

What is a Base Model? Think of It as the AI's "Artistic Brain"

You can think of a base model as the AI's "knowledge repository" and "artistic style amalgamation"—the result of extensive training on massive datasets. It doesn't store thousands of images itself; instead, it contains the visual patterns and artistic principles distilled from that data.Because they encapsulate vast amounts of learned information, base model files are typically very large. Current mainstream models, often based on architectures like SD 1.5 or SDXL, usually range from around 2GB to over 7GB in size—a direct reflection of the "knowledge" they contain.

Why Do You Need Multiple Models? It's Like Hiring Different Specialist Artists

Once you understand that a base model represents a specific set of styles and capabilities, it becomes clear why no single model can do everything perfectly.

• Specializations Vary: Some models are specifically trained to excel at creating photorealistic portraits, while others shine in producing anime-style artwork. You'll find specialists in architectural visualizations, fantasy landscapes, and more.
• Choose the Right Tool: This is why many AI creators build extensive libraries of different base models. It's akin to hiring a portrait painter for a portrait and a cartoonist for a comic—in ComfyUI, you select the most suitable "specialist artist" (base model) for your specific creative task.

A Key Difference Between SD and Midjourney: Open Ecosystem vs. Unified Service

This distinction is crucial for understanding the SD ecosystem:

• Midjourney operates more like a single, highly capable "master artist" with a consistent style. You guide this artist with prompts but cannot fundamentally change its core approach.
• Stable Diffusion (via ComfyUI), in contrast, provides you with an open studio filled with diverse "artist brains" (base models), each with unique specialties. You have the freedom to choose which artist works for you, or even combine their talents.

See the Difference: A Practical Demonstration

The best way to grasp the impact of a base model is through comparison. Using the exact same prompt and generation settings while only swapping the base model will yield strikingly different results.
For instance, if you use a base model specifically trained for realistic portraits (like majicmixRealistic), even a simple prompt can produce a figure with convincing skin textures and lifelike lighting. Feed that same prompt into an anime-style model, and the output will be entirely different in character.

The Key Takeaway: Selecting a base model aligned with your desired artistic style is the first—and most critical—step toward successfully generating your envisioned image.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

When creating images in ComfyUI, your prompts are the main language for communicating with the AI model. How can you precisely express your ideas to generate images that match your vision? Mastering these three core techniques will significantly boost your control over the results.

Technique 1: Control Element Importance with Word Order
Many beginners overlook a crucial detail: Words placed closer to the front of your prompt are generally treated as more important by the AI.
If you find a specific element isn't appearing prominently in your generated image, besides using weight brackets, the most straightforward method is to **move it toward the beginning of your prompt.

Practical Example

• Initial Prompt: masterpiece, best quality, girl, shining eyes, pure girl, solo, long hair... (details omitted) ...teddy bear
• Result: The "teddy bear," mentioned at the very end, was barely noticeable or ignored.
• Adjusted Prompt: masterpiece, best quality, teddy bear, girl, shining eyes... (other words follow)...
• Result: By moving "teddy bear" to the front, its presence and clarity in the image improved dramatically.
  Key Takeaway: Place your most important subjects and core characteristics at the beginning of your prompt.

Technique 2: Isolate Element Descriptions with Prompt Comments
Sometimes, even after adjusting weights, certain descriptions (especially colors) can "bleed" and affect the entire scene. For instance, a word describing clothing color might incorrectly apply to hair or the background.
This is where prompt comments come in handy, allowing you to create isolated description zones for different subjects. The basic format is: subject\(comment1, comment2)

Practical Example

• Initial Prompt: 1girl, silver hair, blue eyes, (yellow business suit:1.4), slim body... black handbag...
• Result: Even though the prompt specified a "black handbag," the highly weighted "yellow business suit" often caused the bag to render in yellow too.
• Adjusted Prompt: 1girl\(silver hair, blue eyes, (yellow business suit:1.4)), slim body... black handbag...
• Result: By encapsulating the girl's description (including her yellow suit) within comments, the handbag correctly appeared black, isolated from the subject's color influence.
  Key Takeaway: Use the \( ) comment syntax to isolate descriptions when you need to assign independent attributes to different parts of your image.

Technique 3: Integrate a Translator Node - Stop App Switching
For non-native English speakers, constantly switching between translation apps and ComfyUI to craft prompts can be a cumbersome workflow. While some plugins offer dictionary-based translation, they fail with words outside their predefined lists.
You can solve this by installing the translation node from the **AlekPet Node Pack**, which handles Chinese-to-English translation directly within your ComfyUI workflow.

Setup Steps

1. Install the Node:

Click "Manager" at the bottom right of the ComfyUI interface.

In the pop-up window, click "Install Node," search for "AlekPet."

Find the node pack, click "Install," and restart ComfyUI after installation.

2. Build the Translation Workflow:

From the node menu, navigate to "Add Node > AlekPet Nodes > Text > Translate Text (Argos Translate)."

To preview the translation, it's helpful to also add a "Preview Text" node (found under "Add Node > AlekPet Nodes > Extras").

3. Configure and Connect:

In the "Translate Text" node, set the source language to Chinese (zh) and the target language to English (en).

Connect the node's "text" output to the "text" input of the "Preview Text" node.

4. Translate and Use the Result:

Type your Chinese prompt into the "Translate Text" node's input box, e.g., "最佳质量，杰作，1女生，衬衫，牛仔裤，长发" (best quality, masterpiece, 1girl, shirt, jeans, long hair).

Click "Queue Prompt" to execute the translation. The translated English text will appear in the "Preview Text" node.

To use this translated text for image generation, connect it to a "CLIP Text Encode" node. By default, this node lacks a direct text input. Right-click on the "CLIP Text Encode" node, select "Convert To Input > Convert Text to Input" – this adds a "text" input port, allowing you to connect it to the "Preview Text" node's output.

Key Takeaway: The translator node seamlessly integrates Chinese prompts into your workflow, significantly improving efficiency and prompt accuracy for non-English users.

Unlock Full-Powered AI Creation!
Experience ComfyUI online instantly:
https://market.cephalon.ai/share/register-landing?invite_id=RS3EwW
Join our global creator community:
https://discord.gg/MSEkCDfNSW
Collaborate with creators worldwide & get real-time admin support.

When writing prompts, you can influence the local effects in an image by adjusting the weights of words in the prompt. This is typically done using different symbols and numbers, as follows.

1.Adjusting weights with curly braces "{}"
If you add “{}” to a word, you can increase its weight by a factor of 1.05 to enhance its presence in the image.

2.Adjusting weights with parentheses "()"
If you add “()” to a word, you can increase its weight by a factor of 1.1.

3.Adjusting weights with double parentheses "(())"
If you use double parentheses, the weights are multiplied, increasing the word's weight to 1.21 times (1.1 × 1.1). You can nest these up to three levels (i.e., "((()))"), for a maximum multiplier of 1.331 (1.1 × 1.1 × 1.1).

For example, when generating an image with the prompt "1girl, shining eyes, pure girl, (full body:0.5), luminous petals, short hair, Hidden in the light yellow flowers, Many flying drops of water, Many scattered leaves, branch, angle, contour deepening, cinematic angle", you get the first image shown on the bottom.

However, if you apply three levels of nesting to "Many flying drops of water" (i.e., "(((Many flying drops of water)))"), you get the second image shown on the bottom. As you can see, the number of water drops has significantly increased.

4.Adjusting weights with square brackets "[]"
The symbols introduced earlier all add weight. If you want to decrease the weight, you can use square brackets to reduce the prominence of that word in the image. When “[]” is added, the inherent weight of the word is reduced by 0.9. Similarly, you can use up to three nested square brackets.

For example, the first image below was generated using the prompt '1girl, shining eyes, pure girl,(full body:0.5),(((falling leaves))),luminous petals, short hair, Hidden in the light yellow flowers, branch, angle, contour deepening, cinematic angle'.

The second image below shows the effect after applying three nested square brackets "[[[]]]" to the prompt 'falling leaves'. It is clearly visible that the number of falling leaves has decreased.

5.Adjusting weights with colon “:”
In addition to using brackets above, you can also use a colon followed by a number to modify weights.For example, (fractal art:1.6) means adding 1.6 times the weight to fractal art.

Tips and Strategies for Adjusting Weights
Tips for Adjusting Weights
After selecting a word in the positive or negative prompts, hold down the Ctrl key and the up or down arrow keys to quickly add brackets to the word and adjust its weight.

Strategies for Adjusting Weights
When adjusting weights, you can first generate an image with no weights assigned to the positive prompts. Then, based on the image's effects, increase or decrease the weights of certain words to precisely modify the image's effect. However, be careful not to reduce weights too much, causing them to disappear; increasing weights too much can completely alter the image's overall appearance.

Unlock Full-Powered AI Creation!

Experience ComfyUI online instantly:

https://market.cephalon.ai/share/register-landing?invite_id=RS3EwWJoin

our global creator community:

https://discord.gg/MSEkCDfNSWCollaborate

with creators worldwide & get real-time admin support.