Kiwi-Edit is an open-source video editing framework from Show Lab. It pairs a multimodal large language model (MLLM) encoder, which reads your instruction, with a video diffusion transformer (DiT) that regenerates the clip. You describe the change in plain language and the model applies it across the whole video.

What makes it different from other AI video editors?

Two things. First, it handles instruction editing and reference-guided editing in one model — most tools do only one. Second, it is fully open: code, weights, training scripts and benchmarks are all published, so you can inspect or fine-tune it rather than calling a closed API.

Do I need a GPU to try it?

No. The Hugging Face Space embedded on this page runs on hosted hardware, so you can upload a clip and run an edit from the browser. A local install is what needs a GPU — the reference setup is Python 3.10 with CUDA 12.8 and PyTorch 2.7.

What is a reference image used for?

It tells the model what something should look like, which is hard to pin down in words. Upload a photo of a specific person, product or location, and Kiwi-Edit puts that exact subject or background into your video instead of inventing its own interpretation.

How long can the input video be?

The demo processes up to 81 frames per run — a few seconds of footage. That limit comes from the Wan2.2-TI2V-5B backbone the model is built on. For longer footage, split it into segments and edit each one.

What makes a good edit prompt?

Say what should change and what should stay the same. "Replace the dog with a white cat, keep the camera movement and background unchanged" works far better than "make it a cat". The demo's Prompt Tips panel groups worked examples by edit type.

Which checkpoint should I use?

kiwi-edit-5b-instruct-reference-diffusers is the general-purpose one and powers the demo. The instruct-only and reference-only checkpoints are specialists, useful if your workload is entirely one kind of edit or you are running comparisons.

Is Kiwi-Edit free to use commercially?

The code and weights are published openly, but licence terms live in the GitHub repository and the individual Hugging Face model cards. Check those directly before any commercial use — and note Kiwi-Edit builds on Wan2.2 and Qwen2.5-VL, whose licences also apply.

Open source · Show Lab

Edit any video by describing the change

Kiwi-Edit is an open-source video editing framework that combines an MLLM encoder with a video diffusion transformer. Write a plain-text instruction — or add a reference image — and it restyles, replaces, adds or removes anything in your clip while keeping the original motion intact.

Try the live demo View on GitHub

5B: Parameters
7: Edit types
81: Frames per run
3: Checkpoints

Live demo

Run Kiwi-Edit in your browser

The official Hugging Face Space, embedded below. Upload a video, write an instruction, optionally add a reference image, and run the edit on hosted hardware — no install required.

linyq-kiwiedit.hf.space

Open on Hugging Face

Load the live editor

Runs the official Kiwi-Edit Space in this page. Upload a clip, write your instruction, add an optional reference image, and hit Run Edit. Free hosted GPU — queue times vary.

The demo is hosted and maintained by the Kiwi-Edit authors on Hugging Face. Availability, queue length and GPU quota are controlled there, not by this site.

Overview

What is Kiwi-Edit?

A research framework that turns a sentence into a video edit — and, when words aren't enough, lets you point at a picture instead.

Most video edits are easy to say and hard to do. Removing one person from a moving shot, swapping a jacket, or restyling a whole clip normally means masking objects frame by frame. Kiwi-Edit collapses that into one instruction: you describe the change, and the model regenerates the footage with the change applied.

It works in two modes. Instruction editing takes text alone — “remove the monkey”, “change the background to a snowy mountain”. Reference-guided editing adds an image, which matters when the target is specific: a particular person, a real product, an actual location. Words can only approximate those; a reference image nails them.

Because the whole project is open — code, weights, training scripts and evaluation benchmarks — you can read exactly how it works, run it on your own hardware, or fine-tune it on your own data. That is the main practical difference from closed commercial video editors.

1. Read the request

An MLLM encoder (Qwen2.5-VL-3B) reads your instruction together with the source video — and the reference image, if you supplied one — so the model understands which object you mean and what should happen to it.

2. Regenerate the clip

A video diffusion transformer (Wan2.2-TI2V-5B) generates the edited footage frame by frame, conditioned on the original video so untouched regions, motion and camera movement carry over.

3. Return an edited video

You get a new clip plus a side-by-side view against the source, so you can check that the edit landed and nothing else drifted.

Capabilities

Seven kinds of edit

Five driven by text alone, two guided by a reference image. Each card shows the kind of prompt that produces it — the same examples the demo's library auto-fills.

Instruction

Style

Restyle an entire clip — painterly, cinematic, anime, retro film — while the motion, framing and timing of the original footage stay intact.

“Apply the dynamic aesthetic of abstract art to this video.”

Instruction

Replace

Swap one subject or object for another. The replacement inherits the original's position, scale and movement across every frame.

“Replace the dog with a white cat.”

Instruction

Add

Insert a new object into the scene. Kiwi-Edit matches lighting and perspective so the addition tracks with the shot.

“Add a pair of sunglasses to the man.”

Instruction

Remove

Delete a subject and inpaint what was behind it, keeping the recovered background stable frame to frame.

“Remove the monkey.”

Instruction

Background Replace

Keep the foreground subject and rebuild the environment around it — a new location, season or time of day.

“Change the background to a snowy mountain at sunset.”

Reference + Instruction

Subject Reference

Upload a reference image of a person, product or character and Kiwi-Edit places that exact subject into the video.

“Replace the woman with the person in the reference image.”

Reference + Instruction

Background Reference

Upload a reference image of an environment and the model transplants the subject into that specific place.

“Change the background to the scene in the reference image.”

At a glance

Kiwi-Edit specifications

The technical facts, in one place.

Project	Kiwi-Edit
Category	Instruction & reference-guided video editing
Published by	Show Lab
Architecture	MLLM encoder (Qwen2.5-VL-3B) + video DiT (Wan2.2-TI2V-5B)
Model size	5B parameters
Inputs	Source video, text instruction, optional reference image
Max frames per run	81 (adjustable from 8 in the demo)
Training resolution	Up to 720×1280
Benchmarks	OpenVE-Bench, RefVIE-Bench
Availability	Open source — code, weights and datasets published

Step by step

How to use Kiwi-Edit

The full workflow in the hosted demo, from upload to a finished clip.

1
Upload your source video
Drop an MP4 into the Source Video panel. Short clips work best — the model processes up to 81 frames in a single run.
2
Write the edit instruction
Describe both halves of the edit: what should change, and what should stay exactly as it is. "Remove the monkey" is a valid prompt; adding "keep the background and camera motion unchanged" gives a cleaner result.
3
Add a reference image (optional)
If the edit involves a specific person, product or place, upload a photo of it. The model uses that image as the visual target instead of inventing its own.
4
Adjust the advanced settings
Pick a checkpoint, set max frames (8–81), inference steps (10–80, default 50) and a seed. More steps means more detail and a longer wait; a fixed seed makes a run reproducible.
5
Run and compare
Hit Run Edit and wait for the queue. The output panel shows the edited video and a side-by-side against the source. If it drifts, tighten the prompt or change the seed and run again.

Prompting

How to write a prompt that works

Kiwi-Edit regenerates the whole clip, so an under-specified prompt lets things drift. Name the change, then name what must stay. These five patterns mirror the demo's Prompt Tips panel.

Global Style

Name the style and say the structure should hold.

Apply a watercolour painting style to the whole video. Keep the subject's motion, layout and camera movement unchanged.

Background Change

Describe the new background in detail, then protect the foreground.

Change the background to a neon-lit city street at night. Keep the man in the foreground, his clothing and his movement exactly the same.

Local Remove

Name one object and say what should fill the gap.

Remove the red car on the left. Fill the area with the road and pavement behind it. Leave everything else untouched.

Local Add

State the object, where it goes, and how it should behave.

Add a small brown dog walking beside the woman on the right. Match the scene's lighting and keep the rest of the frame unchanged.

Local Change / Replace

Pair the target with the replacement in a single sentence.

Replace the woman's blue jacket with a yellow raincoat. Keep her face, hair, pose and the background identical.

Model zoo

Available checkpoints

Three Diffusers-format checkpoints, all 5B parameters, differing only in the data they were fine-tuned on.

kiwi-edit-5b-instruct-only-diffusers

Instruction data only

Text-only edits: style, add, remove, replace, background change.

Model card

kiwi-edit-5b-reference-only-diffusers

Reference data only

Edits driven by a reference image of a subject or background.

Model card

Recommended

kiwi-edit-5b-instruct-reference-diffusers

Instruction + reference data

The general-purpose checkpoint — the one the live demo runs.

Model card

Evaluation benchmarks

Kiwi-Edit is scored on two public benchmark datasets, one per editing mode.

OpenVE-Bench

Instruction-based video editing

Evaluates text-driven edits — style, add, remove, replace and background change — across a standard set of source videos.

RefVIE-Bench

Reference-guided video editing

Pairs source videos with reference images to test whether the specific subject or background actually makes it into the result.

Self-hosting

Run Kiwi-Edit locally

For local inference you need a CUDA GPU, Python 3.10 and PyTorch 2.7. The Diffusers path below is the shortest route to a first edit; the repository also documents the full training environment.

1. Set up the environment

conda create -n diffusers python=3.10 -y
conda activate diffusers

pip install torch==2.7.0 torchvision==0.22.0 \
  torchaudio==2.7.0 \
  --index-url https://download.pytorch.org/whl/cu128

pip install diffusers decord einops accelerate \
  transformers==4.57.0 opencv-python av

Or run bash install_diffusers_env.sh from the repository.

2. Run your first edit

python diffusers_demo.py \
  --video_path ./demo_data/video/source/sample.mp4 \
  --prompt "Remove the monkey." \
  --save_path output.mp4 \
  --model_path linyq/kiwi-edit-5b-instruct-only-diffusers

Swap --model_path for the instruct-reference checkpoint to use reference images.

Training and evaluation

Training runs in three stages — image, image + video, then reference-video — using Qwen2.5-VL-3B with Wan2.2-TI2V-5B at up to 720×1280 and 81 frames. Training metadata is CSV, scripts live in scripts/, and full instructions including DeepSpeed and FlashAttention setup are in the repository.

Read the full documentation

Honest assessment

Strengths and limitations

Kiwi-Edit is a research release. Knowing where it stops saves a lot of wasted runs.

Strengths

One model covers both instruction editing and reference-guided editing
Fully open: code, weights, training scripts and benchmarks are published
Reference images pin down specific subjects and backgrounds that words can't
Preserves motion, framing and camera movement from the source clip
Three checkpoints so you can match the model to the workload
Browser demo needs no local GPU or install

Limitations

81 frames per run — a few seconds of footage, not a full timeline
Local install needs a CUDA GPU, Python 3.10 and PyTorch 2.7
Results depend heavily on how precisely the prompt is written
Hosted demo sits in a shared queue, so waits vary with demand
Fine detail like faces, hands and text can drift across frames
Research code — it is not a finished consumer editing app

FAQ

Frequently asked questions

Common questions about what Kiwi-Edit does, what it needs, and where its limits are.

Try it on your own clip

The hosted demo takes a video, a sentence and an optional reference image. That is the whole interface.

Open the demo Visit the project page

Edit any video by describing the change

Run Kiwi-Edit in your browser

What is Kiwi-Edit?

1. Read the request

2. Regenerate the clip

3. Return an edited video

Seven kinds of edit

Style

Replace

Add

Remove

Background Replace

Subject Reference

Background Reference

Kiwi-Edit specifications

How to use Kiwi-Edit

Upload your source video

Write the edit instruction

Add a reference image (optional)

Adjust the advanced settings

Run and compare

How to write a prompt that works

Global Style

Background Change

Local Remove

Local Add

Local Change / Replace

Available checkpoints

kiwi-edit-5b-instruct-only-diffusers

kiwi-edit-5b-reference-only-diffusers

kiwi-edit-5b-instruct-reference-diffusers

Evaluation benchmarks

OpenVE-Bench

RefVIE-Bench

Run Kiwi-Edit locally

1. Set up the environment

2. Run your first edit

Training and evaluation

Strengths and limitations

Strengths

Limitations

Frequently asked questions

What is Kiwi-Edit?

What makes it different from other AI video editors?

Do I need a GPU to try it?

What is a reference image used for?

How long can the input video be?

What makes a good edit prompt?

Which checkpoint should I use?

Is Kiwi-Edit free to use commercially?

Try it on your own clip