LoRA overfitting: how to spot an overtrained character LoRA and fix it

LoRA overfitting means the LoRA learned too much: on top of her face, it learned the backgrounds, outfits and expressions it saw in training, so it puts them in every image. In a character LoRA the signs are the same background, outfit or expression creeping into every shot, plus plastic skin. The fix is usually to go back 500 steps to an earlier checkpoint, no retrain needed. It shows up when you start generating your consistent AI character for real, not during training.

What does LoRA overfitting look like on a character?

Images stop following the prompt. You ask for a rooftop bar at night and get the same beach light from the dataset. You ask for a red dress and she wears the white top again. Her smile is identical in every shot. Skin turns smooth and waxy. Her face is fine; everything around it is stuck.

The first four rows are the course’s “too late” signs. The second column is a rough guide to the likely cause, not a tested result:

One odd image can be a fluke of one seed. Check across several different prompts before you call it.

Is it overfitting or just strength that’s too high?

A good rule of thumb: run the same prompt and seed at 0.8, the low end of our test range. If the repeats go away and her face holds, strength was the issue. If the same background or outfit is still there at 0.8, the checkpoint learned it, and you need an earlier one.

LoRA strength scales everything the LoRA learned at once: her face and the unwanted extras. Turning it down to hide overfitting also loosens her face. Our generator template sits at 0.9, and the course’s fix for repeats is an earlier checkpoint. More on that in our LoRA strength guide.

How do you fix an overfit LoRA?

Go back 500 steps. A trainer that saves a checkpoint every 250 steps leaves you a trail of earlier versions, so the fix is loading a different file, not training again. Test it with the same prompts and seed. If it’s her and the repeats are gone, that’s your LoRA. If not, go back another 500.

The course’s training job runs 3000 steps, saves every 250 and sets Max Step Saves to Keep to 12, so every save of the run survives. AI Toolkit’s default keeps only the last 4 (new-job defaults, checked 2 Oct 2026). With the default, 2000 to 2750 survive, so going back 500 still works, but 1500 is gone.

The files carry the step in the name:

Video 3 of our course downloads the first three, the final file, 2500 and 2000, and compares them with the same prompt; there, 2500 was the keeper. Go to 1500 only if 2000 still repeats. Keep the last checkpoint before the repeats start: going further back than you need costs likeness. The full side-by-side method is in how to test a LoRA.

Why did the LoRA overfit?

Three causes, and they stack. The dataset repeats something besides her face, so the trainer treats it as part of her. The captions don’t name the repeat, so it has nowhere to go but the trigger word. Or training ran long enough to learn the repeats on top of the face.

The dataset repeats things. The trainer learns whatever stays the same across images as part of the trigger word. If she wears one white top in a third of the dataset, the LoRA learns the top as her. The same goes for one place, one light or one smile. Aim for variety in everything except her face, and a framing mix of about a third close-up or upper body, a third cowboy or three-quarter, a third full body, the split our free LoRA dataset planner uses.

The captions miss the repeats. A caption names what can change: shot and pose, outfit, place, light, framing. Skip the outfit in captions and the outfit has nowhere to go but the trigger word. Our captioning guide covers the full order.

Too many passes for the set. At batch size 1, 3000 steps on 50 images shows the trainer each image about 60 times. That’s our current template’s number. The right stopping point differs per dataset, so save often and pick from the images.

When do you need to retrain instead?

Retrain when no checkpoint is good: the early ones aren’t her yet, and by the time she’s clearly her, the repeats have started. That means the dataset itself carries the repeats. No checkpoint choice fixes that, because the repeats are in what the LoRA learns from.

Fix the dataset first:

1. Find the repeat. Look at what the bad images share: an outfit, a place, a light, an expression.
2. Count it in the dataset. If it appears in more than a few images, cut some or swap them for new ones.
3. Check the captions for those images. The repeated thing should be named in each one.
4. Delete off images. Anything that isn’t clearly her teaches a second face.
5. Retrain and test again: the final file, 2500 and 2000.

How many images you need and what they should cover is in our guide to LoRA dataset size.

What can go wrong?

• You check for repeats on one prompt. One prompt is enough to pick between checkpoints, as video 3 does, but a repeated background or outfit only shows across different scenes. If you suspect overfitting, add two more scenes with the same seed.
• You only kept the last few saves. If your trainer deletes old checkpoints, there’s nothing to go back to. AI Toolkit keeps only the last 4 by default. Set Max Step Saves to Keep to 12 in the job screen before training, as the course’s job does.
• You go back too far. 1500 might be free of repeats and still not quite her. Keep the last good one, not the earliest clean one.
• You fix overfitting with strength. Dropping the strength well below our 0.8 to 1.0 range may hide the background, and it loosens her face too. Pick an earlier checkpoint instead.
• You blame the LoRA for a prompt problem. If your prompts all say “golden hour”, every image will have golden hour. Vary the test prompts before you blame the checkpoint.
• Plastic skin from the prompt, not the LoRA. Words about skin or makeup push toward a polished look. Our prompts end with “candid smartphone photo, natural skin texture” and say nothing else about skin.