Why Your Character LoRA Doesn't Look Like Your Subject
Three runs on the same 24 photos. The first produced women in maxi dresses. Here's every failure mode, with the images that prove it.
TL;DR
Most character LoRA failures aren’t training-parameter problems — they’re naming problems. Your trigger word collides with a prior the model already has, or your class word is too generic, or you’re using a different class word at inference than you trained with. All three produce the same symptom: “it doesn’t look like my subject.”
And one meta-failure that wasted more of my time than any of them: evaluating with an over-specified prompt, which lets the base model fake a success you didn’t actually train.
If you want the settings that work, read the Character LoRA guide. This guide is about what goes wrong and how to tell which thing went wrong.
The setup
Same subject, same 24 phone photos, same GPU (RTX 4090), same base model (Krea 2 Raw), three runs. Only the naming changed.
Ground truth: Maxi. Golden-brown Pomeranian, cream chest, small. Every image below is trying to reproduce this dog.
The base model, the dataset, the rank, and the learning rate were identical across all three. Everything you see below comes from changing two strings.
Failure 1: no class word, so the trigger collapsed to its prior
Run v1 used --trigger maxi with no --class-word. Here’s what it generated:
v1, prompting the bare trigger. Not a dog. Not even close to a dog. Two women photographed at a beach.
Without a class word, the trigger has nothing to anchor to. The model has no reason to associate maxi with an animal, so it falls back on whatever maxi already means to it. During training this showed up in the sample images from the very first checkpoint:
A v1 training sample. The model heard 'maxi' and drew a maxi dress. This kept happening past step 1000 — the run never recovered.
--class-word tells the model what kind of thing the trigger is. Without it, training has to teach both the category and the identity from 24 images. With it, the model already knows what a Pomeranian is and only has to learn which one.
Failure 2: the trigger word collided with a common word
The dog’s name is Maxi, so maxi was the obvious trigger. It was also the worst possible choice — “maxi” is a strong fashion term, and that prior fought the training the whole way.
Pick a token the model has no existing associations with: m4xi, ohwx, sks3. Deliberately misspelled or nonsense. Run v2 switched to m4xi and the human images stopped immediately.
Quick test before you train: modl generate "a photo of <your-trigger>" with no LoRA. If you get anything coherent, the model has a prior for that word — pick a different one.
modl now warns when your trigger looks like a common word and suggests a rare alternative.
Failure 3: a generic class word loses the body type
Run v2 used --class-word dog. The markings and coloring came out right — and the dog was consistently the wrong size and build. “Dog” averages over every breed, so the model learned Maxi’s coloring but anchored her proportions to a generic medium dog.
Run v3 changed one word — --class-word pomeranian — and the size locked in.
Same LoRA, same seed, same everything — only the class word in the prompt changed. Left: a completely different dog. Right: Maxi.
Be as specific as your subject allows: pomeranian not dog, stratocaster not guitar, espresso machine not appliance.
Failure 4: the class word must match at training and inference
This is the subtlest one, and it’s what the image pair above is actually showing.
Both of those images came from the same v3 LoRA — the one trained with --class-word pomeranian. The only difference is the prompt:
Saying “dog” at inference pulls the generic prior back in and overrides what the LoRA learned. The identity doesn’t just lose its size — it collapses into a different animal entirely.
Your trigger and class word are a matched pair, and they have to match between training and inference. If you trained on “pomeranian,” every prompt needs “pomeranian.” Store the phrase with the LoRA so you don’t have to remember it in three months.
Failure 5: overtraining bought nothing
Run v2 trained to 2000 steps with checkpoints every 250. Comparing them on the same prompt and seed, the identity stops improving around step 1000:
500 steps apart. Same pose, same build, same weathered green wall. Cover the labels and you couldn't order them — that's roughly an hour of GPU time for no likeness gain.
(These are v2 samples, so the dog is the generic medium build from Failure 3 — the point here is the absence of change between checkpoints, not the likeness itself.)
Look at the background in that pair, though. It’s not just similar, it’s the same wall — a location from the training set, reproduced across checkpoints and across prompts that never asked for it. That’s the real overtraining tell here: not a loss of likeness, but the model starting to reproduce training context it should have generalized away.
Useful rule: about 40 steps per image. A 24-image dataset peaks near 1000. Save every 250 and keep the best checkpoint, not the last one — modl’s krea2 character preset now targets this range by default.
The classic overfit checks all stayed healthy at 1500 steps — no subject bleed into a golden retriever prompt, watercolor styles still rendered, novel scenes still generalized. If you only run those checks, you’ll conclude everything is fine while quietly wasting an hour. Check the unconditioned prompts for memorized backgrounds too.
Failure 6: I proved it worked with a prompt that proved nothing
This one was my own error and it’s the most important thing in this guide.
While debugging the broken v1 run, I generated this and briefly believed the LoRA was working:
Looks like a win. It isn't. The prompt was 'a golden-brown pomeranian with a white chest' — the base model draws that unaided, with no LoRA involved. This image is evidence of nothing.
If your evaluation prompt describes the subject, the base model can produce a convincing match on its own. You’ve tested the base model’s knowledge of Pomeranians, not your LoRA.
Strip every physical descriptor from the prompt, leaving only the trigger and the class word. Generate twice at the same seed — once with the LoRA off, once on. Any likeness that appears is attributable to the LoRA and nothing else.
Failure 7: a background worker was holding the GPU
Not a training failure, but it cost real time. Generations started OOMing on a 24 GB card for no apparent reason — modl’s persistent Python worker was still resident from an earlier session, holding 13 GB.
Check nvidia-smi before any heavy run. If memory is occupied and you don’t know why, stop the worker.
Diagnosing your own run
Symptom → cause
The recipe that works
Krea 2 character LoRA — verified settings
Cost for reference: roughly 2 hours for 2000 steps on a 4090, and sampling every 250 steps accounted for about 40% of that wall time. Sample less often if you’re iterating on settings rather than watching for convergence.
What’s next
Every failure here came from a string, not a hyperparameter. Two words — a rare trigger and a specific class — separated the run that produced women in maxi dresses from the one that produced the right dog.
For the full training walkthrough including model benchmarks and optimizer choice, see the Character LoRA guide. For style training, see the Style LoRA guide.
If you hit a failure mode this guide missed, open an issue at github.com/modl-org/modl.