How about transfer it to a task about document image restoration?

[feature-space-move]

I try to train my custom dataset of document image restratio followed Readme, but I encountered bad results.
I think the reason is the pretrain--pretrain focusing on natural image and I tried to fine-tune it on document image(many clear characters including chinese and english), maybe there is a huge gap between the two type of images.
May I train the whole HYPIR from scratch? Then I can't take advantage of the diffusion model pretrain.
What a pity and I'm so sad.
Help, please~

[0x3f3f3f3fun]

What's the resolution of your training images and what kind of the degradation you use? If you apply heavy degradation to document image, it's hard for sd2-based model to restore (or generate) the degraded characters.

[feature-space-move]

What's the resolution of your training images and what kind of the degradation you use? If you apply heavy degradation to document image, it's hard for sd2-based model to restore (or generate) the degraded characters.

I had tried training/fine-tuning heavy or light degradation to document image but always got bad results. The character ch/eng completely destroied.
Sad story o(╥﹏╥)o.

[0x3f3f3f3fun]

What's the resolution of your training images and what kind of the degradation you use? If you apply heavy degradation to document image, it's hard for sd2-based model to restore (or generate) the degraded characters.

I had tried training/fine-tuning heavy or light degradation to document image but always got bad results. The character ch/eng completely destroied. Sad story o(╥﹏╥)o.

I think it would be better if you upgrade the base model, for example: z-image. sd2.1 cannot generate characters.

[feature-space-move]

What's the resolution of your training images and what kind of the degradation you use? If you apply heavy degradation to document image, it's hard for sd2-based model to restore (or generate) the degraded characters.

I had tried training/fine-tuning heavy or light degradation to document image but always got bad results. The character ch/eng completely destroied. Sad story o(╥﹏╥)o.

I think it would be better if you upgrade the base model, for example: z-image. sd2.1 cannot generate characters.

Thanks for replying. I'll try it latter. ^_^

[feature-space-move]

What's the resolution of your training images and what kind of the degradation you use? If you apply heavy degradation to document image, it's hard for sd2-based model to restore (or generate) the degraded characters.

I had tried training/fine-tuning heavy or light degradation to document image but always got bad results. The character ch/eng completely destroied. Sad story o(╥﹏╥)o.

I think it would be better if you upgrade the base model, for example: z-image. sd2.1 cannot generate characters.

Thanks for replying. I'll try it latter. ^_^

I've tried z-image, it was too hugue to train.
Sad...

[0x3f3f3f3fun]

You can try FSDP2, but it requires good communication bandwidth between GPUs.