convert lora .bin weights to ckpt or safetensors

[harrywang]

Another question: convert_lora_safetensor_to_diffusers.py converts safetensors to diffusers format. After I trained LoRA model, I have the following in the output folder and checkpoint subfolder:

How to convert them into safetensors like the ones I downloaded from civitai or huggingface so that I can use this via Automatica1111?

Thanks a lot!!

[harrywang]

https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_stable_diffusion.py I found this but have not tried yet. Will report back once I finish the testing.

[haofanwang]

If I remember correctly, webui supports both safetensor and ckpt format? But anyway, the script you are using is exact what you need!

python ./scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path xxx.ckpt --dump_path save_dir --to_safetensors

Then, you can get a safetensors file. I only test it with full model (usually of more than 2GB size, if your file is just the LoRA layers, I suggest to merge them first before converting. Hope this helps.

[harrywang]

Thanks! I will try and report back.

[haofanwang]

@harrywang I guess I have understood your need, please check this response for details.

In short, we don't support convert .bin to .safetensors now. I suggest to save the full model after loading LoRA as pipe.unet.load_attn_procs(), and specify safe_serialization when saving, instead of just LoRA weights in diffusers example code. Then you can get a full model in safetensors format.

[happypuppy20]

Hi @haofanwang , I'm having the SAME issue as @harrywang .
Basically, we trained using lora and got those 6 files, .bin file and a few others.

To use on Automatic1111 it seems like we need to have a different extension. reading your above response as well wasn't clearly gave me any insight how I can do it.

Hope a solution would be found :) Thanks!

[haofanwang]

Well, the main goal of this project is for developers using diffusers, not Automatic1111. But I try my best to help.

I'm not familiar with API of Automatic1111. But to be honest, only pytorch_lora_weights.bin saves the weights of LoRA. Other files in the folders are saved by accelerate as here. What is the exact file needed for diffusers? The folder or the single .bin file? If you need the folder, I cannot support it now, you may ask help from Automatic1111 or huggingface. But if the single .bin file can work, the good news is that I'm working on it.

@happypuppy20

[happypuppy20]

True, I'm referring to Automatic1111 as it's easy for us to run many tests easily and fast.
Regards devs using diffusers, maybe you can tell me cause I wasn't able to find it anywhere else, but how I can inference the model with settings used by Automatic1111 like :

prompts that include large Positive , large Negative , Sampler , CFG #, sampling #, Size 512x512

I couldn't find for the Negative or Sampler , CFG...

[haofanwang]

Please dive into diffusers for detailed usage. There you can find how to add negative prompt in inference.

[haofanwang]

Update:

Hi all, I just added a new script to support converting between .bin and .safetensors. You guys can check it out.

But as I have mentioned here, it has many limitations and may not work as expected. Anyway, you can try it first. If not work, we suggest to save the full model to safetensors directly, instead of only saving LoRA weights in .bin.

[harrywang]

Update:

Hi all, I just added a new script to support converting between .bin and .safetensors. You guys can check it out.

But as I have mentioned here, it has many limitations and may not work as expected. Anyway, you can try it first. If not work, we suggest to save the full model to safetensors directly, instead of only saving LoRA weights in .bin.

Sorry for the slow response - I was traveling in the last few days. Thanks for the script - I tried and the conversion worked but when I load it into Automatic1111 the lora has no effect - I am not sure what went wrong.

[jndietz]

I'm not really familiar with the structure of diffusers or the ckpt/safetensors format. But is it just a matter of mapping keys from one object to another? I think @haofanwang has something we can start with.

[harrywang]

Update:
Hi all, I just added a new script to support converting between .bin and .safetensors. You guys can check it out.
But as I have mentioned here, it has many limitations and may not work as expected. Anyway, you can try it first. If not work, we suggest to save the full model to safetensors directly, instead of only saving LoRA weights in .bin.

Sorry for the slow response - I was traveling in the last few days. Thanks for the script - I tried and the conversion worked but when I load it into Automatic1111 the lora has no effect - I am not sure what went wrong.

Hi, can you help me with this problem. I convert my lora weight with your script to safetensor, but i can not use the converted lora in webui. it show this error. can you tell me what went wrong here

you can also try my script at huggingface/diffusers#2326 (comment), which works for me.

[StanislawKarnacky]

@harrywang, @haofanwang sorry for bothering you, but is there any chance to convert .bin to .safetensors LoRA trained on SDXL Dreambooth? Tried 5 scripts at least, keys mismatch in all cases. Trained with: https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora_sdxl.py

[alivemachine]

Have you ever figured out how to convert your .bin to a .safetensors lora? @StanislawKarnacky