Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add free storage API for FSDP #277

Merged
merged 1 commit into from
Mar 26, 2025
Merged

Conversation

Seventeen17
Copy link
Contributor

@Seventeen17 Seventeen17 commented Mar 26, 2025

Fix #263

When using fsdp with Dit or T5, directly deleting the model with del model does not release the GPU memory occupied by the model.

This is because the memory allocated by FSDP's flat_param requires manual free.

To solve this, an additional free_model interface is added to distributed.fsdp.

This interface allows users to manually release the GPU memory occupied by the model sharded by FSDP in scenarios where different model checkpoints need to be switched.

Example Usage:

diff --git a/generate.py b/generate.py
index 1b1a9d7..307c949 100644
--- a/generate.py
+++ b/generate.py
@@ -321,6 +321,9 @@ def generate(args):
             seed=args.base_seed,
             offload_model=args.offload_model)
 
+        from wan.distributed.fsdp import free_model
+        free_model(wan_t2v.model)
+
     else:
         if args.prompt is None:
             args.prompt = EXAMPLE_PROMPT[args.task]["prompt"]

@WanX-Video-1 WanX-Video-1 merged commit bc3249d into Wan-Video:main Mar 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

使用FSDP加载模型,在切换模型时显存无法释放
2 participants