Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #263
When using fsdp with Dit or T5, directly deleting the model with
del model
does not release the GPU memory occupied by the model.This is because the memory allocated by FSDP's
flat_param
requires manual free.To solve this, an additional
free_model
interface is added todistributed.fsdp
.This interface allows users to manually release the GPU memory occupied by the model sharded by FSDP in scenarios where different model checkpoints need to be switched.
Example Usage: