Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] D4rl direct download #1430

Merged
merged 14 commits into from
Oct 4, 2023
Merged

Conversation

MateuszGuzek
Copy link
Contributor

@MateuszGuzek MateuszGuzek commented Jul 31, 2023

Description

Allow for a direct download of D4RL datasets, without installing D4RL.

This PR restores the previous PR: #1376 which was uintentionally closed.

Motivation and Context

TBD, raised during direct discussion with @vmoens

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 31, 2023
@MateuszGuzek MateuszGuzek changed the title D4rl direct download [WIP] D4rl direct download Jul 31, 2023
vmoens added 4 commits October 4, 2023 10:18
# Conflicts:
#	torchrl/data/datasets/d4rl.py
… d4rl_direct_download

# Conflicts:
#	torchrl/data/datasets/d4rl.py
@vmoens vmoens marked this pull request as ready for review October 4, 2023 10:40
@vmoens vmoens added the enhancement New feature or request label Oct 4, 2023
@vmoens vmoens changed the title [WIP] D4rl direct download [Feature] D4rl direct download Oct 4, 2023
Copy link
Contributor

@vmoens vmoens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very happy with these changes @MateuszGuzek thanks a lot!
The main blocker right now is to document how this works in practice.
Being able to download data directly is great but we need to see what that means for environment execution:

The specific use case I would like to cover is:

$ pip install git+https://github.com/Farama-Foundation/d4rl.git
$ pip install gym -U
$ python -c """
# Works with latest gym installed:
from torchrl.data.datasets import D4RLExperienceReplay
data = D4RLExperienceReplay("halfcheetah-medium-v2", batch_size=10, from_env=False)
for batch in data:
    print(batch)
    break
 
# Not sure this makes sense if gym version is above 0.24
import d4rl
import gym
gym.make("halfcheetah-medium-v2")
"""

If this cannot be done, I think this PR still makes sense (one should have access to the data, regardless of the gym version)

To be discussed to: do we want to make the D4RL-independent download default? It'll be bc-breaking so I would rather make a progressive change

cc @BY571

@vmoens
Copy link
Contributor

vmoens commented Oct 4, 2023

Update on this:
I tested if updating gym would work but in short: no, it does not (as expected).
So this PR solves the problem of accessing the offline data but the env cannot be created after an upgrade to a more recent gym version. I guess this isn't something we can patch so I'm happy to merge the feature as is!

@vmoens vmoens merged commit 3018810 into pytorch:main Oct 4, 2023
vmoens added a commit to hyerra/rl that referenced this pull request Oct 10, 2023
Co-authored-by: Mateusz Guzek <[email protected]>
Co-authored-by: vmoens <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants