Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix offline CatFrames #1953

Merged
merged 6 commits into from
Feb 24, 2024
Merged

[BugFix] Fix offline CatFrames #1953

merged 6 commits into from
Feb 24, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 22, 2024 •
edited
Loading

Currently offline catframes is broken

  • it does not return cat but stacks
  • it ignores reset signals

I'm trying to fix this

Example code:

from torchrl.envs import GymEnv, TransformedEnv, CatFrames, SerialEnv

env = SerialEnv(3, lambda: TransformedEnv(GymEnv("CartPole-v1"), CatFrames(dim=-1, N=4, in_keys=["observation"], out_keys=["observation_cat"], padding="constant")))

r = env.rollout(100, break_when_any_done=False)


c = CatFrames(dim=-1, N=4, in_keys=["observation", ("next", "observation")], out_keys=["observation_cat2", ("next", "observation_cat2")], padding="constant")

r2 = c(r)

assert (r2["observation_cat2"] == r2["observation_cat"]).all()

assert (r2["next", "observation_cat2"] == r2["next", "observation_cat"]).all()

cc @btx0424 @BY571

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 22, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1953

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 34bd5cb with merge base 7782751 (image):

NEW FAILURE - The following job has failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 22, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 61.9913ms 61.5475ms 16.2476 Ops/s 16.0364 Ops/s $\color{#35bf28}+1.32\%$
test_sync 43.2616ms 33.8044ms 29.5820 Ops/s 29.6980 Ops/s $\color{#d91a1a}-0.39\%$
test_async 61.9284ms 30.6311ms 32.6466 Ops/s 32.1754 Ops/s $\color{#35bf28}+1.46\%$
test_simple 0.4844s 0.4343s 2.3024 Ops/s 2.2856 Ops/s $\color{#35bf28}+0.73\%$
test_transformed 0.6369s 0.5858s 1.7071 Ops/s 1.7108 Ops/s $\color{#d91a1a}-0.22\%$
test_serial 1.4835s 1.4434s 0.6928 Ops/s 0.6910 Ops/s $\color{#35bf28}+0.27\%$
test_parallel 1.4940s 1.4254s 0.7016 Ops/s 0.7208 Ops/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[True-True-True-True-True] 0.1638ms 21.4756μs 46.5645 KOps/s 46.4944 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-True-True-True-False] 36.6390μs 13.2143μs 75.6759 KOps/s 76.5332 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-True-True-False-True] 36.2080μs 12.6031μs 79.3458 KOps/s 79.4404 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-True-False-False] 35.8470μs 7.6804μs 130.2018 KOps/s 133.5372 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[True-True-False-True-True] 69.3000μs 22.7190μs 44.0160 KOps/s 44.0632 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-True-False-True-False] 41.2970μs 14.3859μs 69.5123 KOps/s 70.5404 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[True-True-False-False-True] 42.3390μs 13.8255μs 72.3302 KOps/s 72.2343 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-True-False-False-False] 32.6820μs 8.9759μs 111.4092 KOps/s 113.5794 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[True-False-True-True-True] 61.9650μs 24.2901μs 41.1691 KOps/s 41.4172 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-False-True-True-False] 47.0080μs 15.9150μs 62.8338 KOps/s 64.2310 KOps/s $\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-False-True-False-True] 43.1610μs 13.8295μs 72.3093 KOps/s 72.7267 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-False-True-False-False] 34.4350μs 9.0544μs 110.4432 KOps/s 113.5785 KOps/s $\color{#d91a1a}-2.76\%$
test_step_mdp_speed[True-False-False-True-True] 50.4240μs 25.5312μs 39.1678 KOps/s 39.7990 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[True-False-False-True-False] 42.1490μs 17.2149μs 58.0893 KOps/s 59.3941 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-False-False-False-True] 57.5660μs 15.0957μs 66.2441 KOps/s 66.3750 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-False-False-False-False] 45.1360μs 10.1281μs 98.7355 KOps/s 98.6032 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-True-True-True-True] 75.5210μs 24.1791μs 41.3581 KOps/s 41.8435 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-True-True-True-False] 42.4990μs 15.9178μs 62.8226 KOps/s 64.5641 KOps/s $\color{#d91a1a}-2.70\%$
test_step_mdp_speed[False-True-True-False-True] 38.6230μs 16.1103μs 62.0721 KOps/s 62.9189 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[False-True-True-False-False] 44.1400μs 10.1229μs 98.7861 KOps/s 99.1960 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-True-False-True-True] 55.8340μs 25.6786μs 38.9430 KOps/s 38.7740 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-False-True-False] 43.2510μs 17.0954μs 58.4953 KOps/s 59.5621 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-True-False-False-True] 73.7180μs 17.1963μs 58.1520 KOps/s 58.2014 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-False-False-False] 35.2560μs 11.3770μs 87.8964 KOps/s 89.4251 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[False-False-True-True-True] 56.6460μs 26.2598μs 38.0811 KOps/s 37.6553 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-False-True-True-False] 47.4990μs 18.3519μs 54.4902 KOps/s 55.8117 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[False-False-True-False-True] 45.1040μs 17.1642μs 58.2607 KOps/s 58.4930 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-False-True-False-False] 37.7510μs 11.4930μs 87.0097 KOps/s 88.1071 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-False-False-True-True] 57.8080μs 27.8389μs 35.9210 KOps/s 36.2525 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-False-False-True-False] 53.0090μs 19.2242μs 52.0179 KOps/s 52.2784 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-False-False-False-True] 49.9130μs 18.0556μs 55.3844 KOps/s 55.1109 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-False-False-False] 40.1150μs 12.3808μs 80.7705 KOps/s 81.0149 KOps/s $\color{#d91a1a}-0.30\%$
test_values[generalized_advantage_estimate-True-True] 11.4683ms 9.1298ms 109.5313 Ops/s 110.8997 Ops/s $\color{#d91a1a}-1.23\%$
test_values[vec_generalized_advantage_estimate-True-True] 49.1292ms 33.5294ms 29.8246 Ops/s 28.4724 Ops/s $\color{#35bf28}+4.75\%$
test_values[td0_return_estimate-False-False] 0.2137ms 0.1627ms 6.1477 KOps/s 5.9747 KOps/s $\color{#35bf28}+2.89\%$
test_values[td1_return_estimate-False-False] 22.9490ms 22.3951ms 44.6527 Ops/s 43.0266 Ops/s $\color{#35bf28}+3.78\%$
test_values[vec_td1_return_estimate-False-False] 47.7237ms 33.2936ms 30.0358 Ops/s 28.3517 Ops/s $\textbf{\color{#35bf28}+5.94\%}$
test_values[td_lambda_return_estimate-True-False] 35.5702ms 32.4723ms 30.7954 Ops/s 29.8566 Ops/s $\color{#35bf28}+3.14\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.4483ms 32.9587ms 30.3410 Ops/s 28.2089 Ops/s $\textbf{\color{#35bf28}+7.56\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.0768ms 7.9784ms 125.3378 Ops/s 124.2751 Ops/s $\color{#35bf28}+0.86\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2845ms 1.9839ms 504.0524 Ops/s 501.4168 Ops/s $\color{#35bf28}+0.53\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4207ms 0.3495ms 2.8612 KOps/s 2.8309 KOps/s $\color{#35bf28}+1.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 41.4743ms 38.1655ms 26.2016 Ops/s 22.3610 Ops/s $\textbf{\color{#35bf28}+17.18\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.4162ms 3.0347ms 329.5208 Ops/s 331.3700 Ops/s $\color{#d91a1a}-0.56\%$
test_dqn_speed 7.7155ms 1.3815ms 723.8462 Ops/s 720.3252 Ops/s $\color{#35bf28}+0.49\%$
test_ddpg_speed 3.5171ms 2.7765ms 360.1603 Ops/s 357.5379 Ops/s $\color{#35bf28}+0.73\%$
test_sac_speed 78.0657ms 9.0115ms 110.9698 Ops/s 119.1527 Ops/s $\textbf{\color{#d91a1a}-6.87\%}$
test_redq_speed 19.2669ms 12.9969ms 76.9417 Ops/s 76.6102 Ops/s $\color{#35bf28}+0.43\%$
test_redq_deprec_speed 14.8658ms 13.1036ms 76.3148 Ops/s 76.7878 Ops/s $\color{#d91a1a}-0.62\%$
test_td3_speed 11.2589ms 8.3421ms 119.8737 Ops/s 119.9720 Ops/s $\color{#d91a1a}-0.08\%$
test_cql_speed 37.5221ms 36.2722ms 27.5693 Ops/s 27.6375 Ops/s $\color{#d91a1a}-0.25\%$
test_a2c_speed 8.1309ms 7.3706ms 135.6740 Ops/s 136.3918 Ops/s $\color{#d91a1a}-0.53\%$
test_ppo_speed 8.9251ms 7.8798ms 126.9064 Ops/s 131.2029 Ops/s $\color{#d91a1a}-3.27\%$
test_reinforce_speed 7.6676ms 6.6435ms 150.5227 Ops/s 152.3341 Ops/s $\color{#d91a1a}-1.19\%$
test_iql_speed 33.4120ms 32.6355ms 30.6415 Ops/s 30.8096 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9013ms 2.1656ms 461.7631 Ops/s 471.3229 Ops/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1497ms 0.4864ms 2.0560 KOps/s 2.0460 KOps/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7142ms 0.4662ms 2.1448 KOps/s 2.1827 KOps/s $\color{#d91a1a}-1.74\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.5440ms 2.3794ms 420.2813 Ops/s 485.4171 Ops/s $\textbf{\color{#d91a1a}-13.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7159ms 0.4901ms 2.0403 KOps/s 2.0855 KOps/s $\color{#d91a1a}-2.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.6488ms 0.4742ms 2.1090 KOps/s 2.1956 KOps/s $\color{#d91a1a}-3.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5828ms 2.3747ms 421.1012 Ops/s 452.5690 Ops/s $\textbf{\color{#d91a1a}-6.95\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0336ms 0.6036ms 1.6568 KOps/s 1.6664 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8106ms 0.5786ms 1.7284 KOps/s 1.7509 KOps/s $\color{#d91a1a}-1.28\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1868ms 2.0913ms 478.1804 Ops/s 477.2602 Ops/s $\color{#35bf28}+0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8853ms 0.4863ms 2.0566 KOps/s 2.0423 KOps/s $\color{#35bf28}+0.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 91.9896ms 0.5535ms 1.8068 KOps/s 1.8106 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3453ms 2.1833ms 458.0242 Ops/s 474.9150 Ops/s $\color{#d91a1a}-3.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6139ms 0.4828ms 2.0714 KOps/s 2.0735 KOps/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8963ms 0.4668ms 2.1424 KOps/s 2.1802 KOps/s $\color{#d91a1a}-1.73\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3702ms 2.2463ms 445.1676 Ops/s 447.6169 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8398ms 0.6016ms 1.6623 KOps/s 1.6563 KOps/s $\color{#35bf28}+0.36\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9235ms 0.5809ms 1.7213 KOps/s 1.7517 KOps/s $\color{#d91a1a}-1.73\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 89.1169ms 5.3108ms 188.2948 Ops/s 190.8832 Ops/s $\color{#d91a1a}-1.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.4696ms 11.7429ms 85.1575 Ops/s 83.3075 Ops/s $\color{#35bf28}+2.22\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0958ms 1.0213ms 979.1912 Ops/s 991.5312 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 92.7567ms 7.1792ms 139.2922 Ops/s 146.4023 Ops/s $\color{#d91a1a}-4.86\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.0795ms 11.7893ms 84.8224 Ops/s 84.1836 Ops/s $\color{#35bf28}+0.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5726ms 1.0325ms 968.5269 Ops/s 979.8322 Ops/s $\color{#d91a1a}-1.15\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 92.8001ms 7.4305ms 134.5802 Ops/s 136.7021 Ops/s $\color{#d91a1a}-1.55\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.0131ms 12.1484ms 82.3153 Ops/s 81.6104 Ops/s $\color{#35bf28}+0.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.3812ms 1.3351ms 748.9958 Ops/s 750.7874 Ops/s $\color{#d91a1a}-0.24\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1162s 0.1160s 8.6171 Ops/s 8.7153 Ops/s $\color{#d91a1a}-1.13\%$
test_sync 96.2082ms 95.9367ms 10.4235 Ops/s 10.4284 Ops/s $\color{#d91a1a}-0.05\%$
test_async 0.1815s 91.8340ms 10.8892 Ops/s 10.9117 Ops/s $\color{#d91a1a}-0.21\%$
test_single_pixels 0.1301s 0.1287s 7.7698 Ops/s 7.5925 Ops/s $\color{#35bf28}+2.34\%$
test_sync_pixels 83.0539ms 81.0182ms 12.3429 Ops/s 12.4194 Ops/s $\color{#d91a1a}-0.62\%$
test_async_pixels 0.1510s 63.3927ms 15.7747 Ops/s 15.0452 Ops/s $\color{#35bf28}+4.85\%$
test_simple 0.9069s 0.8484s 1.1787 Ops/s 1.1941 Ops/s $\color{#d91a1a}-1.29\%$
test_transformed 1.1335s 1.0745s 0.9307 Ops/s 0.9393 Ops/s $\color{#d91a1a}-0.91\%$
test_serial 2.5303s 2.4772s 0.4037 Ops/s 0.4068 Ops/s $\color{#d91a1a}-0.77\%$
test_parallel 2.1686s 2.1027s 0.4756 Ops/s 0.4783 Ops/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-True-True-True-True] 0.1126ms 33.8766μs 29.5189 KOps/s 30.4003 KOps/s $\color{#d91a1a}-2.90\%$
test_step_mdp_speed[True-True-True-True-False] 39.7410μs 19.8947μs 50.2645 KOps/s 51.3360 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[True-True-True-False-True] 44.8610μs 18.6851μs 53.5186 KOps/s 54.7497 KOps/s $\color{#d91a1a}-2.25\%$
test_step_mdp_speed[True-True-True-False-False] 27.3510μs 11.2998μs 88.4971 KOps/s 90.7073 KOps/s $\color{#d91a1a}-2.44\%$
test_step_mdp_speed[True-True-False-True-True] 69.9520μs 35.3364μs 28.2994 KOps/s 29.0972 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-True-False-True-False] 43.8710μs 21.7824μs 45.9086 KOps/s 46.9007 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[True-True-False-False-True] 71.0810μs 20.6742μs 48.3695 KOps/s 49.0483 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[True-True-False-False-False] 28.8510μs 13.2793μs 75.3051 KOps/s 77.0056 KOps/s $\color{#d91a1a}-2.21\%$
test_step_mdp_speed[True-False-True-True-True] 53.8120μs 37.0588μs 26.9841 KOps/s 27.7148 KOps/s $\color{#d91a1a}-2.64\%$
test_step_mdp_speed[True-False-True-True-False] 40.6810μs 23.8062μs 42.0059 KOps/s 43.2626 KOps/s $\color{#d91a1a}-2.90\%$
test_step_mdp_speed[True-False-True-False-True] 46.1910μs 20.9298μs 47.7788 KOps/s 48.8092 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[True-False-True-False-False] 29.1100μs 13.0368μs 76.7057 KOps/s 76.9402 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-False-True-True] 76.3410μs 39.5720μs 25.2704 KOps/s 26.4098 KOps/s $\color{#d91a1a}-4.31\%$
test_step_mdp_speed[True-False-False-True-False] 43.1910μs 25.1963μs 39.6883 KOps/s 40.2543 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-False-False-False-True] 39.5000μs 22.5569μs 44.3322 KOps/s 45.6049 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[True-False-False-False-False] 38.4710μs 14.7634μs 67.7349 KOps/s 68.2075 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-True-True-True] 78.2520μs 37.2423μs 26.8512 KOps/s 27.2481 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[False-True-True-True-False] 39.6720μs 23.4304μs 42.6797 KOps/s 43.2150 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-True-False-True] 41.4400μs 24.4906μs 40.8320 KOps/s 41.2165 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-True-True-False-False] 30.6290μs 15.0213μs 66.5722 KOps/s 67.9384 KOps/s $\color{#d91a1a}-2.01\%$
test_step_mdp_speed[False-True-False-True-True] 68.1910μs 39.6539μs 25.2182 KOps/s 26.1816 KOps/s $\color{#d91a1a}-3.68\%$
test_step_mdp_speed[False-True-False-True-False] 48.7210μs 25.7070μs 38.9000 KOps/s 40.3467 KOps/s $\color{#d91a1a}-3.59\%$
test_step_mdp_speed[False-True-False-False-True] 45.4110μs 26.3205μs 37.9932 KOps/s 38.8741 KOps/s $\color{#d91a1a}-2.27\%$
test_step_mdp_speed[False-True-False-False-False] 32.9210μs 16.7726μs 59.6209 KOps/s 61.1780 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-False-True-True-True] 71.6810μs 40.5855μs 24.6393 KOps/s 25.0754 KOps/s $\color{#d91a1a}-1.74\%$
test_step_mdp_speed[False-False-True-True-False] 54.5810μs 27.4043μs 36.4906 KOps/s 37.0697 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-False-True-False-True] 45.2810μs 26.3774μs 37.9112 KOps/s 38.6212 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[False-False-True-False-False] 35.8500μs 16.8543μs 59.3320 KOps/s 60.4594 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-False-False-True-True] 67.6620μs 42.3291μs 23.6244 KOps/s 24.4000 KOps/s $\color{#d91a1a}-3.18\%$
test_step_mdp_speed[False-False-False-True-False] 46.9010μs 29.0449μs 34.4295 KOps/s 35.1799 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[False-False-False-False-True] 50.8000μs 27.8964μs 35.8470 KOps/s 36.8088 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-False-False-False] 36.3500μs 18.6345μs 53.6638 KOps/s 55.2955 KOps/s $\color{#d91a1a}-2.95\%$
test_values[generalized_advantage_estimate-True-True] 25.5680ms 24.9450ms 40.0882 Ops/s 40.1945 Ops/s $\color{#d91a1a}-0.26\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.6609ms 3.2413ms 308.5186 Ops/s 310.4591 Ops/s $\color{#d91a1a}-0.63\%$
test_values[td0_return_estimate-False-False] 0.1074ms 64.2253μs 15.5702 KOps/s 16.2005 KOps/s $\color{#d91a1a}-3.89\%$
test_values[td1_return_estimate-False-False] 56.9104ms 54.4643ms 18.3606 Ops/s 18.6101 Ops/s $\color{#d91a1a}-1.34\%$
test_values[vec_td1_return_estimate-False-False] 2.0770ms 1.7637ms 566.9833 Ops/s 568.9897 Ops/s $\color{#d91a1a}-0.35\%$
test_values[td_lambda_return_estimate-True-False] 90.4225ms 88.7848ms 11.2632 Ops/s 11.3859 Ops/s $\color{#d91a1a}-1.08\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.9132ms 1.8004ms 555.4327 Ops/s 556.6772 Ops/s $\color{#d91a1a}-0.22\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.9963ms 24.6413ms 40.5823 Ops/s 42.9958 Ops/s $\textbf{\color{#d91a1a}-5.61\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8999ms 0.7153ms 1.3980 KOps/s 1.4177 KOps/s $\color{#d91a1a}-1.39\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7290ms 0.6590ms 1.5174 KOps/s 1.5357 KOps/s $\color{#d91a1a}-1.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5092ms 1.4640ms 683.0563 Ops/s 685.9565 Ops/s $\color{#d91a1a}-0.42\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9595ms 0.6805ms 1.4695 KOps/s 1.4836 KOps/s $\color{#d91a1a}-0.95\%$
test_dqn_speed 8.4155ms 1.5183ms 658.6153 Ops/s 615.6391 Ops/s $\textbf{\color{#35bf28}+6.98\%}$
test_ddpg_speed 3.4679ms 2.9167ms 342.8573 Ops/s 345.2407 Ops/s $\color{#d91a1a}-0.69\%$
test_sac_speed 8.7581ms 8.2879ms 120.6574 Ops/s 119.5361 Ops/s $\color{#35bf28}+0.94\%$
test_redq_speed 12.6638ms 10.4558ms 95.6408 Ops/s 94.9021 Ops/s $\color{#35bf28}+0.78\%$
test_redq_deprec_speed 11.9142ms 11.4316ms 87.4767 Ops/s 87.9047 Ops/s $\color{#d91a1a}-0.49\%$
test_td3_speed 16.5488ms 8.4502ms 118.3408 Ops/s 121.9279 Ops/s $\color{#d91a1a}-2.94\%$
test_cql_speed 27.2491ms 25.7901ms 38.7746 Ops/s 39.2519 Ops/s $\color{#d91a1a}-1.22\%$
test_a2c_speed 6.3103ms 5.6510ms 176.9612 Ops/s 177.0828 Ops/s $\color{#d91a1a}-0.07\%$
test_ppo_speed 6.2115ms 6.0041ms 166.5533 Ops/s 167.0191 Ops/s $\color{#d91a1a}-0.28\%$
test_reinforce_speed 5.8273ms 4.6709ms 214.0905 Ops/s 216.5613 Ops/s $\color{#d91a1a}-1.14\%$
test_iql_speed 20.9553ms 19.8399ms 50.4036 Ops/s 50.0676 Ops/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1550ms 2.9380ms 340.3677 Ops/s 352.3938 Ops/s $\color{#d91a1a}-3.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4209ms 0.5463ms 1.8305 KOps/s 1.8466 KOps/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7400ms 0.5195ms 1.9248 KOps/s 1.9308 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1374ms 2.9445ms 339.6140 Ops/s 352.9902 Ops/s $\color{#d91a1a}-3.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3376ms 0.5363ms 1.8648 KOps/s 1.8705 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6920ms 0.5144ms 1.9440 KOps/s 1.9587 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1593ms 3.0547ms 327.3608 Ops/s 337.9641 Ops/s $\color{#d91a1a}-3.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3719ms 0.6691ms 1.4946 KOps/s 1.4932 KOps/s $\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8342ms 0.6470ms 1.5456 KOps/s 1.5472 KOps/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0167ms 2.9218ms 342.2515 Ops/s 352.1622 Ops/s $\color{#d91a1a}-2.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1975ms 0.5462ms 1.8309 KOps/s 1.8299 KOps/s $\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7205ms 0.5225ms 1.9140 KOps/s 1.6443 KOps/s $\textbf{\color{#35bf28}+16.40\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1328ms 2.9579ms 338.0825 Ops/s 349.3705 Ops/s $\color{#d91a1a}-3.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2763ms 0.5375ms 1.8603 KOps/s 1.8753 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7339ms 0.5150ms 1.9418 KOps/s 1.9283 KOps/s $\color{#35bf28}+0.70\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2601ms 3.0691ms 325.8335 Ops/s 335.2157 Ops/s $\color{#d91a1a}-2.80\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2123ms 0.6727ms 1.4866 KOps/s 1.4885 KOps/s $\color{#d91a1a}-0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8081ms 0.6480ms 1.5433 KOps/s 1.5356 KOps/s $\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1171s 8.9357ms 111.9104 Ops/s 140.6124 Ops/s $\textbf{\color{#d91a1a}-20.41\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 20.1023ms 15.3437ms 65.1735 Ops/s 67.6968 Ops/s $\color{#d91a1a}-3.73\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3150ms 1.1604ms 861.7734 Ops/s 923.1445 Ops/s $\textbf{\color{#d91a1a}-6.65\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1019s 8.5969ms 116.3208 Ops/s 148.8131 Ops/s $\textbf{\color{#d91a1a}-21.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.1002ms 15.3469ms 65.1596 Ops/s 59.8430 Ops/s $\textbf{\color{#35bf28}+8.88\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.0825ms 1.2763ms 783.5348 Ops/s 901.4784 Ops/s $\textbf{\color{#d91a1a}-13.08\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1034s 7.0751ms 141.3400 Ops/s 140.2600 Ops/s $\color{#35bf28}+0.77\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.4680ms 15.4881ms 64.5656 Ops/s 66.4531 Ops/s $\color{#d91a1a}-2.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.7893ms 1.5123ms 661.2437 Ops/s 691.8120 Ops/s $\color{#d91a1a}-4.42\%$

@vmoens vmoens added the bug Something isn't working label Feb 23, 2024
@vmoens vmoens merged commit 931f70a into main Feb 24, 2024
65 of 68 checks passed
@vmoens vmoens deleted the fix-catframes branch February 27, 2024 00:47
vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[BugFix] Fix offline CatFrames (#1953)
8cb1ee1
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot