Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] better device consistency in EGreedy #1867

Merged
merged 3 commits into from
Feb 4, 2024
Merged

[BugFix] better device consistency in EGreedy #1867

merged 3 commits into from
Feb 4, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 3, 2024 •
edited
Loading

Improves egreedy device when specs have no device

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 3, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1867

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 7aab8de with merge base 0672359 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 3, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 3, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1296s 67.4021ms 14.8363 Ops/s 15.9172 Ops/s $\textbf{\color{#d91a1a}-6.79\%}$
test_sync 47.4649ms 35.7333ms 27.9851 Ops/s 29.7624 Ops/s $\textbf{\color{#d91a1a}-5.97\%}$
test_async 0.1332s 32.7483ms 30.5360 Ops/s 30.7204 Ops/s $\color{#d91a1a}-0.60\%$
test_simple 0.4934s 0.4367s 2.2898 Ops/s 2.2940 Ops/s $\color{#d91a1a}-0.18\%$
test_transformed 0.6532s 0.6038s 1.6562 Ops/s 1.6733 Ops/s $\color{#d91a1a}-1.02\%$
test_serial 1.4237s 1.3755s 0.7270 Ops/s 0.7248 Ops/s $\color{#35bf28}+0.30\%$
test_parallel 1.3474s 1.2476s 0.8016 Ops/s 0.7961 Ops/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-True-True-True] 0.1278ms 21.4919μs 46.5292 KOps/s 48.1489 KOps/s $\color{#d91a1a}-3.36\%$
test_step_mdp_speed[True-True-True-True-False] 38.5020μs 13.1854μs 75.8417 KOps/s 77.0159 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-True-True-False-True] 41.1270μs 12.6832μs 78.8446 KOps/s 80.9380 KOps/s $\color{#d91a1a}-2.59\%$
test_step_mdp_speed[True-True-True-False-False] 0.2091ms 8.1262μs 123.0587 KOps/s 132.6209 KOps/s $\textbf{\color{#d91a1a}-7.21\%}$
test_step_mdp_speed[True-True-False-True-True] 66.1030μs 23.0939μs 43.3015 KOps/s 44.5563 KOps/s $\color{#d91a1a}-2.82\%$
test_step_mdp_speed[True-True-False-True-False] 39.7950μs 14.4385μs 69.2591 KOps/s 69.3540 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-True-False-False-True] 35.4960μs 13.8651μs 72.1236 KOps/s 73.6450 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-True-False-False-False] 30.9970μs 8.9402μs 111.8549 KOps/s 113.6019 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[True-False-True-True-True] 68.3680μs 24.2215μs 41.2857 KOps/s 42.4389 KOps/s $\color{#d91a1a}-2.72\%$
test_step_mdp_speed[True-False-True-True-False] 43.6420μs 15.6232μs 64.0074 KOps/s 64.2118 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-True-False-True] 39.5140μs 13.9462μs 71.7041 KOps/s 74.3772 KOps/s $\color{#d91a1a}-3.59\%$
test_step_mdp_speed[True-False-True-False-False] 29.9450μs 8.8907μs 112.4772 KOps/s 113.2032 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[True-False-False-True-True] 68.5580μs 25.3806μs 39.4002 KOps/s 40.9248 KOps/s $\color{#d91a1a}-3.73\%$
test_step_mdp_speed[True-False-False-True-False] 0.1634ms 17.1615μs 58.2701 KOps/s 59.7505 KOps/s $\color{#d91a1a}-2.48\%$
test_step_mdp_speed[True-False-False-False-True] 41.8680μs 15.0320μs 66.5248 KOps/s 68.1292 KOps/s $\color{#d91a1a}-2.35\%$
test_step_mdp_speed[True-False-False-False-False] 45.5550μs 10.0886μs 99.1220 KOps/s 101.2396 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[False-True-True-True-True] 56.2350μs 24.3439μs 41.0781 KOps/s 42.9460 KOps/s $\color{#d91a1a}-4.35\%$
test_step_mdp_speed[False-True-True-True-False] 56.3050μs 15.7060μs 63.6697 KOps/s 64.9476 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-True-True-False-True] 70.7320μs 16.0837μs 62.1749 KOps/s 64.7468 KOps/s $\color{#d91a1a}-3.97\%$
test_step_mdp_speed[False-True-True-False-False] 37.5700μs 10.0374μs 99.6276 KOps/s 100.4259 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-False-True-True] 62.9070μs 25.4339μs 39.3176 KOps/s 40.5262 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[False-True-False-True-False] 51.9870μs 16.8943μs 59.1917 KOps/s 60.1720 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-True-False-False-True] 42.0280μs 17.2197μs 58.0731 KOps/s 60.1072 KOps/s $\color{#d91a1a}-3.38\%$
test_step_mdp_speed[False-True-False-False-False] 37.4300μs 11.2396μs 88.9714 KOps/s 89.7866 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-False-True-True-True] 67.6670μs 26.9270μs 37.1375 KOps/s 38.4744 KOps/s $\color{#d91a1a}-3.47\%$
test_step_mdp_speed[False-False-True-True-False] 49.3320μs 18.1766μs 55.0159 KOps/s 55.7638 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-False-True-False-True] 44.7530μs 17.2914μs 57.8323 KOps/s 59.4227 KOps/s $\color{#d91a1a}-2.68\%$
test_step_mdp_speed[False-False-True-False-False] 36.6080μs 11.2965μs 88.5232 KOps/s 88.5223 KOps/s $+0.00\%$
test_step_mdp_speed[False-False-False-True-True] 68.9190μs 27.7485μs 36.0380 KOps/s 36.7545 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-False-False-True-False] 67.1950μs 19.1817μs 52.1329 KOps/s 52.4882 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[False-False-False-False-True] 48.2900μs 18.2399μs 54.8250 KOps/s 56.1972 KOps/s $\color{#d91a1a}-2.44\%$
test_step_mdp_speed[False-False-False-False-False] 37.9510μs 12.3418μs 81.0258 KOps/s 81.6987 KOps/s $\color{#d91a1a}-0.82\%$
test_values[generalized_advantage_estimate-True-True] 10.4046ms 9.7833ms 102.2146 Ops/s 106.1997 Ops/s $\color{#d91a1a}-3.75\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.9318ms 34.9562ms 28.6072 Ops/s 28.5527 Ops/s $\color{#35bf28}+0.19\%$
test_values[td0_return_estimate-False-False] 0.2356ms 0.1660ms 6.0258 KOps/s 6.0461 KOps/s $\color{#d91a1a}-0.34\%$
test_values[td1_return_estimate-False-False] 26.7828ms 23.6252ms 42.3278 Ops/s 43.7565 Ops/s $\color{#d91a1a}-3.27\%$
test_values[vec_td1_return_estimate-False-False] 36.8025ms 35.1658ms 28.4367 Ops/s 28.4663 Ops/s $\color{#d91a1a}-0.10\%$
test_values[td_lambda_return_estimate-True-False] 34.2763ms 33.8496ms 29.5425 Ops/s 30.1170 Ops/s $\color{#d91a1a}-1.91\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.5207ms 35.0384ms 28.5401 Ops/s 28.3642 Ops/s $\color{#35bf28}+0.62\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.2569ms 8.3776ms 119.3655 Ops/s 123.4562 Ops/s $\color{#d91a1a}-3.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3329ms 1.8956ms 527.5393 Ops/s 511.0000 Ops/s $\color{#35bf28}+3.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4263ms 0.3442ms 2.9054 KOps/s 2.7345 KOps/s $\textbf{\color{#35bf28}+6.25\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.4976ms 46.1770ms 21.6558 Ops/s 22.7320 Ops/s $\color{#d91a1a}-4.73\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5093ms 3.0182ms 331.3228 Ops/s 332.4117 Ops/s $\color{#d91a1a}-0.33\%$
test_dqn_speed 78.0994ms 6.4206ms 155.7482 Ops/s 166.8415 Ops/s $\textbf{\color{#d91a1a}-6.65\%}$
test_ddpg_speed 13.3092ms 11.7015ms 85.4592 Ops/s 79.4402 Ops/s $\textbf{\color{#35bf28}+7.58\%}$
test_sac_speed 24.7395ms 23.7802ms 42.0518 Ops/s 42.2748 Ops/s $\color{#d91a1a}-0.53\%$
test_redq_speed 15.1166ms 14.0538ms 71.1551 Ops/s 72.4234 Ops/s $\color{#d91a1a}-1.75\%$
test_redq_deprec_speed 23.3967ms 21.7920ms 45.8883 Ops/s 45.8483 Ops/s $\color{#35bf28}+0.09\%$
test_td3_speed 24.8515ms 16.6527ms 60.0504 Ops/s 60.2915 Ops/s $\color{#d91a1a}-0.40\%$
test_cql_speed 75.8023ms 73.8282ms 13.5450 Ops/s 13.5129 Ops/s $\color{#35bf28}+0.24\%$
test_a2c_speed 22.9535ms 21.9720ms 45.5124 Ops/s 45.4879 Ops/s $\color{#35bf28}+0.05\%$
test_ppo_speed 24.2936ms 22.3131ms 44.8166 Ops/s 44.8053 Ops/s $\color{#35bf28}+0.03\%$
test_reinforce_speed 21.7715ms 21.1894ms 47.1934 Ops/s 47.1030 Ops/s $\color{#35bf28}+0.19\%$
test_iql_speed 63.3845ms 56.5742ms 17.6759 Ops/s 17.4571 Ops/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1232ms 2.6826ms 372.7724 Ops/s 377.4484 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8456ms 0.5027ms 1.9893 KOps/s 2.0057 KOps/s $\color{#d91a1a}-0.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7855ms 0.4751ms 2.1047 KOps/s 2.1142 KOps/s $\color{#d91a1a}-0.45\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9603ms 2.6974ms 370.7281 Ops/s 374.7321 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6345ms 0.4948ms 2.0210 KOps/s 2.0206 KOps/s $\color{#35bf28}+0.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7881ms 0.4743ms 2.1081 KOps/s 2.1257 KOps/s $\color{#d91a1a}-0.83\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.7076ms 2.4374ms 410.2750 Ops/s 413.3977 Ops/s $\color{#d91a1a}-0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8607ms 0.6113ms 1.6360 KOps/s 1.6259 KOps/s $\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9378ms 0.5862ms 1.7058 KOps/s 1.7090 KOps/s $\color{#d91a1a}-0.18\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8679ms 2.6709ms 374.4070 Ops/s 378.0626 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6084ms 0.5021ms 1.9915 KOps/s 1.9865 KOps/s $\color{#35bf28}+0.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5847ms 0.4749ms 2.1059 KOps/s 2.1097 KOps/s $\color{#d91a1a}-0.18\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1969ms 2.7494ms 363.7210 Ops/s 379.8963 Ops/s $\color{#d91a1a}-4.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8031ms 0.5016ms 1.9938 KOps/s 2.0051 KOps/s $\color{#d91a1a}-0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5800ms 0.4704ms 2.1260 KOps/s 2.1331 KOps/s $\color{#d91a1a}-0.34\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7701ms 2.4206ms 413.1144 Ops/s 416.5648 Ops/s $\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8518ms 0.6139ms 1.6289 KOps/s 1.6207 KOps/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7426ms 0.5845ms 1.7107 KOps/s 1.7003 KOps/s $\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1014s 8.0302ms 124.5299 Ops/s 128.0265 Ops/s $\color{#d91a1a}-2.73\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.4331ms 13.1752ms 75.9000 Ops/s 75.6378 Ops/s $\color{#35bf28}+0.35\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.2504ms 2.5970ms 385.0667 Ops/s 393.5594 Ops/s $\color{#d91a1a}-2.16\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 94.6516ms 9.6775ms 103.3329 Ops/s 103.7635 Ops/s $\color{#d91a1a}-0.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.2178ms 13.2265ms 75.6058 Ops/s 75.6634 Ops/s $\color{#d91a1a}-0.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.9559ms 2.5572ms 391.0507 Ops/s 399.8934 Ops/s $\color{#d91a1a}-2.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 97.8403ms 10.1037ms 98.9733 Ops/s 101.1233 Ops/s $\color{#d91a1a}-2.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.3268ms 13.3812ms 74.7317 Ops/s 74.2650 Ops/s $\color{#35bf28}+0.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.7057ms 2.7135ms 368.5327 Ops/s 372.6575 Ops/s $\color{#d91a1a}-1.11\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 3, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1227s 0.1212s 8.2539 Ops/s 8.2818 Ops/s $\color{#d91a1a}-0.34\%$
test_sync 0.1378s 0.1360s 7.3526 Ops/s 7.2667 Ops/s $\color{#35bf28}+1.18\%$
test_async 0.3314s 0.1328s 7.5282 Ops/s 7.6235 Ops/s $\color{#d91a1a}-1.25\%$
test_single_pixels 0.1430s 0.1426s 7.0102 Ops/s 7.2591 Ops/s $\color{#d91a1a}-3.43\%$
test_sync_pixels 0.1034s 0.1016s 9.8469 Ops/s 9.7013 Ops/s $\color{#35bf28}+1.50\%$
test_async_pixels 0.2662s 97.7884ms 10.2262 Ops/s 10.2838 Ops/s $\color{#d91a1a}-0.56\%$
test_simple 0.9449s 0.8824s 1.1332 Ops/s 1.1553 Ops/s $\color{#d91a1a}-1.91\%$
test_transformed 1.1853s 1.1228s 0.8906 Ops/s 0.8930 Ops/s $\color{#d91a1a}-0.27\%$
test_serial 2.4494s 2.4480s 0.4085 Ops/s 0.4062 Ops/s $\color{#35bf28}+0.56\%$
test_parallel 3.3869s 3.2242s 0.3102 Ops/s 0.3143 Ops/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-True-True-True-True] 90.2920μs 32.9081μs 30.3877 KOps/s 30.2301 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-True-True-True-False] 41.9610μs 19.7270μs 50.6919 KOps/s 50.7553 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-True-False-True] 35.7910μs 18.8168μs 53.1440 KOps/s 52.3192 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-True-True-False-False] 37.7200μs 11.3969μs 87.7431 KOps/s 88.7069 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-False-True-True] 54.6920μs 35.2928μs 28.3344 KOps/s 28.3685 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-False-True-False] 45.7200μs 21.5306μs 46.4456 KOps/s 45.5035 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-True-False-False-True] 44.9110μs 20.7599μs 48.1698 KOps/s 47.5858 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-False-False-False] 27.8510μs 13.2770μs 75.3183 KOps/s 75.3625 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-False-True-True-True] 72.3310μs 37.0731μs 26.9738 KOps/s 26.8626 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-False-True-True-False] 41.3410μs 23.9200μs 41.8061 KOps/s 41.8778 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-False-True-False-True] 43.6420μs 20.6982μs 48.3134 KOps/s 48.0202 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-True-False-False] 37.0210μs 13.2441μs 75.5055 KOps/s 75.7407 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-False-True-True] 56.5210μs 39.0861μs 25.5846 KOps/s 25.6010 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-False-False-True-False] 47.6520μs 25.5915μs 39.0755 KOps/s 39.0060 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-False-False-True] 48.4810μs 22.4411μs 44.5611 KOps/s 43.7530 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-False-False-False-False] 38.1610μs 15.2126μs 65.7350 KOps/s 66.3789 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[False-True-True-True-True] 63.4420μs 37.1758μs 26.8992 KOps/s 26.6791 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-True-True-False] 42.5820μs 23.7455μs 42.1132 KOps/s 42.1546 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-True-True-False-True] 43.4820μs 25.0140μs 39.9776 KOps/s 39.7507 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-True-True-False-False] 39.7910μs 15.2142μs 65.7281 KOps/s 66.8468 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-True-False-True-True] 62.5420μs 38.6169μs 25.8954 KOps/s 25.5250 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-False-True-False] 50.5110μs 25.7816μs 38.7873 KOps/s 39.2298 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-True-False-False-True] 51.9810μs 26.2709μs 38.0649 KOps/s 36.9044 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[False-True-False-False-False] 33.8910μs 17.0099μs 58.7894 KOps/s 58.4160 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-True-True-True] 63.0020μs 40.7673μs 24.5295 KOps/s 24.3568 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[False-False-True-True-False] 52.8820μs 27.8669μs 35.8849 KOps/s 35.7751 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-True-False-True] 43.9320μs 26.6856μs 37.4734 KOps/s 37.0266 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-True-False-False] 40.3110μs 16.9823μs 58.8847 KOps/s 58.5989 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-False-False-True-True] 61.6520μs 42.3339μs 23.6217 KOps/s 23.6765 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-False-True-False] 49.2320μs 29.4080μs 34.0043 KOps/s 34.1459 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-False-False-False-True] 53.7830μs 28.3157μs 35.3161 KOps/s 34.9915 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-False-False-False-False] 34.7410μs 18.7169μs 53.4276 KOps/s 53.4431 KOps/s $\color{#d91a1a}-0.03\%$
test_values[generalized_advantage_estimate-True-True] 25.5497ms 25.1483ms 39.7641 Ops/s 39.3038 Ops/s $\color{#35bf28}+1.17\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.6053ms 3.4957ms 286.0645 Ops/s 290.2505 Ops/s $\color{#d91a1a}-1.44\%$
test_values[td0_return_estimate-False-False] 97.7020μs 62.1390μs 16.0930 KOps/s 15.6588 KOps/s $\color{#35bf28}+2.77\%$
test_values[td1_return_estimate-False-False] 57.0200ms 55.3178ms 18.0774 Ops/s 18.2864 Ops/s $\color{#d91a1a}-1.14\%$
test_values[vec_td1_return_estimate-False-False] 2.1176ms 1.7665ms 566.0801 Ops/s 564.4363 Ops/s $\color{#35bf28}+0.29\%$
test_values[td_lambda_return_estimate-True-False] 91.0295ms 88.1691ms 11.3418 Ops/s 11.5802 Ops/s $\color{#d91a1a}-2.06\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.9373ms 1.7987ms 555.9720 Ops/s 553.5904 Ops/s $\color{#35bf28}+0.43\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.2270ms 24.1129ms 41.4716 Ops/s 41.2384 Ops/s $\color{#35bf28}+0.57\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8766ms 0.7038ms 1.4208 KOps/s 1.4053 KOps/s $\color{#35bf28}+1.10\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7129ms 0.6555ms 1.5257 KOps/s 1.4808 KOps/s $\color{#35bf28}+3.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4882ms 1.4553ms 687.1365 Ops/s 682.3209 Ops/s $\color{#35bf28}+0.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9430ms 0.6728ms 1.4863 KOps/s 1.4720 KOps/s $\color{#35bf28}+0.97\%$
test_dqn_speed 7.7018ms 7.3653ms 135.7726 Ops/s 136.3773 Ops/s $\color{#d91a1a}-0.44\%$
test_ddpg_speed 15.1461ms 14.1865ms 70.4895 Ops/s 72.8990 Ops/s $\color{#d91a1a}-3.31\%$
test_sac_speed 31.5429ms 28.7145ms 34.8257 Ops/s 36.4093 Ops/s $\color{#d91a1a}-4.35\%$
test_redq_speed 12.6302ms 11.5780ms 86.3705 Ops/s 79.2681 Ops/s $\textbf{\color{#35bf28}+8.96\%}$
test_redq_deprec_speed 23.6824ms 22.6733ms 44.1048 Ops/s 45.4907 Ops/s $\color{#d91a1a}-3.05\%$
test_td3_speed 21.5760ms 18.9536ms 52.7605 Ops/s 54.2189 Ops/s $\color{#d91a1a}-2.69\%$
test_cql_speed 75.6987ms 74.4878ms 13.4250 Ops/s 13.6853 Ops/s $\color{#d91a1a}-1.90\%$
test_a2c_speed 26.1214ms 24.7809ms 40.3537 Ops/s 37.8697 Ops/s $\textbf{\color{#35bf28}+6.56\%}$
test_ppo_speed 26.3936ms 24.9728ms 40.0436 Ops/s 41.1688 Ops/s $\color{#d91a1a}-2.73\%$
test_reinforce_speed 0.1231s 26.5032ms 37.7313 Ops/s 43.1343 Ops/s $\textbf{\color{#d91a1a}-12.53\%}$
test_iql_speed 50.4772ms 49.6907ms 20.1245 Ops/s 20.4869 Ops/s $\color{#d91a1a}-1.77\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8253ms 3.6733ms 272.2331 Ops/s 271.2787 Ops/s $\color{#35bf28}+0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7391ms 0.5578ms 1.7926 KOps/s 1.8021 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6801ms 0.5285ms 1.8921 KOps/s 1.8960 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9011ms 3.7031ms 270.0474 Ops/s 268.2999 Ops/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9540ms 0.5551ms 1.8016 KOps/s 1.8130 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6479ms 0.5217ms 1.9169 KOps/s 1.9165 KOps/s $\color{#35bf28}+0.02\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5582ms 3.3857ms 295.3626 Ops/s 298.9629 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8466ms 0.6805ms 1.4694 KOps/s 1.4800 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7643ms 0.6551ms 1.5265 KOps/s 1.5491 KOps/s $\color{#d91a1a}-1.46\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7786ms 3.6807ms 271.6840 Ops/s 270.9211 Ops/s $\color{#35bf28}+0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7013ms 0.5572ms 1.7946 KOps/s 1.8067 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6587ms 0.5269ms 1.8978 KOps/s 1.9015 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9436ms 3.7288ms 268.1793 Ops/s 268.9746 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6761ms 0.5489ms 1.8220 KOps/s 1.8097 KOps/s $\color{#35bf28}+0.68\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6523ms 0.5265ms 1.8994 KOps/s 1.9099 KOps/s $\color{#d91a1a}-0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5621ms 3.3617ms 297.4649 Ops/s 298.6827 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8435ms 0.6909ms 1.4473 KOps/s 1.4763 KOps/s $\color{#d91a1a}-1.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7979ms 0.6578ms 1.5201 KOps/s 1.5349 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1329s 10.3823ms 96.3182 Ops/s 96.3950 Ops/s $\color{#d91a1a}-0.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.5087ms 17.0002ms 58.8229 Ops/s 59.5633 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.7710ms 3.1222ms 320.2920 Ops/s 322.2339 Ops/s $\color{#d91a1a}-0.60\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1202s 10.4618ms 95.5861 Ops/s 98.9282 Ops/s $\color{#d91a1a}-3.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.6522ms 16.5521ms 60.4152 Ops/s 59.8754 Ops/s $\color{#35bf28}+0.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.3672ms 3.0684ms 325.9075 Ops/s 323.1775 Ops/s $\color{#35bf28}+0.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1212s 12.6228ms 79.2218 Ops/s 79.1180 Ops/s $\color{#35bf28}+0.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.9047ms 16.8634ms 59.3000 Ops/s 59.3566 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.8263ms 3.3197ms 301.2310 Ops/s 303.1244 Ops/s $\color{#d91a1a}-0.62\%$

@vmoens vmoens added the bug Something isn't working label Feb 3, 2024
@vmoens vmoens merged commit 5f82601 into main Feb 4, 2024
@vmoens vmoens deleted the vmoens-patch-2 branch February 4, 2024 20:11
vmoens added a commit that referenced this pull request Feb 6, 2024
@vmoens
[BugFix] better device consistency in EGreedy (#1867)
49449a6
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot