Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix Atari DQN ensembling #1981

Merged
merged 3 commits into from
Feb 29, 2024
Merged

[BugFix] Fix Atari DQN ensembling #1981

merged 3 commits into from
Feb 29, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 28, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 28, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1981

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit b003af4 with merge base 847b1b5 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 28, 2024
@vmoens vmoens added the Data Data-related PR, will launch data-related jobs label Feb 28, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 28, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.1655ms 61.2077ms 16.3378 Ops/s 16.3657 Ops/s $\color{#d91a1a}-0.17\%$
test_sync 42.9954ms 35.6920ms 28.0175 Ops/s 28.3030 Ops/s $\color{#d91a1a}-1.01\%$
test_async 55.2537ms 31.7937ms 31.4527 Ops/s 32.9162 Ops/s $\color{#d91a1a}-4.45\%$
test_simple 0.5016s 0.4382s 2.2820 Ops/s 2.3464 Ops/s $\color{#d91a1a}-2.74\%$
test_transformed 0.6520s 0.5943s 1.6826 Ops/s 1.6981 Ops/s $\color{#d91a1a}-0.91\%$
test_serial 1.4826s 1.4309s 0.6989 Ops/s 0.6872 Ops/s $\color{#35bf28}+1.70\%$
test_parallel 1.4564s 1.3935s 0.7176 Ops/s 0.7120 Ops/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-True-True-True] 0.1789ms 21.3110μs 46.9241 KOps/s 46.6464 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-True-True-False] 66.7350μs 12.8706μs 77.6966 KOps/s 75.8405 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[True-True-True-False-True] 52.1470μs 12.5722μs 79.5408 KOps/s 79.8791 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-True-True-False-False] 28.7430μs 7.5639μs 132.2076 KOps/s 129.7668 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[True-True-False-True-True] 55.4640μs 22.8990μs 43.6700 KOps/s 43.9574 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-False-True-False] 36.0070μs 14.4119μs 69.3869 KOps/s 69.1526 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-False-False-True] 41.4470μs 13.8490μs 72.2074 KOps/s 72.6655 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-False-False-False] 30.7870μs 8.9143μs 112.1787 KOps/s 111.5622 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-False-True-True-True] 68.0570μs 24.4394μs 40.9176 KOps/s 41.3444 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[True-False-True-True-False] 52.6080μs 16.1091μs 62.0768 KOps/s 63.1040 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[True-False-True-False-True] 46.5770μs 13.7814μs 72.5617 KOps/s 72.6490 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-True-False-False] 31.4180μs 8.9862μs 111.2814 KOps/s 110.5758 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-False-True-True] 77.5050μs 25.4634μs 39.2720 KOps/s 39.8061 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-False-False-True-False] 58.5600μs 17.1426μs 58.3342 KOps/s 58.6000 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-False-False-True] 55.1340μs 15.0358μs 66.5079 KOps/s 66.9768 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-False-False-False-False] 37.9200μs 10.0944μs 99.0650 KOps/s 97.6799 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-True-True-True-True] 52.5480μs 24.1981μs 41.3256 KOps/s 41.1055 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-True-True-False] 55.0930μs 15.7534μs 63.4786 KOps/s 63.2672 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-True-False-True] 51.8870μs 15.9781μs 62.5857 KOps/s 61.8841 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-True-True-False-False] 56.9660μs 10.0411μs 99.5912 KOps/s 98.2446 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-True-False-True-True] 52.8590μs 25.5569μs 39.1283 KOps/s 38.4284 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-True-False-True-False] 72.0580μs 16.9276μs 59.0751 KOps/s 58.6096 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-True-False-False-True] 47.8690μs 17.1026μs 58.4705 KOps/s 56.9400 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[False-True-False-False-False] 43.6210μs 11.2408μs 88.9613 KOps/s 86.7113 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[False-False-True-True-True] 74.3090μs 26.4289μs 37.8374 KOps/s 37.1502 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[False-False-True-True-False] 52.7390μs 18.2925μs 54.6673 KOps/s 53.9383 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-False-True-False-True] 47.7690μs 17.3365μs 57.6818 KOps/s 57.6846 KOps/s $-0.00\%$
test_step_mdp_speed[False-False-True-False-False] 54.6130μs 11.2521μs 88.8724 KOps/s 88.6455 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-False-False-True-True] 57.1560μs 27.7389μs 36.0504 KOps/s 36.0691 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-False-False-True-False] 70.6010μs 19.3210μs 51.7571 KOps/s 51.9841 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-False-False-False-True] 52.0070μs 18.3312μs 54.5518 KOps/s 54.4568 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-False-False-False] 42.4990μs 12.4081μs 80.5924 KOps/s 80.5334 KOps/s $\color{#35bf28}+0.07\%$
test_values[generalized_advantage_estimate-True-True] 11.4791ms 9.5864ms 104.3146 Ops/s 108.4809 Ops/s $\color{#d91a1a}-3.84\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.9735ms 33.4709ms 29.8767 Ops/s 30.0175 Ops/s $\color{#d91a1a}-0.47\%$
test_values[td0_return_estimate-False-False] 0.2147ms 0.1657ms 6.0358 KOps/s 5.9075 KOps/s $\color{#35bf28}+2.17\%$
test_values[td1_return_estimate-False-False] 27.4388ms 24.0423ms 41.5933 Ops/s 43.7194 Ops/s $\color{#d91a1a}-4.86\%$
test_values[vec_td1_return_estimate-False-False] 35.5351ms 33.5987ms 29.7631 Ops/s 29.9480 Ops/s $\color{#d91a1a}-0.62\%$
test_values[td_lambda_return_estimate-True-False] 36.8950ms 35.0346ms 28.5432 Ops/s 30.0380 Ops/s $\color{#d91a1a}-4.98\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.4971ms 33.4844ms 29.8646 Ops/s 30.0304 Ops/s $\color{#d91a1a}-0.55\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.6078ms 8.3658ms 119.5346 Ops/s 124.1958 Ops/s $\color{#d91a1a}-3.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3761ms 1.9743ms 506.4967 Ops/s 513.2118 Ops/s $\color{#d91a1a}-1.31\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5553ms 0.3611ms 2.7694 KOps/s 2.8683 KOps/s $\color{#d91a1a}-3.45\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 41.0661ms 40.2340ms 24.8546 Ops/s 25.0205 Ops/s $\color{#d91a1a}-0.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1394ms 3.0316ms 329.8572 Ops/s 329.2021 Ops/s $\color{#35bf28}+0.20\%$
test_dqn_speed 6.7921ms 1.3722ms 728.7325 Ops/s 680.3677 Ops/s $\textbf{\color{#35bf28}+7.11\%}$
test_ddpg_speed 3.3848ms 2.7026ms 370.0143 Ops/s 370.4647 Ops/s $\color{#d91a1a}-0.12\%$
test_sac_speed 9.1254ms 8.3673ms 119.5134 Ops/s 117.5916 Ops/s $\color{#35bf28}+1.63\%$
test_redq_speed 83.7755ms 14.4969ms 68.9805 Ops/s 75.0193 Ops/s $\textbf{\color{#d91a1a}-8.05\%}$
test_redq_deprec_speed 15.2023ms 13.6446ms 73.2889 Ops/s 72.7235 Ops/s $\color{#35bf28}+0.78\%$
test_td3_speed 9.1887ms 8.2158ms 121.7171 Ops/s 120.7125 Ops/s $\color{#35bf28}+0.83\%$
test_cql_speed 37.9019ms 36.1835ms 27.6369 Ops/s 27.5282 Ops/s $\color{#35bf28}+0.40\%$
test_a2c_speed 8.9441ms 7.4345ms 134.5072 Ops/s 128.8340 Ops/s $\color{#35bf28}+4.40\%$
test_ppo_speed 8.4402ms 7.9291ms 126.1176 Ops/s 123.2321 Ops/s $\color{#35bf28}+2.34\%$
test_reinforce_speed 7.9546ms 6.6584ms 150.1871 Ops/s 146.4518 Ops/s $\color{#35bf28}+2.55\%$
test_iql_speed 35.0573ms 32.7217ms 30.5608 Ops/s 30.0607 Ops/s $\color{#35bf28}+1.66\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.4678ms 2.4457ms 408.8860 Ops/s 411.7256 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8312ms 0.5100ms 1.9606 KOps/s 1.9650 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7273ms 0.4798ms 2.0841 KOps/s 2.0340 KOps/s $\color{#35bf28}+2.46\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9284ms 2.4980ms 400.3273 Ops/s 413.8241 Ops/s $\color{#d91a1a}-3.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6863ms 0.5145ms 1.9436 KOps/s 1.9773 KOps/s $\color{#d91a1a}-1.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.7497ms 0.4838ms 2.0670 KOps/s 2.0978 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.7562ms 2.5047ms 399.2515 Ops/s 394.3107 Ops/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9460ms 0.6243ms 1.6018 KOps/s 1.5396 KOps/s $\color{#35bf28}+4.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9824ms 0.5997ms 1.6675 KOps/s 1.4911 KOps/s $\textbf{\color{#35bf28}+11.83\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9721ms 2.3400ms 427.3583 Ops/s 405.9939 Ops/s $\textbf{\color{#35bf28}+5.26\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6295ms 0.5049ms 1.9807 KOps/s 1.9060 KOps/s $\color{#35bf28}+3.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.6074ms 0.4908ms 2.0374 KOps/s 1.6432 KOps/s $\textbf{\color{#35bf28}+23.99\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6903ms 2.3100ms 432.9034 Ops/s 409.8117 Ops/s $\textbf{\color{#35bf28}+5.63\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6802ms 0.4992ms 2.0032 KOps/s 1.9815 KOps/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.7243ms 0.4889ms 2.0455 KOps/s 1.9947 KOps/s $\color{#35bf28}+2.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9324ms 2.5305ms 395.1786 Ops/s 381.0595 Ops/s $\color{#35bf28}+3.71\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7828ms 0.6302ms 1.5867 KOps/s 1.3681 KOps/s $\textbf{\color{#35bf28}+15.98\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1406ms 0.6144ms 1.6275 KOps/s 1.5518 KOps/s $\color{#35bf28}+4.88\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1009s 7.2514ms 137.9048 Ops/s 181.8220 Ops/s $\textbf{\color{#d91a1a}-24.15\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.5441ms 12.0858ms 82.7419 Ops/s 77.0065 Ops/s $\textbf{\color{#35bf28}+7.45\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0793ms 1.0117ms 988.4632 Ops/s 968.5317 Ops/s $\color{#35bf28}+2.06\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 91.4115ms 7.0621ms 141.6009 Ops/s 137.5537 Ops/s $\color{#35bf28}+2.94\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.3331ms 12.2218ms 81.8209 Ops/s 83.5627 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.9766ms 1.1193ms 893.4152 Ops/s 886.0892 Ops/s $\color{#35bf28}+0.83\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 87.6113ms 5.5963ms 178.6900 Ops/s 130.6481 Ops/s $\textbf{\color{#35bf28}+36.77\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1011s 14.1850ms 70.4971 Ops/s 81.3031 Ops/s $\textbf{\color{#d91a1a}-13.29\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.6865ms 1.4606ms 684.6687 Ops/s 695.9510 Ops/s $\color{#d91a1a}-1.62\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 29, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1171s 0.1143s 8.7460 Ops/s 9.0170 Ops/s $\color{#d91a1a}-3.01\%$
test_sync 95.6647ms 95.4093ms 10.4812 Ops/s 10.4619 Ops/s $\color{#35bf28}+0.18\%$
test_async 0.1805s 91.4102ms 10.9397 Ops/s 10.8806 Ops/s $\color{#35bf28}+0.54\%$
test_single_pixels 0.1360s 0.1354s 7.3878 Ops/s 8.0117 Ops/s $\textbf{\color{#d91a1a}-7.79\%}$
test_sync_pixels 81.7326ms 78.8772ms 12.6779 Ops/s 12.4054 Ops/s $\color{#35bf28}+2.20\%$
test_async_pixels 0.1475s 72.9650ms 13.7052 Ops/s 14.6382 Ops/s $\textbf{\color{#d91a1a}-6.37\%}$
test_simple 0.8783s 0.8229s 1.2152 Ops/s 1.2130 Ops/s $\color{#35bf28}+0.18\%$
test_transformed 1.1017s 1.0465s 0.9555 Ops/s 0.9723 Ops/s $\color{#d91a1a}-1.72\%$
test_serial 2.4686s 2.4134s 0.4143 Ops/s 0.4178 Ops/s $\color{#d91a1a}-0.82\%$
test_parallel 2.1550s 2.0893s 0.4786 Ops/s 0.4920 Ops/s $\color{#d91a1a}-2.73\%$
test_step_mdp_speed[True-True-True-True-True] 0.2249ms 33.0246μs 30.2805 KOps/s 30.5134 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-True-True-False] 0.2032ms 19.9584μs 50.1043 KOps/s 50.9637 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-True-True-False-True] 37.1700μs 18.6023μs 53.7569 KOps/s 53.8421 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-True-True-False-False] 0.2058ms 11.3293μs 88.2669 KOps/s 92.0923 KOps/s $\color{#d91a1a}-4.15\%$
test_step_mdp_speed[True-True-False-True-True] 0.2213ms 34.8309μs 28.7102 KOps/s 29.1343 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[True-True-False-True-False] 40.5200μs 21.7878μs 45.8972 KOps/s 46.8166 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[True-True-False-False-True] 0.2085ms 20.6107μs 48.5185 KOps/s 49.2943 KOps/s $\color{#d91a1a}-1.57\%$
test_step_mdp_speed[True-True-False-False-False] 0.2004ms 13.2664μs 75.3785 KOps/s 78.4432 KOps/s $\color{#d91a1a}-3.91\%$
test_step_mdp_speed[True-False-True-True-True] 77.7210μs 36.8077μs 27.1682 KOps/s 27.5493 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[True-False-True-True-False] 0.2147ms 24.0243μs 41.6245 KOps/s 42.6947 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[True-False-True-False-True] 0.2044ms 20.5429μs 48.6787 KOps/s 48.8964 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-True-False-False] 35.7710μs 13.2492μs 75.4764 KOps/s 78.5618 KOps/s $\color{#d91a1a}-3.93\%$
test_step_mdp_speed[True-False-False-True-True] 0.2252ms 38.2680μs 26.1315 KOps/s 26.1030 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-False-True-False] 0.2086ms 25.6774μs 38.9448 KOps/s 40.2601 KOps/s $\color{#d91a1a}-3.27\%$
test_step_mdp_speed[True-False-False-False-True] 0.2092ms 22.3446μs 44.7536 KOps/s 45.5333 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-False-False-False] 40.6800μs 15.2079μs 65.7553 KOps/s 66.8488 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[False-True-True-True-True] 0.2212ms 37.3874μs 26.7470 KOps/s 27.5882 KOps/s $\color{#d91a1a}-3.05\%$
test_step_mdp_speed[False-True-True-True-False] 0.2082ms 23.8528μs 41.9237 KOps/s 43.6753 KOps/s $\color{#d91a1a}-4.01\%$
test_step_mdp_speed[False-True-True-False-True] 0.2084ms 24.7468μs 40.4093 KOps/s 42.1492 KOps/s $\color{#d91a1a}-4.13\%$
test_step_mdp_speed[False-True-True-False-False] 37.1300μs 15.3577μs 65.1138 KOps/s 68.2178 KOps/s $\color{#d91a1a}-4.55\%$
test_step_mdp_speed[False-True-False-True-True] 0.2339ms 39.4226μs 25.3661 KOps/s 25.9779 KOps/s $\color{#d91a1a}-2.36\%$
test_step_mdp_speed[False-True-False-True-False] 57.2610μs 25.8479μs 38.6879 KOps/s 39.4856 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-True-False-False-True] 0.2152ms 26.5117μs 37.7192 KOps/s 38.2989 KOps/s $\color{#d91a1a}-1.51\%$
test_step_mdp_speed[False-True-False-False-False] 0.2048ms 17.1863μs 58.1857 KOps/s 61.3649 KOps/s $\textbf{\color{#d91a1a}-5.18\%}$
test_step_mdp_speed[False-False-True-True-True] 0.2263ms 40.6438μs 24.6040 KOps/s 24.8271 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-True-True-False] 52.0120μs 27.9870μs 35.7308 KOps/s 37.5721 KOps/s $\color{#d91a1a}-4.90\%$
test_step_mdp_speed[False-False-True-False-True] 0.2115ms 26.3954μs 37.8853 KOps/s 38.2413 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-False-True-False-False] 0.2042ms 16.9528μs 58.9872 KOps/s 61.6781 KOps/s $\color{#d91a1a}-4.36\%$
test_step_mdp_speed[False-False-False-True-True] 0.2282ms 42.3585μs 23.6080 KOps/s 24.2113 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-False-False-True-False] 52.6810μs 29.6398μs 33.7385 KOps/s 34.8236 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[False-False-False-False-True] 49.4210μs 27.9526μs 35.7749 KOps/s 35.5119 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-False-False-False-False] 0.2011ms 18.7031μs 53.4671 KOps/s 53.1154 KOps/s $\color{#35bf28}+0.66\%$
test_values[generalized_advantage_estimate-True-True] 24.3442ms 23.6683ms 42.2506 Ops/s 40.2281 Ops/s $\textbf{\color{#35bf28}+5.03\%}$
test_values[vec_generalized_advantage_estimate-True-True] 81.9546ms 3.1889ms 313.5926 Ops/s 300.0853 Ops/s $\color{#35bf28}+4.50\%$
test_values[td0_return_estimate-False-False] 96.2110μs 58.9852μs 16.9534 KOps/s 15.8755 KOps/s $\textbf{\color{#35bf28}+6.79\%}$
test_values[td1_return_estimate-False-False] 51.7848ms 51.3671ms 19.4677 Ops/s 18.3074 Ops/s $\textbf{\color{#35bf28}+6.34\%}$
test_values[vec_td1_return_estimate-False-False] 2.0814ms 1.7427ms 573.8271 Ops/s 574.5291 Ops/s $\color{#d91a1a}-0.12\%$
test_values[td_lambda_return_estimate-True-False] 88.5428ms 82.8673ms 12.0675 Ops/s 11.4627 Ops/s $\textbf{\color{#35bf28}+5.28\%}$
test_values[vec_td_lambda_return_estimate-True-False] 3.8660ms 1.7814ms 561.3553 Ops/s 560.3333 Ops/s $\color{#35bf28}+0.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.7991ms 22.5969ms 44.2539 Ops/s 43.8396 Ops/s $\color{#35bf28}+0.94\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8667ms 0.6816ms 1.4671 KOps/s 1.4729 KOps/s $\color{#d91a1a}-0.40\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7051ms 0.6356ms 1.5734 KOps/s 1.5908 KOps/s $\color{#d91a1a}-1.09\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6416ms 1.4385ms 695.1735 Ops/s 699.3950 Ops/s $\color{#d91a1a}-0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9229ms 0.6537ms 1.5297 KOps/s 1.5385 KOps/s $\color{#d91a1a}-0.57\%$
test_dqn_speed 7.7827ms 1.4222ms 703.1584 Ops/s 713.4887 Ops/s $\color{#d91a1a}-1.45\%$
test_ddpg_speed 2.9484ms 2.6779ms 373.4244 Ops/s 378.3712 Ops/s $\color{#d91a1a}-1.31\%$
test_sac_speed 8.4406ms 7.8294ms 127.7243 Ops/s 129.4444 Ops/s $\color{#d91a1a}-1.33\%$
test_redq_speed 11.0200ms 10.0703ms 99.3015 Ops/s 94.4489 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_redq_deprec_speed 11.6913ms 11.1181ms 89.9433 Ops/s 93.2175 Ops/s $\color{#d91a1a}-3.51\%$
test_td3_speed 15.3353ms 7.8308ms 127.7009 Ops/s 131.2017 Ops/s $\color{#d91a1a}-2.67\%$
test_cql_speed 25.8223ms 24.8044ms 40.3154 Ops/s 41.0418 Ops/s $\color{#d91a1a}-1.77\%$
test_a2c_speed 5.7438ms 5.4219ms 184.4369 Ops/s 187.2308 Ops/s $\color{#d91a1a}-1.49\%$
test_ppo_speed 6.0846ms 5.7195ms 174.8408 Ops/s 177.5466 Ops/s $\color{#d91a1a}-1.52\%$
test_reinforce_speed 5.3129ms 4.4466ms 224.8926 Ops/s 229.3323 Ops/s $\color{#d91a1a}-1.94\%$
test_iql_speed 19.0228ms 18.4311ms 54.2561 Ops/s 52.7716 Ops/s $\color{#35bf28}+2.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0472ms 2.8425ms 351.8051 Ops/s 354.0801 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 88.5472ms 0.5984ms 1.6710 KOps/s 1.8981 KOps/s $\textbf{\color{#d91a1a}-11.96\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7193ms 0.5121ms 1.9528 KOps/s 1.9800 KOps/s $\color{#d91a1a}-1.38\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1735ms 2.8837ms 346.7803 Ops/s 352.9901 Ops/s $\color{#d91a1a}-1.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7472ms 0.5349ms 1.8695 KOps/s 1.9112 KOps/s $\color{#d91a1a}-2.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.2484ms 0.5047ms 1.9812 KOps/s 1.9976 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0939ms 2.9903ms 334.4178 Ops/s 341.7978 Ops/s $\color{#d91a1a}-2.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8354ms 0.6557ms 1.5250 KOps/s 1.5558 KOps/s $\color{#d91a1a}-1.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8676ms 0.6335ms 1.5785 KOps/s 1.6062 KOps/s $\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0633ms 2.8700ms 348.4263 Ops/s 357.2468 Ops/s $\color{#d91a1a}-2.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7366ms 0.5305ms 1.8851 KOps/s 1.8977 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7237ms 0.5103ms 1.9595 KOps/s 1.9787 KOps/s $\color{#d91a1a}-0.97\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1397ms 2.8692ms 348.5336 Ops/s 350.8977 Ops/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6562ms 0.5249ms 1.9051 KOps/s 1.9163 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7301ms 0.5058ms 1.9771 KOps/s 1.9963 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1757ms 2.9818ms 335.3684 Ops/s 341.3851 Ops/s $\color{#d91a1a}-1.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7297ms 0.6549ms 1.5270 KOps/s 1.5405 KOps/s $\color{#d91a1a}-0.88\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8187ms 0.6321ms 1.5821 KOps/s 1.5985 KOps/s $\color{#d91a1a}-1.03\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1201s 8.8714ms 112.7214 Ops/s 114.4498 Ops/s $\color{#d91a1a}-1.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.6679ms 14.4556ms 69.1772 Ops/s 69.2628 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.1932ms 1.0450ms 956.9018 Ops/s 885.0052 Ops/s $\textbf{\color{#35bf28}+8.12\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 98.8876ms 8.4220ms 118.7365 Ops/s 117.7331 Ops/s $\color{#35bf28}+0.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.6578ms 14.4386ms 69.2590 Ops/s 70.6195 Ops/s $\color{#d91a1a}-1.93\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1201ms 1.0297ms 971.1713 Ops/s 826.3756 Ops/s $\textbf{\color{#35bf28}+17.52\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 99.5647ms 6.9212ms 144.4839 Ops/s 145.2333 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.1833ms 14.7029ms 68.0136 Ops/s 69.4398 Ops/s $\color{#d91a1a}-2.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.6355ms 1.4604ms 684.7673 Ops/s 683.7058 Ops/s $\color{#35bf28}+0.16\%$

@vmoens vmoens marked this pull request as ready for review February 29, 2024 13:16
@vmoens vmoens merged commit 1c1d558 into main Feb 29, 2024
64 of 68 checks passed
@vmoens vmoens deleted the fix-atari-dqn-ens branch April 3, 2024 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Data Data-related PR, will launch data-related jobs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot