Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Versioning] Versions for 0.6 #2509

Merged
merged 1 commit into from
Oct 22, 2024
Merged

[Versioning] Versions for 0.6 #2509

merged 1 commit into from
Oct 22, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Oct 21, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 21, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2509

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 4 Unrelated Failures

As of commit 0fe3b56 with merge base 7ff03a2 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 21, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 21, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 143. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7313s 0.7307s 1.3686 Ops/s 1.3292 Ops/s $\color{#35bf28}+2.97\%$
test_transformed 1.0622s 0.9862s 1.0140 Ops/s 1.0033 Ops/s $\color{#35bf28}+1.07\%$
test_serial 2.2089s 2.1366s 0.4680 Ops/s 0.4561 Ops/s $\color{#35bf28}+2.63\%$
test_parallel 2.0715s 1.9956s 0.5011 Ops/s 0.4906 Ops/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-True-True-True] 0.1497ms 39.5233μs 25.3015 KOps/s 24.8768 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-True-True-True-False] 56.1020μs 23.2204μs 43.0656 KOps/s 42.8923 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-True-False-True] 61.8330μs 21.4608μs 46.5966 KOps/s 45.8515 KOps/s $\color{#35bf28}+1.62\%$
test_step_mdp_speed[True-True-True-False-False] 53.2230μs 12.6887μs 78.8100 KOps/s 80.4959 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[True-True-False-True-True] 89.7540μs 42.5893μs 23.4801 KOps/s 22.9592 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-True-False-True-False] 59.1030μs 25.9210μs 38.5787 KOps/s 38.5707 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-True-False-False-True] 63.2530μs 24.6140μs 40.6274 KOps/s 40.3368 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-False] 53.3320μs 15.5077μs 64.4840 KOps/s 65.2941 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[True-False-True-True-True] 0.1033ms 44.5498μs 22.4468 KOps/s 22.0180 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-False-True-True-False] 61.1630μs 28.7070μs 34.8348 KOps/s 34.6986 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-True-False-True] 56.0820μs 24.2653μs 41.2111 KOps/s 40.2140 KOps/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[True-False-True-False-False] 54.4520μs 15.5999μs 64.1031 KOps/s 65.0293 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-False-False-True-True] 94.0850μs 48.5989μs 20.5766 KOps/s 20.5987 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-False-False-True-False] 67.1130μs 31.0265μs 32.2305 KOps/s 32.0425 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-False-False-True] 64.9230μs 26.7635μs 37.3643 KOps/s 36.9163 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-False-False-False] 48.0920μs 18.0789μs 55.3130 KOps/s 55.5900 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-True-True-True-True] 80.9930μs 45.0704μs 22.1875 KOps/s 21.8034 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-True-True-True-False] 61.7830μs 28.9377μs 34.5570 KOps/s 35.1027 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[False-True-True-False-True] 57.5520μs 29.4724μs 33.9301 KOps/s 33.9856 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-True-True-False-False] 52.7120μs 17.9489μs 55.7136 KOps/s 55.4558 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-False-True-True] 83.4540μs 48.5402μs 20.6015 KOps/s 20.5766 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-True-False-True-False] 63.6130μs 31.8074μs 31.4393 KOps/s 32.0865 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-True-False-False-True] 3.0872ms 32.0304μs 31.2204 KOps/s 31.3489 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-True-False-False-False] 55.2730μs 20.7174μs 48.2685 KOps/s 51.3362 KOps/s $\textbf{\color{#d91a1a}-5.98\%}$
test_step_mdp_speed[False-False-True-True-True] 92.7750μs 51.2942μs 19.4954 KOps/s 19.9774 KOps/s $\color{#d91a1a}-2.41\%$
test_step_mdp_speed[False-False-True-True-False] 67.8530μs 34.1912μs 29.2473 KOps/s 29.5985 KOps/s $\color{#d91a1a}-1.19\%$
test_step_mdp_speed[False-False-True-False-True] 64.5430μs 32.0507μs 31.2006 KOps/s 31.6392 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[False-False-True-False-False] 51.3520μs 20.7101μs 48.2856 KOps/s 49.5174 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-False-False-True-True] 83.7040μs 53.1466μs 18.8159 KOps/s 18.9270 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-True-False] 69.8730μs 36.9099μs 27.0930 KOps/s 27.2214 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-False-False-False-True] 64.1830μs 33.1148μs 30.1980 KOps/s 29.1830 KOps/s $\color{#35bf28}+3.48\%$
test_step_mdp_speed[False-False-False-False-False] 58.2320μs 22.9672μs 43.5404 KOps/s 43.0894 KOps/s $\color{#35bf28}+1.05\%$
test_values[generalized_advantage_estimate-True-True] 26.0496ms 24.8603ms 40.2248 Ops/s 41.2742 Ops/s $\color{#d91a1a}-2.54\%$
test_values[vec_generalized_advantage_estimate-True-True] 98.4611ms 2.8559ms 350.1474 Ops/s 352.5499 Ops/s $\color{#d91a1a}-0.68\%$
test_values[td0_return_estimate-False-False] 86.4940μs 65.1553μs 15.3479 KOps/s 15.4330 KOps/s $\color{#d91a1a}-0.55\%$
test_values[td1_return_estimate-False-False] 55.3549ms 54.5566ms 18.3296 Ops/s 18.4756 Ops/s $\color{#d91a1a}-0.79\%$
test_values[vec_td1_return_estimate-False-False] 1.3693ms 1.0724ms 932.5039 Ops/s 930.4293 Ops/s $\color{#35bf28}+0.22\%$
test_values[td_lambda_return_estimate-True-False] 90.2899ms 86.4389ms 11.5689 Ops/s 11.5170 Ops/s $\color{#35bf28}+0.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3654ms 1.0657ms 938.3739 Ops/s 936.0993 Ops/s $\color{#35bf28}+0.24\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.5785ms 24.3430ms 41.0796 Ops/s 40.9628 Ops/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0309ms 0.7455ms 1.3414 KOps/s 1.3355 KOps/s $\color{#35bf28}+0.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7525ms 0.6578ms 1.5203 KOps/s 1.5187 KOps/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5083ms 1.4688ms 680.8247 Ops/s 680.2461 Ops/s $\color{#35bf28}+0.09\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7946ms 0.6854ms 1.4591 KOps/s 1.4722 KOps/s $\color{#d91a1a}-0.89\%$
test_dqn_speed[False-None] 6.5003ms 1.3468ms 742.5100 Ops/s 723.7291 Ops/s $\color{#35bf28}+2.60\%$
test_dqn_speed[False-backward] 1.9116ms 1.8301ms 546.4168 Ops/s 536.9202 Ops/s $\color{#35bf28}+1.77\%$
test_dqn_speed[True-None] 0.7206ms 0.5865ms 1.7051 KOps/s 1.6711 KOps/s $\color{#35bf28}+2.03\%$
test_dqn_speed[True-backward] 1.1136ms 1.0572ms 945.8924 Ops/s 846.2824 Ops/s $\textbf{\color{#35bf28}+11.77\%}$
test_dqn_speed[reduce-overhead-None] 0.7173ms 0.6027ms 1.6591 KOps/s 1.7273 KOps/s $\color{#d91a1a}-3.95\%$
test_dqn_speed[reduce-overhead-backward] 1.1681ms 1.0376ms 963.8052 Ops/s 970.9620 Ops/s $\color{#d91a1a}-0.74\%$
test_ddpg_speed[False-None] 3.2188ms 2.7630ms 361.9275 Ops/s 360.7569 Ops/s $\color{#35bf28}+0.32\%$
test_ddpg_speed[False-backward] 4.4456ms 4.0153ms 249.0484 Ops/s 247.6010 Ops/s $\color{#35bf28}+0.58\%$
test_ddpg_speed[True-None] 1.4847ms 1.3061ms 765.6195 Ops/s 786.8414 Ops/s $\color{#d91a1a}-2.70\%$
test_ddpg_speed[True-backward] 2.3662ms 2.2823ms 438.1510 Ops/s 369.2192 Ops/s $\textbf{\color{#35bf28}+18.67\%}$
test_ddpg_speed[reduce-overhead-None] 1.6728ms 1.2947ms 772.3784 Ops/s 781.3464 Ops/s $\color{#d91a1a}-1.15\%$
test_ddpg_speed[reduce-overhead-backward] 2.3791ms 2.2645ms 441.5982 Ops/s 373.4914 Ops/s $\textbf{\color{#35bf28}+18.24\%}$
test_sac_speed[False-None] 8.2791ms 7.7321ms 129.3302 Ops/s 127.7051 Ops/s $\color{#35bf28}+1.27\%$
test_sac_speed[False-backward] 12.0901ms 11.0022ms 90.8909 Ops/s 90.9476 Ops/s $\color{#d91a1a}-0.06\%$
test_sac_speed[True-None] 2.2282ms 2.0700ms 483.0859 Ops/s 477.5029 Ops/s $\color{#35bf28}+1.17\%$
test_sac_speed[True-backward] 4.1467ms 4.0059ms 249.6324 Ops/s 239.1956 Ops/s $\color{#35bf28}+4.36\%$
test_sac_speed[reduce-overhead-None] 2.1715ms 2.0598ms 485.4839 Ops/s 479.2367 Ops/s $\color{#35bf28}+1.30\%$
test_sac_speed[reduce-overhead-backward] 4.2283ms 4.0562ms 246.5362 Ops/s 249.6839 Ops/s $\color{#d91a1a}-1.26\%$
test_redq_speed[False-None] 15.4700ms 10.6673ms 93.7447 Ops/s 91.4898 Ops/s $\color{#35bf28}+2.46\%$
test_redq_speed[False-backward] 18.6099ms 17.4458ms 57.3202 Ops/s 57.1470 Ops/s $\color{#35bf28}+0.30\%$
test_redq_speed[True-None] 3.8427ms 3.4510ms 289.7724 Ops/s 270.1626 Ops/s $\textbf{\color{#35bf28}+7.26\%}$
test_redq_speed[True-backward] 8.5721ms 8.3600ms 119.6178 Ops/s 115.9923 Ops/s $\color{#35bf28}+3.13\%$
test_redq_speed[reduce-overhead-None] 3.7355ms 3.3582ms 297.7767 Ops/s 276.7437 Ops/s $\textbf{\color{#35bf28}+7.60\%}$
test_redq_speed[reduce-overhead-backward] 8.5428ms 8.2567ms 121.1141 Ops/s 117.0010 Ops/s $\color{#35bf28}+3.52\%$
test_redq_deprec_speed[False-None] 11.2426ms 10.7411ms 93.1002 Ops/s 92.4460 Ops/s $\color{#35bf28}+0.71\%$
test_redq_deprec_speed[False-backward] 16.0678ms 15.6694ms 63.8185 Ops/s 64.6458 Ops/s $\color{#d91a1a}-1.28\%$
test_redq_deprec_speed[True-None] 3.4919ms 3.3446ms 298.9853 Ops/s 304.0588 Ops/s $\color{#d91a1a}-1.67\%$
test_redq_deprec_speed[True-backward] 7.9026ms 7.4020ms 135.0992 Ops/s 137.7552 Ops/s $\color{#d91a1a}-1.93\%$
test_redq_deprec_speed[reduce-overhead-None] 3.4347ms 3.3000ms 303.0264 Ops/s 307.1086 Ops/s $\color{#d91a1a}-1.33\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.5426ms 7.3885ms 135.3464 Ops/s 141.9481 Ops/s $\color{#d91a1a}-4.65\%$
test_td3_speed[False-None] 7.7241ms 7.5418ms 132.5938 Ops/s 127.8506 Ops/s $\color{#35bf28}+3.71\%$
test_td3_speed[False-backward] 10.9178ms 10.3967ms 96.1846 Ops/s 92.2489 Ops/s $\color{#35bf28}+4.27\%$
test_td3_speed[True-None] 1.9849ms 1.9450ms 514.1469 Ops/s 512.7374 Ops/s $\color{#35bf28}+0.27\%$
test_td3_speed[True-backward] 3.8975ms 3.7860ms 264.1295 Ops/s 223.7916 Ops/s $\textbf{\color{#35bf28}+18.02\%}$
test_td3_speed[reduce-overhead-None] 2.0167ms 1.9578ms 510.7649 Ops/s 502.3660 Ops/s $\color{#35bf28}+1.67\%$
test_td3_speed[reduce-overhead-backward] 3.8625ms 3.7582ms 266.0843 Ops/s 262.0493 Ops/s $\color{#35bf28}+1.54\%$
test_cql_speed[False-None] 27.9992ms 25.2029ms 39.6780 Ops/s 39.4586 Ops/s $\color{#35bf28}+0.56\%$
test_cql_speed[False-backward] 38.1552ms 34.6846ms 28.8312 Ops/s 28.1902 Ops/s $\color{#35bf28}+2.27\%$
test_cql_speed[True-None] 11.1423ms 10.7683ms 92.8651 Ops/s 92.3704 Ops/s $\color{#35bf28}+0.54\%$
test_cql_speed[True-backward] 16.8270ms 16.5315ms 60.4906 Ops/s 58.3853 Ops/s $\color{#35bf28}+3.61\%$
test_cql_speed[reduce-overhead-None] 11.2240ms 10.8384ms 92.2647 Ops/s 91.0340 Ops/s $\color{#35bf28}+1.35\%$
test_cql_speed[reduce-overhead-backward] 16.8948ms 16.5764ms 60.3267 Ops/s 59.3543 Ops/s $\color{#35bf28}+1.64\%$
test_a2c_speed[False-None] 5.8281ms 5.3210ms 187.9341 Ops/s 183.5122 Ops/s $\color{#35bf28}+2.41\%$
test_a2c_speed[False-backward] 12.6584ms 12.0645ms 82.8875 Ops/s 83.8379 Ops/s $\color{#d91a1a}-1.13\%$
test_a2c_speed[True-None] 3.3220ms 3.0564ms 327.1855 Ops/s 325.5699 Ops/s $\color{#35bf28}+0.50\%$
test_a2c_speed[True-backward] 8.7580ms 8.5973ms 116.3150 Ops/s 110.8601 Ops/s $\color{#35bf28}+4.92\%$
test_a2c_speed[reduce-overhead-None] 3.5461ms 3.1058ms 321.9754 Ops/s 322.8685 Ops/s $\color{#d91a1a}-0.28\%$
test_a2c_speed[reduce-overhead-backward] 8.8475ms 8.6421ms 115.7126 Ops/s 117.4725 Ops/s $\color{#d91a1a}-1.50\%$
test_ppo_speed[False-None] 6.1014ms 5.7253ms 174.6632 Ops/s 176.3316 Ops/s $\color{#d91a1a}-0.95\%$
test_ppo_speed[False-backward] 12.7662ms 12.4435ms 80.3635 Ops/s 81.1184 Ops/s $\color{#d91a1a}-0.93\%$
test_ppo_speed[True-None] 3.8200ms 3.4749ms 287.7749 Ops/s 275.2265 Ops/s $\color{#35bf28}+4.56\%$
test_ppo_speed[True-backward] 9.0679ms 8.5161ms 117.4250 Ops/s 120.0270 Ops/s $\color{#d91a1a}-2.17\%$
test_ppo_speed[reduce-overhead-None] 3.8759ms 3.4691ms 288.2562 Ops/s 286.1785 Ops/s $\color{#35bf28}+0.73\%$
test_ppo_speed[reduce-overhead-backward] 8.7774ms 8.3314ms 120.0282 Ops/s 118.8834 Ops/s $\color{#35bf28}+0.96\%$
test_reinforce_speed[False-None] 4.8593ms 4.4903ms 222.7006 Ops/s 217.8953 Ops/s $\color{#35bf28}+2.21\%$
test_reinforce_speed[False-backward] 7.5525ms 7.3280ms 136.4634 Ops/s 134.7723 Ops/s $\color{#35bf28}+1.25\%$
test_reinforce_speed[True-None] 2.6401ms 2.2340ms 447.6303 Ops/s 432.5472 Ops/s $\color{#35bf28}+3.49\%$
test_reinforce_speed[True-backward] 7.5850ms 7.2155ms 138.5901 Ops/s 139.5555 Ops/s $\color{#d91a1a}-0.69\%$
test_reinforce_speed[reduce-overhead-None] 2.7230ms 2.2370ms 447.0318 Ops/s 440.5030 Ops/s $\color{#35bf28}+1.48\%$
test_reinforce_speed[reduce-overhead-backward] 7.3320ms 7.1602ms 139.6611 Ops/s 140.3401 Ops/s $\color{#d91a1a}-0.48\%$
test_iql_speed[False-None] 19.7091ms 19.2880ms 51.8458 Ops/s 51.2349 Ops/s $\color{#35bf28}+1.19\%$
test_iql_speed[False-backward] 30.4594ms 29.8418ms 33.5100 Ops/s 33.4287 Ops/s $\color{#35bf28}+0.24\%$
test_iql_speed[True-None] 7.1700ms 6.7478ms 148.1963 Ops/s 141.0297 Ops/s $\textbf{\color{#35bf28}+5.08\%}$
test_iql_speed[True-backward] 16.1851ms 15.6644ms 63.8392 Ops/s 63.7641 Ops/s $\color{#35bf28}+0.12\%$
test_iql_speed[reduce-overhead-None] 7.1780ms 6.7556ms 148.0245 Ops/s 145.7791 Ops/s $\color{#35bf28}+1.54\%$
test_iql_speed[reduce-overhead-backward] 15.9435ms 15.6449ms 63.9187 Ops/s 63.6198 Ops/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7629ms 6.3648ms 157.1130 Ops/s 157.1695 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9671ms 0.3344ms 2.9907 KOps/s 4.1713 KOps/s $\textbf{\color{#d91a1a}-28.30\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8522ms 0.3177ms 3.1476 KOps/s 4.5836 KOps/s $\textbf{\color{#d91a1a}-31.33\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.6891ms 6.1600ms 162.3366 Ops/s 165.5313 Ops/s $\color{#d91a1a}-1.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8580ms 0.3168ms 3.1569 KOps/s 4.2224 KOps/s $\textbf{\color{#d91a1a}-25.24\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7448ms 0.3070ms 3.2574 KOps/s 4.6527 KOps/s $\textbf{\color{#d91a1a}-29.99\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6884ms 1.3491ms 741.2344 Ops/s 812.3584 Ops/s $\textbf{\color{#d91a1a}-8.76\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5490ms 1.2882ms 776.2650 Ops/s 844.7005 Ops/s $\textbf{\color{#d91a1a}-8.10\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7756ms 6.3131ms 158.4019 Ops/s 161.1522 Ops/s $\color{#d91a1a}-1.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.4894ms 0.4576ms 2.1854 KOps/s 2.0992 KOps/s $\color{#35bf28}+4.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9761ms 0.4538ms 2.2035 KOps/s 2.1675 KOps/s $\color{#35bf28}+1.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.6251ms 6.1869ms 161.6319 Ops/s 164.6401 Ops/s $\color{#d91a1a}-1.83\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6312ms 0.2601ms 3.8451 KOps/s 4.0953 KOps/s $\textbf{\color{#d91a1a}-6.11\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5878ms 0.3383ms 2.9556 KOps/s 4.4594 KOps/s $\textbf{\color{#d91a1a}-33.72\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 13.2759ms 6.3475ms 157.5425 Ops/s 165.9433 Ops/s $\textbf{\color{#d91a1a}-5.06\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8663ms 0.2380ms 4.2014 KOps/s 4.1692 KOps/s $\color{#35bf28}+0.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5173ms 0.3497ms 2.8595 KOps/s 4.5995 KOps/s $\textbf{\color{#d91a1a}-37.83\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4188ms 6.2850ms 159.1096 Ops/s 161.9005 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.5447ms 0.4550ms 2.1980 KOps/s 649.7164 Ops/s $\textbf{\color{#35bf28}+238.30\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5959ms 0.4249ms 2.3535 KOps/s 2.6982 KOps/s $\textbf{\color{#d91a1a}-12.78\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.8393ms 5.2621ms 190.0373 Ops/s 182.5709 Ops/s $\color{#35bf28}+4.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.7328ms 2.0249ms 493.8431 Ops/s 482.2205 Ops/s $\color{#35bf28}+2.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.2857ms 1.2220ms 818.3082 Ops/s 819.6630 Ops/s $\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4008s 13.2772ms 75.3173 Ops/s 186.4218 Ops/s $\textbf{\color{#d91a1a}-59.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.4185ms 2.0224ms 494.4700 Ops/s 489.4621 Ops/s $\color{#35bf28}+1.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.0338ms 1.2561ms 796.1362 Ops/s 803.3769 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.1041ms 5.5395ms 180.5225 Ops/s 179.7725 Ops/s $\color{#35bf28}+0.42\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.5254ms 2.2528ms 443.8973 Ops/s 419.7672 Ops/s $\textbf{\color{#35bf28}+5.75\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.1596ms 1.3986ms 715.0069 Ops/s 663.5332 Ops/s $\textbf{\color{#35bf28}+7.76\%}$

This was referenced Oct 22, 2024
Open
Closed
@vmoens vmoens added the versioning Versioning change (version number etc) label Oct 22, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 143. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4132s 0.4112s 2.4320 Ops/s 2.3221 Ops/s $\color{#35bf28}+4.73\%$
test_transformed 0.6810s 0.6050s 1.6528 Ops/s 1.6522 Ops/s $\color{#35bf28}+0.03\%$
test_serial 1.4180s 1.3414s 0.7455 Ops/s 0.7476 Ops/s $\color{#d91a1a}-0.29\%$
test_parallel 1.3699s 1.3137s 0.7612 Ops/s 0.7575 Ops/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-True-True-True-True] 0.1738ms 28.8425μs 34.6710 KOps/s 34.5198 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-True-True-True-False] 54.7930μs 17.1606μs 58.2730 KOps/s 57.0514 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-True-False-True] 59.1800μs 15.9649μs 62.6376 KOps/s 61.7391 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-True-True-False-False] 29.8160μs 9.4652μs 105.6506 KOps/s 104.0214 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-False-True-True] 84.1670μs 30.9061μs 32.3560 KOps/s 31.2710 KOps/s $\color{#35bf28}+3.47\%$
test_step_mdp_speed[True-True-False-True-False] 68.4580μs 19.1080μs 52.3341 KOps/s 50.9243 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[True-True-False-False-True] 49.4020μs 18.2254μs 54.8685 KOps/s 54.2078 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-True-False-False-False] 51.6370μs 11.6586μs 85.7739 KOps/s 84.7176 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-True-True-True] 86.3510μs 32.8951μs 30.3996 KOps/s 27.9201 KOps/s $\textbf{\color{#35bf28}+8.88\%}$
test_step_mdp_speed[True-False-True-True-False] 52.0370μs 21.3574μs 46.8222 KOps/s 45.1335 KOps/s $\color{#35bf28}+3.74\%$
test_step_mdp_speed[True-False-True-False-True] 45.7650μs 18.0477μs 55.4087 KOps/s 55.0070 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-True-False-False] 41.1770μs 11.4406μs 87.4078 KOps/s 84.6359 KOps/s $\color{#35bf28}+3.28\%$
test_step_mdp_speed[True-False-False-True-True] 97.2210μs 34.8340μs 28.7076 KOps/s 28.0263 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[True-False-False-True-False] 57.1370μs 23.2767μs 42.9615 KOps/s 41.8209 KOps/s $\color{#35bf28}+2.73\%$
test_step_mdp_speed[True-False-False-False-True] 80.6520μs 19.8275μs 50.4351 KOps/s 48.9857 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[True-False-False-False-False] 43.8110μs 13.6064μs 73.4951 KOps/s 74.7428 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-True-True-True-True] 76.0410μs 32.5532μs 30.7189 KOps/s 29.6593 KOps/s $\color{#35bf28}+3.57\%$
test_step_mdp_speed[False-True-True-True-False] 51.4560μs 21.1581μs 47.2631 KOps/s 45.8185 KOps/s $\color{#35bf28}+3.15\%$
test_step_mdp_speed[False-True-True-False-True] 55.6340μs 21.5397μs 46.4260 KOps/s 46.2961 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-True-True-False-False] 48.1890μs 13.5387μs 73.8623 KOps/s 74.5691 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-False-True-True] 71.4030μs 35.0519μs 28.5291 KOps/s 28.3448 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-False-True-False] 56.0340μs 23.5600μs 42.4449 KOps/s 41.0523 KOps/s $\color{#35bf28}+3.39\%$
test_step_mdp_speed[False-True-False-False-True] 2.5121ms 23.7381μs 42.1264 KOps/s 41.9539 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-True-False-False-False] 49.6520μs 15.2838μs 65.4290 KOps/s 64.3464 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-False-True-True-True] 0.1004ms 36.9668μs 27.0513 KOps/s 26.8137 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-True-False] 66.2130μs 25.5839μs 39.0871 KOps/s 38.4305 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-False-True-False-True] 73.6350μs 22.9972μs 43.4836 KOps/s 41.8635 KOps/s $\color{#35bf28}+3.87\%$
test_step_mdp_speed[False-False-True-False-False] 51.6760μs 15.1391μs 66.0543 KOps/s 64.3565 KOps/s $\color{#35bf28}+2.64\%$
test_step_mdp_speed[False-False-False-True-True] 88.4450μs 38.2940μs 26.1138 KOps/s 25.0918 KOps/s $\color{#35bf28}+4.07\%$
test_step_mdp_speed[False-False-False-True-False] 96.7500μs 27.5727μs 36.2678 KOps/s 35.9919 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-False-False-False-True] 99.4400μs 24.6298μs 40.6013 KOps/s 39.4820 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-False-False-False-False] 64.5600μs 17.0138μs 58.7758 KOps/s 56.8551 KOps/s $\color{#35bf28}+3.38\%$
test_values[generalized_advantage_estimate-True-True] 9.7247ms 9.4017ms 106.3633 Ops/s 102.6333 Ops/s $\color{#35bf28}+3.63\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.1481ms 33.3282ms 30.0046 Ops/s 30.0689 Ops/s $\color{#d91a1a}-0.21\%$
test_values[td0_return_estimate-False-False] 0.2304ms 0.1698ms 5.8902 KOps/s 5.9524 KOps/s $\color{#d91a1a}-1.04\%$
test_values[td1_return_estimate-False-False] 24.9332ms 23.9103ms 41.8230 Ops/s 40.8014 Ops/s $\color{#35bf28}+2.50\%$
test_values[vec_td1_return_estimate-False-False] 36.7158ms 33.1973ms 30.1229 Ops/s 29.7355 Ops/s $\color{#35bf28}+1.30\%$
test_values[td_lambda_return_estimate-True-False] 38.0086ms 34.0871ms 29.3366 Ops/s 28.8748 Ops/s $\color{#35bf28}+1.60\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.8598ms 33.2649ms 30.0617 Ops/s 29.6995 Ops/s $\color{#35bf28}+1.22\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.1133ms 8.3605ms 119.6097 Ops/s 116.8549 Ops/s $\color{#35bf28}+2.36\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.8061ms 2.3435ms 426.7169 Ops/s 548.4821 Ops/s $\textbf{\color{#d91a1a}-22.20\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4401ms 0.3537ms 2.8271 KOps/s 2.7858 KOps/s $\color{#35bf28}+1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.4648ms 44.2323ms 22.6079 Ops/s 23.3540 Ops/s $\color{#d91a1a}-3.19\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8714ms 3.0422ms 328.7104 Ops/s 326.1959 Ops/s $\color{#35bf28}+0.77\%$
test_dqn_speed[False-None] 5.5369ms 1.3576ms 736.6205 Ops/s 734.0895 Ops/s $\color{#35bf28}+0.34\%$
test_dqn_speed[False-backward] 2.9433ms 1.8693ms 534.9522 Ops/s 548.6938 Ops/s $\color{#d91a1a}-2.50\%$
test_dqn_speed[True-None] 0.6025ms 0.4621ms 2.1638 KOps/s 2.1315 KOps/s $\color{#35bf28}+1.51\%$
test_dqn_speed[True-backward] 1.0566ms 0.8808ms 1.1354 KOps/s 802.1326 Ops/s $\textbf{\color{#35bf28}+41.54\%}$
test_dqn_speed[reduce-overhead-None] 0.6150ms 0.4638ms 2.1563 KOps/s 2.0691 KOps/s $\color{#35bf28}+4.21\%$
test_dqn_speed[reduce-overhead-backward] 0.9189ms 0.8793ms 1.1373 KOps/s 1.1275 KOps/s $\color{#35bf28}+0.86\%$
test_ddpg_speed[False-None] 3.4538ms 2.8180ms 354.8651 Ops/s 354.9943 Ops/s $\color{#d91a1a}-0.04\%$
test_ddpg_speed[False-backward] 5.0479ms 4.0113ms 249.2959 Ops/s 248.1426 Ops/s $\color{#35bf28}+0.46\%$
test_ddpg_speed[True-None] 1.3305ms 1.0082ms 991.8528 Ops/s 973.8392 Ops/s $\color{#35bf28}+1.85\%$
test_ddpg_speed[True-backward] 1.9720ms 1.8925ms 528.4085 Ops/s 518.1681 Ops/s $\color{#35bf28}+1.98\%$
test_ddpg_speed[reduce-overhead-None] 1.3131ms 1.0123ms 987.8146 Ops/s 984.1341 Ops/s $\color{#35bf28}+0.37\%$
test_ddpg_speed[reduce-overhead-backward] 2.0792ms 1.9570ms 510.9862 Ops/s 525.0123 Ops/s $\color{#d91a1a}-2.67\%$
test_sac_speed[False-None] 10.4712ms 8.4697ms 118.0674 Ops/s 126.1703 Ops/s $\textbf{\color{#d91a1a}-6.42\%}$
test_sac_speed[False-backward] 12.6035ms 11.4535ms 87.3092 Ops/s 92.7244 Ops/s $\textbf{\color{#d91a1a}-5.84\%}$
test_sac_speed[True-None] 2.0635ms 1.9026ms 525.6094 Ops/s 538.5749 Ops/s $\color{#d91a1a}-2.41\%$
test_sac_speed[True-backward] 4.3813ms 3.8906ms 257.0281 Ops/s 282.3997 Ops/s $\textbf{\color{#d91a1a}-8.98\%}$
test_sac_speed[reduce-overhead-None] 2.2273ms 1.9492ms 513.0220 Ops/s 533.1823 Ops/s $\color{#d91a1a}-3.78\%$
test_sac_speed[reduce-overhead-backward] 4.0450ms 3.5757ms 279.6670 Ops/s 283.3385 Ops/s $\color{#d91a1a}-1.30\%$
test_redq_speed[False-None] 15.4419ms 13.0335ms 76.7254 Ops/s 79.4833 Ops/s $\color{#d91a1a}-3.47\%$
test_redq_speed[False-backward] 23.9568ms 22.1367ms 45.1740 Ops/s 45.1937 Ops/s $\color{#d91a1a}-0.04\%$
test_redq_speed[True-None] 5.8527ms 4.7262ms 211.5885 Ops/s 219.8861 Ops/s $\color{#d91a1a}-3.77\%$
test_redq_speed[True-backward] 13.2475ms 12.0826ms 82.7638 Ops/s 80.6972 Ops/s $\color{#35bf28}+2.56\%$
test_redq_speed[reduce-overhead-None] 5.2809ms 4.5266ms 220.9187 Ops/s 208.2050 Ops/s $\textbf{\color{#35bf28}+6.11\%}$
test_redq_speed[reduce-overhead-backward] 13.6144ms 12.1756ms 82.1312 Ops/s 83.4009 Ops/s $\color{#d91a1a}-1.52\%$
test_redq_deprec_speed[False-None] 14.6759ms 12.7774ms 78.2629 Ops/s 78.8034 Ops/s $\color{#d91a1a}-0.69\%$
test_redq_deprec_speed[False-backward] 21.2917ms 18.9844ms 52.6747 Ops/s 54.0264 Ops/s $\color{#d91a1a}-2.50\%$
test_redq_deprec_speed[True-None] 4.5867ms 3.8526ms 259.5661 Ops/s 279.3884 Ops/s $\textbf{\color{#d91a1a}-7.09\%}$
test_redq_deprec_speed[True-backward] 9.7748ms 8.5100ms 117.5082 Ops/s 125.1835 Ops/s $\textbf{\color{#d91a1a}-6.13\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.4753ms 3.8762ms 257.9865 Ops/s 279.6999 Ops/s $\textbf{\color{#d91a1a}-7.76\%}$
test_redq_deprec_speed[reduce-overhead-backward] 9.1383ms 8.7609ms 114.1435 Ops/s 126.5176 Ops/s $\textbf{\color{#d91a1a}-9.78\%}$
test_td3_speed[False-None] 8.2674ms 7.8680ms 127.0978 Ops/s 126.5865 Ops/s $\color{#35bf28}+0.40\%$
test_td3_speed[False-backward] 12.2987ms 10.3351ms 96.7577 Ops/s 96.7065 Ops/s $\color{#35bf28}+0.05\%$
test_td3_speed[True-None] 1.9910ms 1.7664ms 566.1342 Ops/s 561.5100 Ops/s $\color{#35bf28}+0.82\%$
test_td3_speed[True-backward] 3.5566ms 3.4345ms 291.1633 Ops/s 293.7616 Ops/s $\color{#d91a1a}-0.88\%$
test_td3_speed[reduce-overhead-None] 1.8689ms 1.7354ms 576.2387 Ops/s 568.6166 Ops/s $\color{#35bf28}+1.34\%$
test_td3_speed[reduce-overhead-backward] 3.4049ms 3.3163ms 301.5439 Ops/s 294.4823 Ops/s $\color{#35bf28}+2.40\%$
test_cql_speed[False-None] 37.0854ms 34.9622ms 28.6023 Ops/s 27.5948 Ops/s $\color{#35bf28}+3.65\%$
test_cql_speed[False-backward] 0.3012s 50.1941ms 19.9227 Ops/s 22.0524 Ops/s $\textbf{\color{#d91a1a}-9.66\%}$
test_cql_speed[True-None] 16.5020ms 15.4366ms 64.7809 Ops/s 64.2797 Ops/s $\color{#35bf28}+0.78\%$
test_cql_speed[True-backward] 22.7751ms 21.6239ms 46.2451 Ops/s 45.2676 Ops/s $\color{#35bf28}+2.16\%$
test_cql_speed[reduce-overhead-None] 17.5435ms 15.5852ms 64.1633 Ops/s 63.0701 Ops/s $\color{#35bf28}+1.73\%$
test_cql_speed[reduce-overhead-backward] 24.1819ms 22.3814ms 44.6799 Ops/s 44.8792 Ops/s $\color{#d91a1a}-0.44\%$
test_a2c_speed[False-None] 8.4867ms 7.0969ms 140.9073 Ops/s 140.6144 Ops/s $\color{#35bf28}+0.21\%$
test_a2c_speed[False-backward] 15.6241ms 14.1887ms 70.4785 Ops/s 69.7988 Ops/s $\color{#35bf28}+0.97\%$
test_a2c_speed[True-None] 3.6026ms 3.3166ms 301.5129 Ops/s 298.5270 Ops/s $\color{#35bf28}+1.00\%$
test_a2c_speed[True-backward] 10.0982ms 9.6906ms 103.1923 Ops/s 101.8688 Ops/s $\color{#35bf28}+1.30\%$
test_a2c_speed[reduce-overhead-None] 3.6913ms 3.3107ms 302.0539 Ops/s 298.0604 Ops/s $\color{#35bf28}+1.34\%$
test_a2c_speed[reduce-overhead-backward] 10.0119ms 9.7044ms 103.0456 Ops/s 103.1060 Ops/s $\color{#d91a1a}-0.06\%$
test_ppo_speed[False-None] 8.7894ms 7.3297ms 136.4318 Ops/s 133.6018 Ops/s $\color{#35bf28}+2.12\%$
test_ppo_speed[False-backward] 16.3359ms 14.5622ms 68.6711 Ops/s 68.4093 Ops/s $\color{#35bf28}+0.38\%$
test_ppo_speed[True-None] 4.1413ms 3.6965ms 270.5226 Ops/s 266.2704 Ops/s $\color{#35bf28}+1.60\%$
test_ppo_speed[True-backward] 10.2010ms 9.5967ms 104.2021 Ops/s 103.8591 Ops/s $\color{#35bf28}+0.33\%$
test_ppo_speed[reduce-overhead-None] 4.4494ms 3.7113ms 269.4439 Ops/s 266.4053 Ops/s $\color{#35bf28}+1.14\%$
test_ppo_speed[reduce-overhead-backward] 9.9024ms 9.5518ms 104.6918 Ops/s 104.4906 Ops/s $\color{#35bf28}+0.19\%$
test_reinforce_speed[False-None] 9.6062ms 6.6349ms 150.7182 Ops/s 153.3618 Ops/s $\color{#d91a1a}-1.72\%$
test_reinforce_speed[False-backward] 10.7906ms 9.9173ms 100.8340 Ops/s 102.7456 Ops/s $\color{#d91a1a}-1.86\%$
test_reinforce_speed[True-None] 3.2871ms 2.6331ms 379.7749 Ops/s 375.8748 Ops/s $\color{#35bf28}+1.04\%$
test_reinforce_speed[True-backward] 9.3064ms 8.5835ms 116.5030 Ops/s 116.2682 Ops/s $\color{#35bf28}+0.20\%$
test_reinforce_speed[reduce-overhead-None] 3.0717ms 2.6300ms 380.2253 Ops/s 368.3765 Ops/s $\color{#35bf28}+3.22\%$
test_reinforce_speed[reduce-overhead-backward] 9.8255ms 8.6970ms 114.9824 Ops/s 114.3237 Ops/s $\color{#35bf28}+0.58\%$
test_iql_speed[False-None] 33.8020ms 31.9726ms 31.2768 Ops/s 31.2965 Ops/s $\color{#d91a1a}-0.06\%$
test_iql_speed[False-backward] 46.6541ms 44.9193ms 22.2622 Ops/s 22.0245 Ops/s $\color{#35bf28}+1.08\%$
test_iql_speed[True-None] 12.5309ms 10.6012ms 94.3288 Ops/s 93.8794 Ops/s $\color{#35bf28}+0.48\%$
test_iql_speed[True-backward] 23.6371ms 21.4401ms 46.6416 Ops/s 46.1691 Ops/s $\color{#35bf28}+1.02\%$
test_iql_speed[reduce-overhead-None] 11.6367ms 10.8209ms 92.4141 Ops/s 93.0866 Ops/s $\color{#d91a1a}-0.72\%$
test_iql_speed[reduce-overhead-backward] 22.6933ms 21.9945ms 45.4660 Ops/s 44.8262 Ops/s $\color{#35bf28}+1.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.2728ms 4.9754ms 200.9892 Ops/s 196.1090 Ops/s $\color{#35bf28}+2.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.8577ms 0.4842ms 2.0653 KOps/s 2.0366 KOps/s $\color{#35bf28}+1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7034ms 0.4615ms 2.1670 KOps/s 2.1771 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8779ms 4.7795ms 209.2275 Ops/s 218.8328 Ops/s $\color{#d91a1a}-4.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2849ms 0.4756ms 2.1028 KOps/s 2.0833 KOps/s $\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6555ms 0.4532ms 2.2066 KOps/s 2.1974 KOps/s $\color{#35bf28}+0.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1937ms 1.5879ms 629.7817 Ops/s 624.0680 Ops/s $\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2815ms 1.5728ms 635.8093 Ops/s 635.6065 Ops/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1475ms 4.9134ms 203.5267 Ops/s 197.6443 Ops/s $\color{#35bf28}+2.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.3958ms 0.6206ms 1.6113 KOps/s 1.5831 KOps/s $\color{#35bf28}+1.78\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8586ms 0.5923ms 1.6884 KOps/s 1.6668 KOps/s $\color{#35bf28}+1.30\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9502ms 4.7784ms 209.2766 Ops/s 210.0597 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9915ms 0.4872ms 2.0526 KOps/s 2.0699 KOps/s $\color{#d91a1a}-0.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6161ms 0.4589ms 2.1790 KOps/s 2.1717 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.4174ms 4.7964ms 208.4876 Ops/s 215.4677 Ops/s $\color{#d91a1a}-3.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.6611ms 0.4800ms 2.0835 KOps/s 2.0592 KOps/s $\color{#35bf28}+1.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6703ms 0.4551ms 2.1975 KOps/s 2.2333 KOps/s $\color{#d91a1a}-1.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.2719ms 4.9217ms 203.1835 Ops/s 207.4891 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.6037ms 0.6315ms 1.5836 KOps/s 1.5888 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8070ms 0.5964ms 1.6766 KOps/s 1.6792 KOps/s $\color{#d91a1a}-0.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.4593ms 4.2030ms 237.9250 Ops/s 234.8942 Ops/s $\color{#35bf28}+1.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.5249ms 2.2456ms 445.3204 Ops/s 444.5078 Ops/s $\color{#35bf28}+0.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.5023ms 1.2743ms 784.7262 Ops/s 641.7431 Ops/s $\textbf{\color{#35bf28}+22.28\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4064s 12.2541ms 81.6057 Ops/s 235.3285 Ops/s $\textbf{\color{#d91a1a}-65.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 11.6144ms 2.3791ms 420.3247 Ops/s 425.9836 Ops/s $\color{#d91a1a}-1.33\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.9675ms 1.2581ms 794.8339 Ops/s 775.8017 Ops/s $\color{#35bf28}+2.45\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.4963ms 4.2858ms 233.3265 Ops/s 225.4994 Ops/s $\color{#35bf28}+3.47\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 5.1001ms 2.3556ms 424.5215 Ops/s 415.0262 Ops/s $\color{#35bf28}+2.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.8785ms 1.4748ms 678.0590 Ops/s 686.5221 Ops/s $\color{#d91a1a}-1.23\%$

@vmoens vmoens added Environments Adds or modifies an environment wrapper Data Data-related PR, will launch data-related jobs labels Oct 22, 2024
@vmoens vmoens merged commit 1431ae5 into main Oct 22, 2024
74 of 76 checks passed
@vmoens vmoens deleted the v0.6.0-changes branch October 22, 2024 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Data Data-related PR, will launch data-related jobs Environments Adds or modifies an environment wrapper versioning Versioning change (version number etc)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot