Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Deprecate direct usage of memmap tensors #1699

Merged
merged 12 commits into from
Nov 15, 2023
Merged

[Refactor] Deprecate direct usage of memmap tensors #1699

merged 12 commits into from
Nov 15, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 15, 2023

Linted version of #1684

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 15, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1699

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 22 Unrelated Failures

As of commit 6595212 with merge base 02ff00d (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens marked this pull request as ready for review November 15, 2023 13:55
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 15, 2023
@vmoens vmoens merged commit 0a38cbc into main Nov 15, 2023
@vmoens vmoens added the Refactoring Refactoring of an existing feature label Nov 15, 2023
@vmoens vmoens deleted the deprecate_direct_memmap branch November 15, 2023 13:55
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 64.1957ms 62.7594ms 15.9339 Ops/s 14.6376 Ops/s $\textbf{\color{#35bf28}+8.86\%}$
test_sync 35.4797ms 34.9251ms 28.6327 Ops/s 28.5305 Ops/s $\color{#35bf28}+0.36\%$
test_async 82.7547ms 32.1677ms 31.0871 Ops/s 30.0093 Ops/s $\color{#35bf28}+3.59\%$
test_simple 0.5168s 0.4515s 2.2149 Ops/s 2.2438 Ops/s $\color{#d91a1a}-1.29\%$
test_transformed 0.6773s 0.6137s 1.6294 Ops/s 1.5942 Ops/s $\color{#35bf28}+2.21\%$
test_serial 1.3567s 1.2881s 0.7763 Ops/s 0.7818 Ops/s $\color{#d91a1a}-0.69\%$
test_parallel 1.3601s 1.3003s 0.7691 Ops/s 0.7794 Ops/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[True-True-True-True-True] 0.1572ms 24.2792μs 41.1876 KOps/s 40.9342 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-True-True-False] 42.6800μs 14.4001μs 69.4442 KOps/s 69.2496 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-True-True-False-True] 89.5540μs 15.0162μs 66.5948 KOps/s 65.7111 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-False-False] 38.8830μs 8.9966μs 111.1534 KOps/s 112.0968 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-False-True-True] 63.3980μs 25.6424μs 38.9979 KOps/s 38.4054 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-True-False-True-False] 48.2700μs 15.8325μs 63.1613 KOps/s 62.6315 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-False-True] 50.1640μs 16.3915μs 61.0071 KOps/s 60.3431 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-False-False-False] 36.8390μs 10.3473μs 96.6432 KOps/s 96.5144 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-False-True-True-True] 84.9180μs 27.1842μs 36.7861 KOps/s 36.2570 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-False-True-True-False] 50.4640μs 17.2644μs 57.9226 KOps/s 57.4052 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-False-True-False-True] 48.4910μs 16.6558μs 60.0392 KOps/s 60.9828 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[True-False-True-False-False] 45.3150μs 10.2049μs 97.9917 KOps/s 96.1526 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[True-False-False-True-True] 62.5170μs 28.5123μs 35.0726 KOps/s 35.0472 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-False-True-False] 70.3810μs 18.5488μs 53.9118 KOps/s 53.5524 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-False-False-False-True] 46.2860μs 17.7351μs 56.3854 KOps/s 56.7700 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-False-False-False-False] 49.1120μs 11.7278μs 85.2673 KOps/s 85.6721 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-True-True-True-True] 66.0930μs 27.6096μs 36.2193 KOps/s 36.6777 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-True-True-True-False] 49.1520μs 17.4997μs 57.1439 KOps/s 57.4817 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-True-False-True] 58.4290μs 19.1079μs 52.3345 KOps/s 52.6995 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-True-False-False] 42.2290μs 11.6241μs 86.0281 KOps/s 85.4586 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-True-False-True-True] 70.7620μs 28.5969μs 34.9689 KOps/s 34.8405 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-True-False-True-False] 50.9850μs 18.6997μs 53.4768 KOps/s 53.7711 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-False-True] 49.1320μs 20.3676μs 49.0975 KOps/s 48.8054 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-True-False-False-False] 38.9930μs 12.8433μs 77.8618 KOps/s 77.1200 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-True-True-True] 74.5090μs 29.8099μs 33.5459 KOps/s 33.2506 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-True-False] 48.4810μs 20.0808μs 49.7987 KOps/s 50.0880 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-True-False-True] 75.1900μs 20.1408μs 49.6506 KOps/s 49.0882 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[False-False-True-False-False] 43.4210μs 13.0794μs 76.4561 KOps/s 77.0398 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-False-False-True-True] 0.1086ms 30.8303μs 32.4356 KOps/s 32.5000 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-False-True-False] 0.7486ms 21.1320μs 47.3216 KOps/s 47.1698 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-False-False-False-True] 76.2020μs 21.2588μs 47.0393 KOps/s 46.8943 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-False-False] 46.4770μs 14.0272μs 71.2900 KOps/s 71.0424 KOps/s $\color{#35bf28}+0.35\%$
test_values[generalized_advantage_estimate-True-True] 12.6098ms 12.1484ms 82.3152 Ops/s 79.8955 Ops/s $\color{#35bf28}+3.03\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.0378ms 26.9375ms 37.1230 Ops/s 37.1546 Ops/s $\color{#d91a1a}-0.09\%$
test_values[td0_return_estimate-False-False] 0.3517ms 0.2016ms 4.9599 KOps/s 4.7713 KOps/s $\color{#35bf28}+3.95\%$
test_values[td1_return_estimate-False-False] 25.9774ms 25.6310ms 39.0152 Ops/s 38.6047 Ops/s $\color{#35bf28}+1.06\%$
test_values[vec_td1_return_estimate-False-False] 65.3528ms 28.8817ms 34.6240 Ops/s 37.1358 Ops/s $\textbf{\color{#d91a1a}-6.76\%}$
test_values[td_lambda_return_estimate-True-False] 38.8091ms 35.8150ms 27.9213 Ops/s 27.6072 Ops/s $\color{#35bf28}+1.14\%$
test_values[vec_td_lambda_return_estimate-True-False] 0.1058s 28.8665ms 34.6423 Ops/s 36.9765 Ops/s $\textbf{\color{#d91a1a}-6.31\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.1149ms 8.1551ms 122.6220 Ops/s 123.0100 Ops/s $\color{#d91a1a}-0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5740ms 1.8631ms 536.7461 Ops/s 530.1954 Ops/s $\color{#35bf28}+1.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.8571ms 0.4345ms 2.3016 KOps/s 2.3290 KOps/s $\color{#d91a1a}-1.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 52.0938ms 41.5507ms 24.0670 Ops/s 24.5346 Ops/s $\color{#d91a1a}-1.91\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.6595ms 2.5129ms 397.9449 Ops/s 383.8597 Ops/s $\color{#35bf28}+3.67\%$
test_dqn_speed 10.6976ms 1.6965ms 589.4566 Ops/s 583.3842 Ops/s $\color{#35bf28}+1.04\%$
test_ddpg_speed 11.7878ms 3.0882ms 323.8103 Ops/s 291.0083 Ops/s $\textbf{\color{#35bf28}+11.27\%}$
test_sac_speed 18.5336ms 9.0500ms 110.4967 Ops/s 109.2238 Ops/s $\color{#35bf28}+1.17\%$
test_redq_speed 27.5365ms 17.0493ms 58.6535 Ops/s 57.5220 Ops/s $\color{#35bf28}+1.97\%$
test_redq_deprec_speed 24.2029ms 15.7915ms 63.3253 Ops/s 63.1924 Ops/s $\color{#35bf28}+0.21\%$
test_td3_speed 11.4970ms 9.2821ms 107.7338 Ops/s 105.7204 Ops/s $\color{#35bf28}+1.90\%$
test_cql_speed 47.6124ms 37.8897ms 26.3924 Ops/s 26.4039 Ops/s $\color{#d91a1a}-0.04\%$
test_a2c_speed 18.0663ms 8.7146ms 114.7503 Ops/s 116.6953 Ops/s $\color{#d91a1a}-1.67\%$
test_ppo_speed 18.7867ms 8.7932ms 113.7237 Ops/s 112.0995 Ops/s $\color{#35bf28}+1.45\%$
test_reinforce_speed 17.2769ms 7.6014ms 131.5554 Ops/s 134.0801 Ops/s $\color{#d91a1a}-1.88\%$
test_iql_speed 43.3194ms 33.7373ms 29.6408 Ops/s 29.4345 Ops/s $\color{#35bf28}+0.70\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3706ms 1.9847ms 503.8505 Ops/s 465.0502 Ops/s $\textbf{\color{#35bf28}+8.34\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.2355ms 2.1161ms 472.5742 Ops/s 449.2953 Ops/s $\textbf{\color{#35bf28}+5.18\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.6789ms 2.1390ms 467.5099 Ops/s 453.3137 Ops/s $\color{#35bf28}+3.13\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.7445ms 2.0196ms 495.1520 Ops/s 484.9862 Ops/s $\color{#35bf28}+2.10\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.7789ms 2.1085ms 474.2801 Ops/s 450.3316 Ops/s $\textbf{\color{#35bf28}+5.32\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.9406ms 2.1162ms 472.5553 Ops/s 447.5623 Ops/s $\textbf{\color{#35bf28}+5.58\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.2006ms 2.0455ms 488.8667 Ops/s 468.1848 Ops/s $\color{#35bf28}+4.42\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.1692ms 2.1385ms 467.6133 Ops/s 455.8958 Ops/s $\color{#35bf28}+2.57\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.5516ms 2.1546ms 464.1340 Ops/s 452.1138 Ops/s $\color{#35bf28}+2.66\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3038ms 2.0330ms 491.8919 Ops/s 456.4282 Ops/s $\textbf{\color{#35bf28}+7.77\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.7347ms 2.1421ms 466.8317 Ops/s 397.8096 Ops/s $\textbf{\color{#35bf28}+17.35\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.1249ms 2.1513ms 464.8294 Ops/s 452.8047 Ops/s $\color{#35bf28}+2.66\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6280ms 2.0327ms 491.9486 Ops/s 482.1067 Ops/s $\color{#35bf28}+2.04\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.5710ms 2.1448ms 466.2454 Ops/s 463.7524 Ops/s $\color{#35bf28}+0.54\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.2527ms 2.1522ms 464.6464 Ops/s 381.9464 Ops/s $\textbf{\color{#35bf28}+21.65\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7851ms 2.0158ms 496.0798 Ops/s 492.8562 Ops/s $\color{#35bf28}+0.65\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.2025ms 2.0873ms 479.0927 Ops/s 455.2984 Ops/s $\textbf{\color{#35bf28}+5.23\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1413s 2.4393ms 409.9512 Ops/s 457.2509 Ops/s $\textbf{\color{#d91a1a}-10.34\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3113s 24.8941ms 40.1702 Ops/s 37.5292 Ops/s $\textbf{\color{#35bf28}+7.04\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1378s 23.3109ms 42.8985 Ops/s 46.4045 Ops/s $\textbf{\color{#d91a1a}-7.56\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1365s 20.6845ms 48.3453 Ops/s 41.0856 Ops/s $\textbf{\color{#35bf28}+17.67\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1445s 23.3264ms 42.8699 Ops/s 43.9719 Ops/s $\color{#d91a1a}-2.51\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1435s 18.6320ms 53.6712 Ops/s 39.8589 Ops/s $\textbf{\color{#35bf28}+34.65\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1386s 23.1571ms 43.1834 Ops/s 45.0821 Ops/s $\color{#d91a1a}-4.21\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1314s 20.4653ms 48.8632 Ops/s 40.0856 Ops/s $\textbf{\color{#35bf28}+21.90\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1402s 23.0782ms 43.3309 Ops/s 44.1694 Ops/s $\color{#d91a1a}-1.90\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1498s 21.2198ms 47.1257 Ops/s 40.1743 Ops/s $\textbf{\color{#35bf28}+17.30\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.0473ms 57.4712ms 17.4000 Ops/s 16.4949 Ops/s $\textbf{\color{#35bf28}+5.49\%}$
test_sync 36.6815ms 31.2438ms 32.0063 Ops/s 31.2166 Ops/s $\color{#35bf28}+2.53\%$
test_async 58.7357ms 29.1236ms 34.3365 Ops/s 32.9137 Ops/s $\color{#35bf28}+4.32\%$
test_simple 0.4662s 0.4125s 2.4243 Ops/s 2.4447 Ops/s $\color{#d91a1a}-0.83\%$
test_transformed 0.6015s 0.5669s 1.7640 Ops/s 1.7685 Ops/s $\color{#d91a1a}-0.25\%$
test_serial 1.2175s 1.1709s 0.8540 Ops/s 0.8527 Ops/s $\color{#35bf28}+0.15\%$
test_parallel 1.1093s 1.0634s 0.9404 Ops/s 0.9293 Ops/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-True-True-True-True] 0.1360ms 20.1309μs 49.6749 KOps/s 49.6749 KOps/s $-0.00\%$
test_step_mdp_speed[True-True-True-True-False] 64.7340μs 11.9092μs 83.9688 KOps/s 82.9664 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-True-True-False-True] 34.1740μs 12.2485μs 81.6426 KOps/s 80.6017 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-True-True-False-False] 41.9890μs 7.2221μs 138.4630 KOps/s 136.3127 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-True-False-True-True] 54.7330μs 20.9516μs 47.7290 KOps/s 47.1865 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-False-True-False] 85.9010μs 13.1591μs 75.9932 KOps/s 75.8507 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-True-False-False-True] 47.4990μs 13.3352μs 74.9894 KOps/s 73.9359 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-True-False-False-False] 22.5620μs 8.3641μs 119.5584 KOps/s 118.5652 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-False-True-True-True] 56.3350μs 22.1525μs 45.1416 KOps/s 44.9850 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-False-True-True-False] 80.0500μs 14.1865μs 70.4898 KOps/s 69.5804 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-False-True-False-True] 46.4970μs 13.5218μs 73.9545 KOps/s 74.1738 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-True-False-False] 42.1200μs 8.3164μs 120.2437 KOps/s 117.3466 KOps/s $\color{#35bf28}+2.47\%$
test_step_mdp_speed[True-False-False-True-True] 61.5060μs 23.2919μs 42.9333 KOps/s 42.8539 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-False-True-False] 33.6530μs 15.1729μs 65.9069 KOps/s 64.6106 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[True-False-False-False-True] 61.9360μs 14.5721μs 68.6245 KOps/s 68.8972 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-False-False] 35.0660μs 9.2313μs 108.3271 KOps/s 103.9442 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[False-True-True-True-True] 53.5110μs 21.9875μs 45.4804 KOps/s 44.8636 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-True-True-True-False] 29.8460μs 14.1328μs 70.7573 KOps/s 69.2007 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-True-True-False-True] 46.3880μs 15.2883μs 65.4093 KOps/s 63.7670 KOps/s $\color{#35bf28}+2.58\%$
test_step_mdp_speed[False-True-True-False-False] 24.8570μs 9.3505μs 106.9458 KOps/s 104.1474 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[False-True-False-True-True] 66.1440μs 22.9547μs 43.5640 KOps/s 43.1340 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-True-False] 60.2130μs 15.2095μs 65.7483 KOps/s 65.2561 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-True-False-False-True] 43.8730μs 16.1431μs 61.9460 KOps/s 59.7852 KOps/s $\color{#35bf28}+3.61\%$
test_step_mdp_speed[False-True-False-False-False] 37.7310μs 10.4487μs 95.7059 KOps/s 94.2469 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-False-True-True-True] 69.5210μs 24.2258μs 41.2784 KOps/s 40.5804 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[False-False-True-True-False] 44.5230μs 16.2426μs 61.5666 KOps/s 61.0244 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-False-True] 75.4590μs 16.2542μs 61.5224 KOps/s 59.7290 KOps/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[False-False-True-False-False] 40.3780μs 10.4470μs 95.7214 KOps/s 93.3429 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-False-False-True-True] 74.5690μs 25.0601μs 39.9041 KOps/s 39.3909 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-False-True-False] 62.9480μs 17.2811μs 57.8668 KOps/s 57.6572 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-True] 47.4290μs 17.0399μs 58.6857 KOps/s 57.2150 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[False-False-False-False-False] 55.8250μs 11.5128μs 86.8595 KOps/s 86.2253 KOps/s $\color{#35bf28}+0.74\%$
test_values[generalized_advantage_estimate-True-True] 12.4000ms 11.6836ms 85.5902 Ops/s 84.3105 Ops/s $\color{#35bf28}+1.52\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.6003ms 26.5915ms 37.6061 Ops/s 37.0664 Ops/s $\color{#35bf28}+1.46\%$
test_values[td0_return_estimate-False-False] 0.2678ms 0.1763ms 5.6718 KOps/s 5.7081 KOps/s $\color{#d91a1a}-0.64\%$
test_values[td1_return_estimate-False-False] 25.8091ms 25.0456ms 39.9272 Ops/s 39.5307 Ops/s $\color{#35bf28}+1.00\%$
test_values[vec_td1_return_estimate-False-False] 36.6499ms 26.6825ms 37.4777 Ops/s 36.8349 Ops/s $\color{#35bf28}+1.75\%$
test_values[td_lambda_return_estimate-True-False] 39.2162ms 35.5860ms 28.1009 Ops/s 28.1322 Ops/s $\color{#d91a1a}-0.11\%$
test_values[vec_td_lambda_return_estimate-True-False] 75.0613ms 28.1777ms 35.4890 Ops/s 37.4296 Ops/s $\textbf{\color{#d91a1a}-5.18\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.5899ms 7.7128ms 129.6546 Ops/s 126.4737 Ops/s $\color{#35bf28}+2.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2449ms 1.8469ms 541.4358 Ops/s 523.5845 Ops/s $\color{#35bf28}+3.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5102ms 0.4311ms 2.3195 KOps/s 1.8326 KOps/s $\textbf{\color{#35bf28}+26.56\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.0986ms 37.5356ms 26.6414 Ops/s 25.8236 Ops/s $\color{#35bf28}+3.17\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 12.5295ms 2.5090ms 398.5720 Ops/s 395.6193 Ops/s $\color{#35bf28}+0.75\%$
test_dqn_speed 12.5292ms 1.5902ms 628.8510 Ops/s 624.7546 Ops/s $\color{#35bf28}+0.66\%$
test_ddpg_speed 14.2267ms 2.8709ms 348.3198 Ops/s 350.3232 Ops/s $\color{#d91a1a}-0.57\%$
test_sac_speed 19.1700ms 8.2643ms 121.0020 Ops/s 119.2105 Ops/s $\color{#35bf28}+1.50\%$
test_redq_speed 26.7777ms 15.3464ms 65.1620 Ops/s 65.0666 Ops/s $\color{#35bf28}+0.15\%$
test_redq_deprec_speed 25.0210ms 13.6577ms 73.2188 Ops/s 72.6672 Ops/s $\color{#35bf28}+0.76\%$
test_td3_speed 9.8861ms 8.3252ms 120.1178 Ops/s 120.4542 Ops/s $\color{#d91a1a}-0.28\%$
test_cql_speed 46.3407ms 35.4737ms 28.1899 Ops/s 28.3493 Ops/s $\color{#d91a1a}-0.56\%$
test_a2c_speed 18.8405ms 7.7825ms 128.4941 Ops/s 128.0165 Ops/s $\color{#35bf28}+0.37\%$
test_ppo_speed 19.5391ms 8.0762ms 123.8203 Ops/s 123.6438 Ops/s $\color{#35bf28}+0.14\%$
test_reinforce_speed 18.0472ms 6.9289ms 144.3236 Ops/s 143.8136 Ops/s $\color{#35bf28}+0.35\%$
test_iql_speed 44.9756ms 31.9062ms 31.3418 Ops/s 31.3492 Ops/s $\color{#d91a1a}-0.02\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 1.8935ms 1.6136ms 619.7176 Ops/s 626.6799 Ops/s $\color{#d91a1a}-1.11\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.3926ms 1.6983ms 588.8224 Ops/s 572.7952 Ops/s $\color{#35bf28}+2.80\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.7434ms 1.6953ms 589.8596 Ops/s 582.0309 Ops/s $\color{#35bf28}+1.35\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.1438ms 1.5820ms 632.1098 Ops/s 633.7583 Ops/s $\color{#d91a1a}-0.26\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.9033ms 1.7078ms 585.5397 Ops/s 582.1004 Ops/s $\color{#35bf28}+0.59\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2.4339ms 1.6955ms 589.7919 Ops/s 576.8390 Ops/s $\color{#35bf28}+2.25\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.1252ms 1.5741ms 635.2781 Ops/s 632.1824 Ops/s $\color{#35bf28}+0.49\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.6585ms 1.7128ms 583.8359 Ops/s 589.0915 Ops/s $\color{#d91a1a}-0.89\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2.6622ms 1.7018ms 587.6066 Ops/s 590.7378 Ops/s $\color{#d91a1a}-0.53\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2754ms 1.5791ms 633.2594 Ops/s 622.7099 Ops/s $\color{#35bf28}+1.69\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.9938ms 1.7289ms 578.3908 Ops/s 506.1112 Ops/s $\textbf{\color{#35bf28}+14.28\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.1256s 1.9636ms 509.2721 Ops/s 572.5025 Ops/s $\textbf{\color{#d91a1a}-11.04\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.3129ms 1.6256ms 615.1723 Ops/s 625.8569 Ops/s $\color{#d91a1a}-1.71\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.5385ms 1.6852ms 593.3927 Ops/s 571.6681 Ops/s $\color{#35bf28}+3.80\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.3936ms 1.7011ms 587.8704 Ops/s 587.6919 Ops/s $\color{#35bf28}+0.03\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.2577ms 1.5900ms 628.9185 Ops/s 635.0820 Ops/s $\color{#d91a1a}-0.97\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.3432ms 1.6901ms 591.6949 Ops/s 583.4292 Ops/s $\color{#35bf28}+1.42\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2.6085ms 1.7042ms 586.7799 Ops/s 565.1628 Ops/s $\color{#35bf28}+3.82\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2097s 19.6253ms 50.9546 Ops/s 47.6241 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1104s 19.3452ms 51.6925 Ops/s 57.1536 Ops/s $\textbf{\color{#d91a1a}-9.56\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1130s 17.6293ms 56.7236 Ops/s 52.1815 Ops/s $\textbf{\color{#35bf28}+8.70\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1093s 19.1787ms 52.1412 Ops/s 57.6503 Ops/s $\textbf{\color{#d91a1a}-9.56\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1107s 19.1487ms 52.2229 Ops/s 50.5727 Ops/s $\color{#35bf28}+3.26\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1254s 17.8542ms 56.0093 Ops/s 50.8177 Ops/s $\textbf{\color{#35bf28}+10.22\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1132s 19.4584ms 51.3917 Ops/s 57.0010 Ops/s $\textbf{\color{#d91a1a}-9.84\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1047s 17.1851ms 58.1899 Ops/s 49.9727 Ops/s $\textbf{\color{#35bf28}+16.44\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1053s 19.0145ms 52.5915 Ops/s 57.2271 Ops/s $\textbf{\color{#d91a1a}-8.10\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot