[BugFix, Test] Fix torch.vmap call in RNN tests

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1749

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (27 Unrelated Failures)

As of commit 960f5d1 with merge base b3d2aa6 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Brax Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Build M1 Wheels / pytorch/rl / wheel-py3_8-cpu (gh)
  ImportError: cannot import name 'MemoryMappedTensor' from 'tensordict' (/Users/ec2-user/runner/_work/_temp/conda_environment_7221907375/lib/python3.8/site-packages/tensordict/__init__.py)
• Envpool Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Generate documentation / build-docs (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Jumanji Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Lint / c-source / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Lint / python-source-and-configs / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Olddeps Tests on Linux / unittests (3.8, 11.6) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Optdeps Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• PettingZoo Tests on Linux / unittests / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Robohive Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Sklearn Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Tests on Linux GPU (Gym) / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux CPU / tests (3.10) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux CPU / tests (3.11) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux CPU / tests (3.8) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux CPU / tests (3.9) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux GPU / tests (3.8, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux GPU, latest stable release / tests (3.8, 11.8) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  The process '/usr/local/bin/git' failed with exit code 128
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  The process '/usr/local/bin/git' failed with exit code 128
• Unit-tests on Windows CPU / unittests / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Unit-tests on Windows GPU / unittests / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Vmas Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	62.8076ms	62.4393ms	16.0155 Ops/s	15.1526 Ops/s	$\textbf{\color{#35bf28}+5.70\%}$
test_sync	47.8621ms	40.0243ms	24.9848 Ops/s	28.2639 Ops/s	$\textbf{\color{#d91a1a}-11.60\%}$
test_async	66.2293ms	33.5234ms	29.8299 Ops/s	30.4774 Ops/s	$\color{#d91a1a}-2.12\%$
test_simple	0.4996s	0.4468s	2.2384 Ops/s	2.3024 Ops/s	$\color{#d91a1a}-2.78\%$
test_transformed	0.6625s	0.6163s	1.6225 Ops/s	1.6763 Ops/s	$\color{#d91a1a}-3.21\%$
test_serial	1.4350s	1.3858s	0.7216 Ops/s	0.7381 Ops/s	$\color{#d91a1a}-2.24\%$
test_parallel	1.3839s	1.3254s	0.7545 Ops/s	0.7574 Ops/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-True-True-True-True]	0.2561ms	21.4897μs	46.5339 KOps/s	45.9318 KOps/s	$\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-True-True-True-False]	48.7220μs	12.9777μs	77.0551 KOps/s	75.3643 KOps/s	$\color{#35bf28}+2.24\%$
test_step_mdp_speed[True-True-True-False-True]	33.6030μs	12.8733μs	77.6799 KOps/s	77.5502 KOps/s	$\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-True-False-False]	31.6690μs	7.7691μs	128.7143 KOps/s	128.5190 KOps/s	$\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-True-False-True-True]	67.4160μs	22.9657μs	43.5432 KOps/s	43.2449 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-False-True-False]	50.6650μs	14.3357μs	69.7559 KOps/s	69.0043 KOps/s	$\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-True-False-False-True]	37.0700μs	13.9750μs	71.5564 KOps/s	70.8159 KOps/s	$\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-True-False-False-False]	34.6150μs	8.9624μs	111.5767 KOps/s	111.2325 KOps/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-True-True-True]	68.7880μs	24.3442μs	41.0776 KOps/s	41.1792 KOps/s	$\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-False-True-True-False]	68.5080μs	15.6103μs	64.0604 KOps/s	63.2762 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-True-False-True]	61.4350μs	14.6749μs	68.1438 KOps/s	71.4461 KOps/s	$\color{#d91a1a}-4.62\%$
test_step_mdp_speed[True-False-True-False-False]	36.5980μs	8.8602μs	112.8639 KOps/s	111.0069 KOps/s	$\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-False-True-True]	72.4460μs	25.2419μs	39.6166 KOps/s	39.0399 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[True-False-False-True-False]	42.4190μs	16.6475μs	60.0691 KOps/s	58.2320 KOps/s	$\color{#35bf28}+3.15\%$
test_step_mdp_speed[True-False-False-False-True]	49.4220μs	15.3227μs	65.2625 KOps/s	66.2650 KOps/s	$\color{#d91a1a}-1.51\%$
test_step_mdp_speed[True-False-False-False-False]	39.5340μs	10.0545μs	99.4576 KOps/s	98.1677 KOps/s	$\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-True-True-True-True]	50.7550μs	24.2190μs	41.2898 KOps/s	41.2527 KOps/s	$\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-True-True-True-False]	47.5090μs	15.7782μs	63.3786 KOps/s	63.2676 KOps/s	$\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-True-True-False-True]	45.2350μs	16.2675μs	61.4722 KOps/s	61.4435 KOps/s	$\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-True-False-False]	36.2880μs	10.1836μs	98.1969 KOps/s	97.9691 KOps/s	$\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-True-False-True-True]	73.9880μs	25.3311μs	39.4771 KOps/s	39.2229 KOps/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-False-True-False]	46.1670μs	16.8802μs	59.2410 KOps/s	58.6539 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-False-True]	44.4530μs	17.4252μs	57.3883 KOps/s	56.6256 KOps/s	$\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-False-False-False]	35.9870μs	11.3162μs	88.3686 KOps/s	87.5879 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-True-True]	75.1910μs	26.7101μs	37.4391 KOps/s	37.0483 KOps/s	$\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-False-True-True-False]	47.3690μs	18.3027μs	54.6369 KOps/s	54.0563 KOps/s	$\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-True-False-True]	45.0940μs	17.3046μs	57.7881 KOps/s	57.0386 KOps/s	$\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-False-True-False-False]	60.2520μs	11.4440μs	87.3822 KOps/s	86.6602 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-False-False-True-True]	65.9630μs	27.5827μs	36.2546 KOps/s	35.7494 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-False-False-True-False]	71.4530μs	19.4284μs	51.4709 KOps/s	51.6640 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-False-False-True]	63.2980μs	18.1446μs	55.1128 KOps/s	53.6618 KOps/s	$\color{#35bf28}+2.70\%$
test_step_mdp_speed[False-False-False-False-False]	43.4610μs	12.3743μs	80.8125 KOps/s	80.3298 KOps/s	$\color{#35bf28}+0.60\%$
test_values[generalized_advantage_estimate-True-True]	15.5545ms	12.0308ms	83.1197 Ops/s	82.0772 Ops/s	$\color{#35bf28}+1.27\%$
test_values[vec_generalized_advantage_estimate-True-True]	29.3489ms	27.5844ms	36.2523 Ops/s	37.3307 Ops/s	$\color{#d91a1a}-2.89\%$
test_values[td0_return_estimate-False-False]	0.2574ms	0.1762ms	5.6756 KOps/s	5.6760 KOps/s	$-0.01\%$
test_values[td1_return_estimate-False-False]	27.0316ms	25.8990ms	38.6116 Ops/s	38.8456 Ops/s	$\color{#d91a1a}-0.60\%$
test_values[vec_td1_return_estimate-False-False]	38.9585ms	27.8653ms	35.8869 Ops/s	36.7606 Ops/s	$\color{#d91a1a}-2.38\%$
test_values[td_lambda_return_estimate-True-False]	40.6832ms	36.3487ms	27.5113 Ops/s	28.0228 Ops/s	$\color{#d91a1a}-1.83\%$
test_values[vec_td_lambda_return_estimate-True-False]	37.4609ms	27.8805ms	35.8673 Ops/s	36.1307 Ops/s	$\color{#d91a1a}-0.73\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.1490ms	8.1151ms	123.2276 Ops/s	118.1126 Ops/s	$\color{#35bf28}+4.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	4.4376ms	1.7421ms	574.0098 Ops/s	547.9495 Ops/s	$\color{#35bf28}+4.76\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	17.1286ms	0.4494ms	2.2249 KOps/s	2.2428 KOps/s	$\color{#d91a1a}-0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	97.5582ms	43.5046ms	22.9861 Ops/s	25.0742 Ops/s	$\textbf{\color{#d91a1a}-8.33\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	13.1552ms	2.6493ms	377.4518 Ops/s	375.1538 Ops/s	$\color{#35bf28}+0.61\%$
test_dqn_speed	12.7713ms	1.6453ms	607.7835 Ops/s	631.6965 Ops/s	$\color{#d91a1a}-3.79\%$
test_ddpg_speed	14.1834ms	3.6481ms	274.1134 Ops/s	279.3947 Ops/s	$\color{#d91a1a}-1.89\%$
test_sac_speed	20.8433ms	10.2926ms	97.1574 Ops/s	98.3404 Ops/s	$\color{#d91a1a}-1.20\%$
test_redq_speed	30.8718ms	19.2190ms	52.0318 Ops/s	52.9653 Ops/s	$\color{#d91a1a}-1.76\%$
test_redq_deprec_speed	26.4381ms	15.1365ms	66.0656 Ops/s	66.7137 Ops/s	$\color{#d91a1a}-0.97\%$
test_td3_speed	17.7734ms	10.5285ms	94.9799 Ops/s	94.9441 Ops/s	$\color{#35bf28}+0.04\%$
test_cql_speed	43.1089ms	41.0103ms	24.3841 Ops/s	24.2604 Ops/s	$\color{#35bf28}+0.51\%$
test_a2c_speed	20.6110ms	8.6492ms	115.6177 Ops/s	115.4410 Ops/s	$\color{#35bf28}+0.15\%$
test_ppo_speed	20.5676ms	8.9425ms	111.8256 Ops/s	111.8898 Ops/s	$\color{#d91a1a}-0.06\%$
test_reinforce_speed	19.1376ms	7.7254ms	129.4437 Ops/s	129.7481 Ops/s	$\color{#d91a1a}-0.23\%$
test_iql_speed	47.0078ms	35.8034ms	27.9303 Ops/s	26.6412 Ops/s	$\color{#35bf28}+4.84\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.8089ms	1.8494ms	540.7050 Ops/s	536.2294 Ops/s	$\color{#35bf28}+0.83\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	97.1502ms	2.1169ms	472.3926 Ops/s	523.8074 Ops/s	$\textbf{\color{#d91a1a}-9.82\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	2.7381ms	1.9215ms	520.4337 Ops/s	509.0000 Ops/s	$\color{#35bf28}+2.25\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.4165ms	1.8443ms	542.1990 Ops/s	548.8476 Ops/s	$\color{#d91a1a}-1.21\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1100s	2.1481ms	465.5305 Ops/s	525.4201 Ops/s	$\textbf{\color{#d91a1a}-11.40\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	2.6940ms	1.9223ms	520.2203 Ops/s	517.4729 Ops/s	$\color{#35bf28}+0.53\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.9204ms	1.8580ms	538.2257 Ops/s	550.8556 Ops/s	$\color{#d91a1a}-2.29\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1006s	2.1280ms	469.9359 Ops/s	520.9244 Ops/s	$\textbf{\color{#d91a1a}-9.79\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	2.6111ms	1.9229ms	520.0508 Ops/s	516.6992 Ops/s	$\color{#35bf28}+0.65\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.9200ms	1.8647ms	536.2792 Ops/s	545.1883 Ops/s	$\color{#d91a1a}-1.63\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.1015s	2.1184ms	472.0507 Ops/s	516.6658 Ops/s	$\textbf{\color{#d91a1a}-8.64\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.1451ms	1.9252ms	519.4134 Ops/s	524.6772 Ops/s	$\color{#d91a1a}-1.00\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.2732ms	1.8586ms	538.0489 Ops/s	549.9556 Ops/s	$\color{#d91a1a}-2.17\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	98.7475ms	2.1240ms	470.8172 Ops/s	517.8122 Ops/s	$\textbf{\color{#d91a1a}-9.08\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	2.6841ms	1.9207ms	520.6324 Ops/s	459.9442 Ops/s	$\textbf{\color{#35bf28}+13.19\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.9523ms	1.8604ms	537.5328 Ops/s	546.7617 Ops/s	$\color{#d91a1a}-1.69\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1018s	2.1347ms	468.4467 Ops/s	524.6523 Ops/s	$\textbf{\color{#d91a1a}-10.71\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.0823ms	1.9389ms	515.7632 Ops/s	473.8928 Ops/s	$\textbf{\color{#35bf28}+8.84\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1430s	16.6582ms	60.0303 Ops/s	65.0311 Ops/s	$\textbf{\color{#d91a1a}-7.69\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	99.1697ms	14.0789ms	71.0284 Ops/s	64.1647 Ops/s	$\textbf{\color{#35bf28}+10.70\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1007s	15.9766ms	62.5916 Ops/s	72.3814 Ops/s	$\textbf{\color{#d91a1a}-13.53\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	98.6974ms	15.8126ms	63.2405 Ops/s	57.7917 Ops/s	$\textbf{\color{#35bf28}+9.43\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	98.1289ms	15.7747ms	63.3927 Ops/s	63.7324 Ops/s	$\color{#d91a1a}-0.53\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	97.3867ms	15.7415ms	63.5263 Ops/s	62.9115 Ops/s	$\color{#35bf28}+0.98\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	97.8906ms	15.6483ms	63.9047 Ops/s	64.2612 Ops/s	$\color{#d91a1a}-0.55\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	98.0763ms	15.7963ms	63.3059 Ops/s	64.2036 Ops/s	$\color{#d91a1a}-1.40\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	97.7369ms	15.5310ms	64.3875 Ops/s	63.4531 Ops/s	$\color{#35bf28}+1.47\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1225s	0.1220s	8.1972 Ops/s	8.1116 Ops/s	$\color{#35bf28}+1.06\%$
test_sync	0.1052s	0.1022s	9.7802 Ops/s	9.7395 Ops/s	$\color{#35bf28}+0.42\%$
test_async	0.2780s	99.3409ms	10.0663 Ops/s	9.9743 Ops/s	$\color{#35bf28}+0.92\%$
test_single_pixels	0.1315s	0.1312s	7.6192 Ops/s	6.8149 Ops/s	$\textbf{\color{#35bf28}+11.80\%}$
test_sync_pixels	97.7161ms	95.9467ms	10.4224 Ops/s	10.3017 Ops/s	$\color{#35bf28}+1.17\%$
test_async_pixels	0.2440s	90.2837ms	11.0762 Ops/s	10.9162 Ops/s	$\color{#35bf28}+1.47\%$
test_simple	0.9557s	0.8882s	1.1259 Ops/s	1.1056 Ops/s	$\color{#35bf28}+1.83\%$
test_transformed	1.2010s	1.1367s	0.8798 Ops/s	0.8842 Ops/s	$\color{#d91a1a}-0.50\%$
test_serial	2.5702s	2.5081s	0.3987 Ops/s	0.3947 Ops/s	$\color{#35bf28}+1.02\%$
test_parallel	2.5688s	2.4917s	0.4013 Ops/s	0.4002 Ops/s	$\color{#35bf28}+0.29\%$
test_step_mdp_speed[True-True-True-True-True]	0.1100ms	32.0479μs	31.2033 KOps/s	30.3586 KOps/s	$\color{#35bf28}+2.78\%$
test_step_mdp_speed[True-True-True-True-False]	42.1710μs	19.0853μs	52.3962 KOps/s	50.7519 KOps/s	$\color{#35bf28}+3.24\%$
test_step_mdp_speed[True-True-True-False-True]	44.0600μs	18.1625μs	55.0585 KOps/s	52.6591 KOps/s	$\color{#35bf28}+4.56\%$
test_step_mdp_speed[True-True-True-False-False]	31.0500μs	10.8837μs	91.8806 KOps/s	87.2812 KOps/s	$\textbf{\color{#35bf28}+5.27\%}$
test_step_mdp_speed[True-True-False-True-True]	62.7020μs	33.9303μs	29.4722 KOps/s	28.5018 KOps/s	$\color{#35bf28}+3.40\%$
test_step_mdp_speed[True-True-False-True-False]	45.8100μs	20.6211μs	48.4940 KOps/s	46.4573 KOps/s	$\color{#35bf28}+4.38\%$
test_step_mdp_speed[True-True-False-False-True]	45.2910μs	20.1269μs	49.6847 KOps/s	48.2051 KOps/s	$\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-True-False-False-False]	54.0310μs	12.8257μs	77.9683 KOps/s	76.0390 KOps/s	$\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-True-True-True]	65.6110μs	35.8300μs	27.9096 KOps/s	26.8490 KOps/s	$\color{#35bf28}+3.95\%$
test_step_mdp_speed[True-False-True-True-False]	48.0900μs	22.6789μs	44.0938 KOps/s	41.9339 KOps/s	$\textbf{\color{#35bf28}+5.15\%}$
test_step_mdp_speed[True-False-True-False-True]	46.8110μs	20.0614μs	49.8469 KOps/s	48.3588 KOps/s	$\color{#35bf28}+3.08\%$
test_step_mdp_speed[True-False-True-False-False]	79.5610μs	12.7517μs	78.4211 KOps/s	75.2299 KOps/s	$\color{#35bf28}+4.24\%$
test_step_mdp_speed[True-False-False-True-True]	66.0610μs	36.9649μs	27.0527 KOps/s	25.5737 KOps/s	$\textbf{\color{#35bf28}+5.78\%}$
test_step_mdp_speed[True-False-False-True-False]	50.0300μs	24.4241μs	40.9432 KOps/s	39.1238 KOps/s	$\color{#35bf28}+4.65\%$
test_step_mdp_speed[True-False-False-False-True]	59.0810μs	21.7028μs	46.0769 KOps/s	44.1386 KOps/s	$\color{#35bf28}+4.39\%$
test_step_mdp_speed[True-False-False-False-False]	34.8190μs	14.3636μs	69.6204 KOps/s	64.9933 KOps/s	$\textbf{\color{#35bf28}+7.12\%}$
test_step_mdp_speed[False-True-True-True-True]	0.1025ms	35.1501μs	28.4494 KOps/s	27.1386 KOps/s	$\color{#35bf28}+4.83\%$
test_step_mdp_speed[False-True-True-True-False]	43.0500μs	22.5369μs	44.3716 KOps/s	41.7999 KOps/s	$\textbf{\color{#35bf28}+6.15\%}$
test_step_mdp_speed[False-True-True-False-True]	60.6900μs	24.0510μs	41.5783 KOps/s	39.3361 KOps/s	$\textbf{\color{#35bf28}+5.70\%}$
test_step_mdp_speed[False-True-True-False-False]	38.6710μs	14.6533μs	68.2441 KOps/s	63.6217 KOps/s	$\textbf{\color{#35bf28}+7.27\%}$
test_step_mdp_speed[False-True-False-True-True]	0.1064ms	38.0699μs	26.2675 KOps/s	25.2716 KOps/s	$\color{#35bf28}+3.94\%$
test_step_mdp_speed[False-True-False-True-False]	46.2910μs	24.3170μs	41.1234 KOps/s	38.6854 KOps/s	$\textbf{\color{#35bf28}+6.30\%}$
test_step_mdp_speed[False-True-False-False-True]	95.3420μs	25.7404μs	38.8494 KOps/s	36.8558 KOps/s	$\textbf{\color{#35bf28}+5.41\%}$
test_step_mdp_speed[False-True-False-False-False]	40.8110μs	16.1846μs	61.7873 KOps/s	57.7907 KOps/s	$\textbf{\color{#35bf28}+6.92\%}$
test_step_mdp_speed[False-False-True-True-True]	69.3010μs	39.1597μs	25.5365 KOps/s	24.4304 KOps/s	$\color{#35bf28}+4.53\%$
test_step_mdp_speed[False-False-True-True-False]	51.4110μs	26.2563μs	38.0862 KOps/s	36.2705 KOps/s	$\textbf{\color{#35bf28}+5.01\%}$
test_step_mdp_speed[False-False-True-False-True]	50.9500μs	26.2068μs	38.1581 KOps/s	36.9246 KOps/s	$\color{#35bf28}+3.34\%$
test_step_mdp_speed[False-False-True-False-False]	77.6010μs	16.2805μs	61.4230 KOps/s	56.2097 KOps/s	$\textbf{\color{#35bf28}+9.27\%}$
test_step_mdp_speed[False-False-False-True-True]	72.0510μs	40.9749μs	24.4052 KOps/s	23.5806 KOps/s	$\color{#35bf28}+3.50\%$
test_step_mdp_speed[False-False-False-True-False]	53.6110μs	28.0367μs	35.6675 KOps/s	33.8049 KOps/s	$\textbf{\color{#35bf28}+5.51\%}$
test_step_mdp_speed[False-False-False-False-True]	50.0500μs	27.2383μs	36.7130 KOps/s	34.9153 KOps/s	$\textbf{\color{#35bf28}+5.15\%}$
test_step_mdp_speed[False-False-False-False-False]	39.5200μs	18.0202μs	55.4931 KOps/s	51.7892 KOps/s	$\textbf{\color{#35bf28}+7.15\%}$
test_values[generalized_advantage_estimate-True-True]	26.4578ms	25.7166ms	38.8854 Ops/s	39.2919 Ops/s	$\color{#d91a1a}-1.03\%$
test_values[vec_generalized_advantage_estimate-True-True]	83.1391ms	3.2432ms	308.3330 Ops/s	303.4988 Ops/s	$\color{#35bf28}+1.59\%$
test_values[td0_return_estimate-False-False]	0.1001ms	64.6685μs	15.4635 KOps/s	15.2039 KOps/s	$\color{#35bf28}+1.71\%$
test_values[td1_return_estimate-False-False]	55.1457ms	54.6413ms	18.3012 Ops/s	17.6576 Ops/s	$\color{#35bf28}+3.64\%$
test_values[vec_td1_return_estimate-False-False]	2.1025ms	1.7825ms	560.9994 Ops/s	560.3275 Ops/s	$\color{#35bf28}+0.12\%$
test_values[td_lambda_return_estimate-True-False]	92.5892ms	87.6684ms	11.4066 Ops/s	11.4279 Ops/s	$\color{#d91a1a}-0.19\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.1112ms	1.7799ms	561.8292 Ops/s	562.3896 Ops/s	$\color{#d91a1a}-0.10\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	26.2025ms	25.7324ms	38.8615 Ops/s	41.2378 Ops/s	$\textbf{\color{#d91a1a}-5.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9043ms	0.7194ms	1.3900 KOps/s	1.3869 KOps/s	$\color{#35bf28}+0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7517ms	0.6770ms	1.4772 KOps/s	1.4381 KOps/s	$\color{#35bf28}+2.72\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5564ms	1.4752ms	677.8586 Ops/s	678.6614 Ops/s	$\color{#d91a1a}-0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9609ms	0.6958ms	1.4373 KOps/s	1.4472 KOps/s	$\color{#d91a1a}-0.68\%$
test_dqn_speed	8.0850ms	1.4918ms	670.3166 Ops/s	667.2743 Ops/s	$\color{#35bf28}+0.46\%$
test_ddpg_speed	4.3855ms	3.3963ms	294.4421 Ops/s	287.4405 Ops/s	$\color{#35bf28}+2.44\%$
test_sac_speed	10.4132ms	9.4392ms	105.9409 Ops/s	104.9820 Ops/s	$\color{#35bf28}+0.91\%$
test_redq_speed	17.2237ms	16.6052ms	60.2221 Ops/s	59.7465 Ops/s	$\color{#35bf28}+0.80\%$
test_redq_deprec_speed	0.1031s	14.3621ms	69.6279 Ops/s	75.0448 Ops/s	$\textbf{\color{#d91a1a}-7.22\%}$
test_td3_speed	9.7352ms	9.6019ms	104.1465 Ops/s	102.0893 Ops/s	$\color{#35bf28}+2.02\%$
test_cql_speed	36.8292ms	34.6499ms	28.8601 Ops/s	28.5496 Ops/s	$\color{#35bf28}+1.09\%$
test_a2c_speed	8.2994ms	7.3339ms	136.3536 Ops/s	133.3931 Ops/s	$\color{#35bf28}+2.22\%$
test_ppo_speed	8.8733ms	7.6374ms	130.9346 Ops/s	128.5463 Ops/s	$\color{#35bf28}+1.86\%$
test_reinforce_speed	7.6710ms	6.4507ms	155.0225 Ops/s	156.4143 Ops/s	$\color{#d91a1a}-0.89\%$
test_iql_speed	29.2099ms	27.9990ms	35.7156 Ops/s	35.7360 Ops/s	$\color{#d91a1a}-0.06\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0311ms	2.4542ms	407.4643 Ops/s	401.5102 Ops/s	$\color{#35bf28}+1.48\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.6860ms	2.5965ms	385.1301 Ops/s	335.1977 Ops/s	$\textbf{\color{#35bf28}+14.90\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.2261ms	2.6279ms	380.5359 Ops/s	374.9521 Ops/s	$\color{#35bf28}+1.49\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0683ms	2.4592ms	406.6323 Ops/s	401.2521 Ops/s	$\color{#35bf28}+1.34\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.4357ms	2.6194ms	381.7735 Ops/s	333.2219 Ops/s	$\textbf{\color{#35bf28}+14.57\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.2759ms	2.6197ms	381.7287 Ops/s	375.7210 Ops/s	$\color{#35bf28}+1.60\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.0917ms	2.4614ms	406.2805 Ops/s	400.9802 Ops/s	$\color{#35bf28}+1.32\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.6899ms	2.6191ms	381.8133 Ops/s	332.9180 Ops/s	$\textbf{\color{#35bf28}+14.69\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.6108ms	2.6099ms	383.1513 Ops/s	373.7467 Ops/s	$\color{#35bf28}+2.52\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1096ms	2.4212ms	413.0163 Ops/s	400.4355 Ops/s	$\color{#35bf28}+3.14\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.6211ms	2.6254ms	380.8923 Ops/s	331.5702 Ops/s	$\textbf{\color{#35bf28}+14.88\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.5025ms	2.6138ms	382.5794 Ops/s	373.5629 Ops/s	$\color{#35bf28}+2.41\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.1694ms	2.4316ms	411.2585 Ops/s	398.2394 Ops/s	$\color{#35bf28}+3.27\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	4.1148ms	2.6374ms	379.1631 Ops/s	326.4806 Ops/s	$\textbf{\color{#35bf28}+16.14\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.8942ms	2.6290ms	380.3690 Ops/s	372.3251 Ops/s	$\color{#35bf28}+2.16\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.1916ms	2.4550ms	407.3320 Ops/s	401.1702 Ops/s	$\color{#35bf28}+1.54\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.8699ms	2.6212ms	381.5066 Ops/s	374.7938 Ops/s	$\color{#35bf28}+1.79\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.3378ms	2.6457ms	377.9657 Ops/s	374.5741 Ops/s	$\color{#35bf28}+0.91\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1969s	18.6060ms	53.7461 Ops/s	53.2901 Ops/s	$\color{#35bf28}+0.86\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1204s	16.9817ms	58.8870 Ops/s	57.4047 Ops/s	$\color{#35bf28}+2.58\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1210s	14.8040ms	67.5494 Ops/s	57.8341 Ops/s	$\textbf{\color{#35bf28}+16.80\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1195s	16.9653ms	58.9437 Ops/s	57.8664 Ops/s	$\color{#35bf28}+1.86\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1204s	16.9524ms	58.9886 Ops/s	57.2912 Ops/s	$\color{#35bf28}+2.96\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1210s	16.8921ms	59.1992 Ops/s	65.8044 Ops/s	$\textbf{\color{#d91a1a}-10.04\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1203s	16.9446ms	59.0159 Ops/s	57.6665 Ops/s	$\color{#35bf28}+2.34\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1221s	17.0507ms	58.6488 Ops/s	57.9571 Ops/s	$\color{#35bf28}+1.19\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1198s	16.9739ms	58.9140 Ops/s	65.7874 Ops/s	$\textbf{\color{#d91a1a}-10.45\%}$