[BugFix] MLFlow logger

[GJBoth]

Currently, the MLFlow logger always tries to create a new experiment by calling create_experiment. If the experiment name already exists, this throws an error - so in practice running your script for the second time will crash the run. Additionally, MLFlow considers an experiment as a collection of runs, so we don't want to create a new experiment for each run.

This PR modifies this logic by first checking if an experiment exists, creating one if necessary, and otherwise just retrieves the experiment ID.

Additionally, the MLFlow runs should be closed using mlflow.end_run() when it's not called as a context manager (as is the case for us). Do we want to add a shutdown method to the Logger base class to handle that? I can do that in this PR or in another.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2152

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 2 Unrelated Failures

As of commit 98fb66f with merge base 5b9cb44 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 60edc504b6f4d059b60800ede96b3977113305e245c7f29978dce7594dc6afea /exec failed with exit code 139
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t eb25683f40ac3fb430c3a637de561b2bba287e70a6e59c5fcb5b5d94251268ca /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure)
  test/test_rb.py::TestRBMultidim::test_rb_multidim_collector[env_device0-transform1-sampler_cls4-rbtype1-LazyTensorStorage-TensorDictRoundRobinWriter]
• Unit-tests on Windows / unittests-cpu / windows-job (gh) (trunk failure)
  test/test_collector.py::TestLibThreading::test_auto_num_threads

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

Thanks for this! MLFlow hasn't been used a lot so any bug fix of feature improvement is more than welcome!

[vmoens]

LGTM