[Doc] Set action_spec instead of input_spec

[FrankTianTT]

Description

input_spec is protected and does not support with direct setting. Example code in CEM and MPPI documentation will cause error:

Traceback (most recent call last):
  File "/Users/frank/Projects/target-domain-fast-adaptation/tdfa/mpc/try.py", line 50, in <module>
    env = MyMBEnv(world_model)
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/envs/common.py", line 137, in __call__
    instance: EnvBase = super().__call__(*args, **kwargs)
  File "/Users/frank/Projects/target-domain-fast-adaptation/tdfa/mpc/try.py", line 19, in __init__
    self.input_spec = CompositeSpec(
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/envs/common.py", line 315, in __setattr__
    return super().__setattr__(key, value)
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1754, in __setattr__
    super().__setattr__(name, value)
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/envs/common.py", line 427, in input_spec
    raise RuntimeError("input_spec is protected.")
RuntimeError: input_spec is protected.

Let's set the action_spec and let the program decide the input_spec.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[vmoens]

Good catch!
The remaining field in input spec should be placed in state_spec which can be edited.

self.state_spec = CompositeSpec(...)

[FrankTianTT]

@vmoens I have corrected it, if we are on the same page:)

[FrankTianTT]

I have corrected it, if we are on the same page:)

@vmoens Sorry, I just checked and directly changing self.observation_spec to self.state_spec will cause new problems:

Traceback (most recent call last):
  File "/Users/frank/Projects/target-domain-fast-adaptation/tdfa/mpc/try2.py", line 54, in <module>
    planner = MPPIPlanner(
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/modules/planners/mppi.py", line 136, in __init__
    super().__init__(env=env, action_key=action_key)
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/modules/planners/common.py", line 37, in __init__
    in_keys = list(env.observation_spec.keys(True, True))
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/envs/common.py", line 1159, in observation_spec
    observation_spec = self.output_spec["full_observation_spec"]
  File "/Users/frank/opt/anaconda3/envs/tdfa/lib/python3.8/site-packages/torchrl/data/tensor_specs.py", line 3271, in __getitem__
    return self._specs[idx_unravel]
KeyError: 'full_observation_spec'

This seems to be because MPCPlannerBase expects an observation_spec that can normalize the input to the algorithm. In the example code of ModelBasedEnvBase, MyMBEnv initializes both observation_spec and state_spec:

...         self.observation_spec = CompositeSpec(
...             hidden_observation=UnboundedContinuousTensorSpec((4,))
...         )
...         self.state_spec = CompositeSpec(
...             hidden_observation=UnboundedContinuousTensorSpec((4,)),
...         )

Actually this caused me more doubts, are the state_spec and observation_spec above the same thing? What I mean is that ModelBasedEnvBase is a stateless environment, and its step should accept $s_t, a_t$ and return $s_{t+1}, r_{t+1}$. At this time, whether state_spec corresponds to $s_t$ and observation_spec corresponds to $s_{t+1}$?

More specifically, if I want to use ModelBasedEnvBase to learn an environment (a stateful environment, such as gym or dmc), should its observation_spec and state_spec both be set to the observation_spec of the environment being learned, In order to support both model learning and MPC at the same time?

[vmoens]

Thanks for working on this!