Add generic analog optimizer

[diego-plan9]

Related issues

#48

Description

Add a mechanism for "enhancing" a regular Optimizer for using it in an analog context (customizing the optimization step for triggering the analog update needed for analog tiles), as aihwkit.optim.AnalogOptimizer:

from torch.optim import SGD
from torch.nn import Linear

from aihwkit.simulator.configs import InferenceRPUConfig
from aihwkit.optim import AnalogOptimizer


model = AnalogLinear(3, 4, rpu_config=InferenceRPUConfig)
optimizer = AnalogOptimizer(SGD, model.parameters(), lr=0.02)

The new optimizer behaves like the regular optimizer (which is passed as the first parameter to the constructor).

Details

As part of the changes:

• AnalogContext has been moved to aihwkit.optim
• AnalogSGD has been redefined in the terms of the newly introduced constructs, still keeping the original class in the package for convenience.

Kudos and thanks to @maljoras for the joint collaboration on this PR!