Number formats and bugfixes

[maljoras]

Related issues

Make ready for RPU tiles with different number format (fp16 / double).

Description

The C++ code and python code is adapted to use different simulation number format. For that a new runtime.data_type field is added to the RPUConfig. Once constructed the tile can be used to run the specified input data types.

However: full FP16 support is currently experimental (turned off by default) and only available for CUDA >=12.

There is also an option to store the parameter of pulsed devices into FP16 which reduces the memory for analog training.

Additional, an C++ aihwkit extension module for additional C++ code is added (optionally). Currently, the extension module only has a fake FP32 format conversion operator (for variable mantissa / exponent with saturation and rounding).

All options are experimental and turned off by default and need to be turned off when compiling.

Additional bugfixes and changes:

• convert to digital of an analog model added
• bugfixes / feature additions to the specialized in-memory learning rules
• AnalogWrapper improvements (now a direct subclass)
• some reshuffling of the configs/ parameter files to reduce python file size
• RNN analog linear layer more flexible
• custom tiles improvements
• RPUConfig base classes changes

[kaoutar55]

Thanks @maljoras