Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend version update #456

Merged
merged 7 commits into from
Jan 5, 2023
Merged

Backend version update #456

merged 7 commits into from
Jan 5, 2023

Conversation

maljoras
Copy link
Collaborator

@maljoras maljoras commented Dec 9, 2022

Related issues

closes #346, closes #291, closes #290

Description

Many updates and fixes in particular to the RPUCuda backend, including

  • Supporting analog tiles moving from GPU to CPU
  • non-blocking option for tile computation in CUDA
  • Input range learning for inference
  • bump pylint / mypy / pycodestyle versions
  • Added support for N-degree polynomial weight noise models for hardware-aware training for analog inference .
  • Reset method for tiles on updated device
  • Many more non-linearties for the analog mat-vec including multiple passes (positive / negative)
  • Hidden weight write noise fixes
  • Added a remap_weights for AnalogSequential and AnalogModuleBase level.
  • Improved memory management of CUDA tiles
  • PowStepReference and SoftBoundsReference device, with option to subtract symmetry point (see Tool/tutorial for computing the symmetry points of a given device and setting up a TransferCompound  #291)
  • Fitting utility (Tool for fitting device data to models in AIHWKIT #290)

Details

Input range learning

Learning of the input clipping bound.

  • Gradient depends on clipping of input as well as output
  • Decay
  • adjustable parameters in InputRangeParameters

Yields very good results (see example 06)
image

non blocking

  • CudaTiles can be made no blocking by setting tile.non_blocking = True

@kaoutar55
Copy link
Collaborator

Thanks @maljoras! can we merge this before the 0.7.0 release?

@maljoras maljoras merged commit 3b961d3 into IBM:master Jan 5, 2023
@maljoras maljoras deleted the backend-merge branch January 5, 2023 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants