System Recollection – Fast Weights

Hinton and Platutt (1987)

These two suggested Fast, Slow, and Effective Weights. Fast Weights are weights that dissolve to zero. They are placed into the network to aid in de-blurring after it has been reorganized by a new set of data. Fast Weights are meant as a catalyst; they provide an initial memory jolt in hopes that the network will then be able to remember what values it held the last time. The Slow Weights take over thereafter. The following are the equations

The Effective Weights equation is the following: