New IEEE Research Equips Gradient Descent with Angular Information to Boost DNN Training | Synced
An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect ...
Source: Synced | AI Technology & Industry Review
An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect in the optimization trajectory and speeds up convergence.