We cordially invite you to the colloquium of Prof. Arnulf Jentzen!
On July 3, 10:00 to 12:00 p.m., Prof. Arnulf Jentzen, Professor at the Chinese University of Hong Kong (China) and at the University of Münster, will give a research talk in English.
The research talk is dedicated to the topic “On stochastic optimization and the Adam optimizer: Divergence, convergence rates, and acceleration techniques“.
Abstract:
Stochastic gradient descent (SGD) optimization methods are nowadays the method of choice for the training of deep neural networks (DNNs) in artificial intelligence systems. In practically relevant training problems, often not the plain vanilla standard SGD method is the employed optimization scheme but instead suitably accelerated and adaptive SGD optimization methods such as the famous Adam optimizer are applied. In this talk we show that Adam does typically not converge to minimizers or criticial points of the objective function (the function one intends to minimize) but instead converges to zeros of another function, which we refer to as Adam vector field. Moreover, we establish convergence rates in terms of the number of Adam steps and the size of the mini-batch for all strongly convex stochastic optimization problems. Finally, we present acceleration techniques for Adam in the context of deep learning approximations for partial differential equation and optimal control problems. The talk is based on joint works with Steffen Dereich, Thang Do, Robin Graeber, and Adrian Riekert.
The event is free of charge. Interested individuals are warmly invited to attend!