Wir laden herzlich zum Kolloquium von Prof. Arnulf Jentzen ein!
Am 3. Juli, von 10:00 Uhr bis 12:00 Uhr, wird Prof. Arnulf Jentzen, Professor an der Chinesischen Universität Hongkong (China) und der Universität Münster, einen Research Talk (in englischer Sprache) halten.
Der Research Talk widmet sich dem Thema „On stochastic optimization and the Adam optimizer: Divergence, convergence rates, and acceleration techniques“.
Abstract:
Stochastic gradient descent (SGD) optimization methods are nowadays the method of choice for the training of deep neural networks (DNNs) in artificial intelligence systems. In practically relevant training problems, often not the plain vanilla standard SGD method is the employed optimization scheme but instead suitably accelerated and adaptive SGD optimization methods such as the famous Adam optimizer are applied. In this talk we show that Adam does typically not converge to minimizers or criticial points of the objective function (the function one intends to minimize) but instead converges to zeros of another function, which we refer to as Adam vector field. Moreover, we establish convergence rates in terms of the number of Adam steps and the size of the mini-batch for all strongly convex stochastic optimization problems. Finally, we present acceleration techniques for Adam in the context of deep learning approximations for partial differential equation and optimal control problems. The talk is based on joint works with Steffen Dereich, Thang Do, Robin Graeber, and Adrian Riekert.
Die Veranstaltung ist kostenfrei. Interessierte sind herzlich eingeladen, teilzunehmen!