Abstract
The recent development of machine learning has provided new and challenging research problems in the areas of non-convex optimization and dynamical systems. We focus on two fundamental aspects of machine learning: training the network parameters and selecting the architecture of a neural network, where the techniques of differentiable and topological dynamical systems play an essential role. In the first part of the talk, we present a result that the meta first-order methods (gradient descent, mirror descent, manifold gradient descent, etc) avoid saddle points by introducing the global stability theory of dynamical systems. In the second part, we present a new result on the expressivity power of deep neural networks: the depth-width trade-offs for ReLU networks via Sharkovskii’s Theorem. The related research has a variety of applications in deep learning, approximation theory, partial differential equations, etc, and has connections to Hilbert’s 13th problem. As an open ended area in representation learning, there are a lot more open problems to be answered in the future.Time
2020-09-08 10:00 ~ 11:00Speaker
Xiao Wang, Singapore University of Technology and DesignRoom
Zoom ID: 91389575421; PW:123456
