[cs-talks] IVC Seminar, Ziming Zhang, Monday April 10th, 1pm @ MCS 148
Harrington, Jacob Walter
jwharrin at bu.edu
Fri Apr 7 11:10:28 EDT 2017
Training DNNs with Tikhonov Regularization
Ziming Zhang, Research Scientist, Mitsubishi Electric Research Laboratories (MERL)
Monday April 10th, 1pm – 2pm, MCS 148
Convergence and computational efficiency in training as well as testing are both very important issues in deep learning. Stochastic gradient descent (SGD) is the most widely-used technique in deep learning solvers in the literature. It has been shown recently that SGD has weak convergence (in probability) for non-convex optimization, but its convergence rate in such cases is unknown yet. In this work, we propose a novel training algorithm with Tikhonov regularization for (feed-forward) deep neural networks (DNNs), which can: (1) converge to stationary points deterministically (2) with sublinear convergence rate, ie. O(1/t), (3) and suitable for parallel computing as well as (4) learning not only dense but also sparse network architectures directly. During the talk, I will present some preliminary empirical results to demonstrate the ability of our proposed algorithm compared with SGD based solvers in Caffe.
Currently, Ziming is a Research Scientist at Mitsubishi Electric Research Laboratories (MERL). Before joining MERL, he was a research assistant professor at Boston University, MA. His research interest lies in computer vision and machine learning, including object recognition and detection, zero-shot learning, optimization, etc. His works have appeared in TPAMI, CVPR, ICCV, ECCV, ACM MM and NIPS. Personal website: https://zimingzhang.wordpress.com/.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the cs-talks