Seminars | OptML @ Lehigh

Every week, the OptML group gathers to update each other about their recent work and discuss various problems. In these one hour meetings, one student or faculty member gives a presentation, either to describe what they have learned in the past week or discuss the recent challenges they have been facing. The presentation is interactive with all members of the group participating, asking questions, and offering ideas. The group also occasionally invites researchers from industry or academia to present.

Seminars: Fall 2023 – present

Seminars: Fall 2021 – Spring 2023

During this time period, our seminars were listed on Twitter. Please refer to the LehighOptML handle on that service.

Seminars: Spring 2015 – Spring 2021

2021 Spring

04/20/2021	Tao Li	OptML Student
	Canonical Capsules: Unsupervised Capsules in Canonical Pose paper1 paper2
04/14/2021	Yutong Dai	OptML Student
	Adversarial Robustness and Model Compression paper1 paper2
04/07/2021	Liyuan Cao	OptML Student
	Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling paper
03/31/2021	Suyun Liu	OptML Student
	Stochastic Alternating Balance Fair k-means and Alternating Bi-Objective Gradient Descent
03/24/2021	Shihong Xie	OptML Guest
	Interpretable, Robust, and Fair Learning on Graphs
03/10/2021	Tommaso Giovannelli, Oumaima Sohab	OptML Student
	Ridge Functions: Exploiting Lower-dimensional Structure paper1 paper2
03/03/2021	Qi Wang	OptML Student
	Deep Neural Networks as Gaussian Processes paper1 paper2
02/24/2021	Minhan Li	OptML Student
	On Regularization and Active-set Methods with Complexity for Constrained Optimization paper
02/17/2021	Tommaso Giovannelli	Visiting Student
	Derivative-free Methods for Mixed-integer Nonsmooth Constrained Optimization Problems paper1 paper2
02/10/2021	Michael O’Neill	OptML PostDoc
	Analysis of the BFGS Method with Errors paper1 paper2
02/03/2021	Baoyu Zhou	OptML Student
	Constrained and Composite Optimization via Adaptive Sampling Methods paper1 paper2

2020 Fall

12/09/2020	Minhan Li	OptML Student
	A High Probability Analysis of Adaptive SGD with Momentum paper
12/02/2020	Zheng Shi	OptML Student
	Neural Network Pruning paper1 paper2 paper3 paper4
11/18/2020	Tao Li	OptML Student
	Adaptive Attention via A Visual Sentinel for Image Captioning paper1 paper2
11/04/2020	Liyuan Cao	OptML Student
	Cubic Regularization of Newton Method and its Global Performance paper
10/28/2020	Sergey Rusakov	OptML Student
	Transformers: Architecture and Results paper1 paper2
10/21/2020	Oumaima Sohab	OptML Student
	Estimate Noise in Derivative-Free Optimization paper1 paper2 paper3
10/14/2020	Suyun Liu	OptML Student
	Fair Clustering Algorithms paper1 paper2 paper3
10/07/2020	Yuan Zeng	OptML Guest
	Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Network paper
09/30/2020	Yutong Dai	OptML Student
	Active Set Identification paper
09/23/2020	Mertcan Yetkin	OptML Student
	Meta-learning: Basics and Recent Advancements slides
09/16/2020	Michael O’Neill	OptML PostDoc
	Worst Case Complexity paper1 paper2
09/09/2020	Tommaso Giovannelli	Visiting Student
	A Black-Box Optimization Approach for Emergency Department paper1 paper2 paper3 paper4
09/02/2020	Baoyu Zhou	OptML Student
	Nonsmooth BFGS Method paper1 paper2 paper3 paper4

2020 Spring

04/29/2020	Oumaima Sohab	OptML Student
	Direct Search Based on Probabilistic Descent paper
04/22/2020	Zheng Shi	OptML Student
	Residual Learning, Attention Mechanism and Multi-tasks Learning Networks slides
04/15/2020	Mohammad Pirhooshyaran	OptML Student
	Quantum Neuron paper
04/08/2020	Tao Li	OptML Student
	Can Deep Reinforcement Learning Improve Inventory Management? paper
04/01/2020	Suyun Liu	OptML Student
	A Review of Multi-Objective Optimization: Theory and Algorithms slides
03/25/2020	Minhan Li	OptML Student
	Stochastic Model-based Minimization of Weakly Convex functions paper
03/18/2020	Xin Shi	OptML Student
	A Survey of Recent Scalability Improvements for Semidefinite Programming paper
03/04/2020	Mertcan Yetkin	OptML Student
	Neural Architecture Search (NAS) Series 3 paper1 paper2
02/26/2020	Rusakov Sergey	OptML Student
	Neural Architecture Search (NAS) Series 2 paper1 paper2
02/19/2020	Majid Jahani	OptML Student
	Differentiable Neural ARchiTecture Search (DARTS) paper
02/12/2020	Baoyu Zhou	OptML Student
	Manifold Sampling for L1 Nonconvex Optimization paper
02/05/2020	Liyuan Cao	OptML Student
	Lagrange Quadratic Interpolation paper slides
01/29/2020	Haidong Gu	OptML Student
	A Decent Algorithm and a Homotopy Method for Solving Lasso Problem paper
01/22/2020	Rui Shi	OptML Student
	Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization paper

2019 Fall

11/13/2019	Xin Shi, Minhan Li	OptML Student
	Adversarial Machine Learning Algorithms slides
11/06/2019	Xin Shi, Minhan Li	OptML Student
	Introduction to Adversarial Machine Learning slides
10/30/2019	Suyun Liu	OptML Student
	Online Convex Optimization in the Bandit Setting slides
	Frank E. Curtis	OptML Faculty
	Systematic Insights on the Fisher Matrix and Comments on paper
10/16/2019	Rui Shi	OptML Student
	Recent papers regarding Online Learning paper1 paper2
	Majid Jahani, Baoyu Zhou	OptML Student
	Second-Order Methods for Deep Learning paper
10/09/2019	Tao Li	OptML Student
	First Order Methods for Online Convex Optimization slides
	Majid Jahani, Baoyu Zhou	OptML Student
	Second-Order Methods for Deep Learning paper
10/02/2019	Lili Song	OptML Student
	Introduction to Online Convex Optimization slides
	Majid Jahani, Baoyu Zhou	OptML Student
	Second-Order Methods for Deep Learning slides
09/25/2019	Rusakov Sergey	OptML Student
	Approaches to Solving Semantic Segmentation slides
09/18/2019	Mertcan Yetkin, Mohammad Pirhooshyaran	OptML Student
	Convolutional Neural Network (CNN): Basics and Recent Advancements slides
09/11/2019	Liyuan Cao, Haidong Gu	OptML Student
	Introduction to Object Detection slides
09/04/2019	Liyuan Cao, Haidong Gu	OptML Student
	Computer Vision Tutorial slides

2019 Spring

04/17/2019	Mertcan Yetkin	OptML Student
	Generative Adversarial Nets paper
04/10/2019	Suyun Liu	OptML Student
	Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization paper
04/03/2019	Albert Berahas	OptML PostDoc
	Analysis of the BFGS method with errors paper
03/20/2019	Minhan Li	OptML Student
	Decentralized Quasi-Newton Methods paper
03/06/2019	Liyuan Cao	OptML Student
	Random Gradient-Free Minimization of Convex Functions paper
02/20/2019	Baoyu Zhou	OptML Student
	A Newton-Based Method for Nonconvex Optimization with Fast Evasion of Saddle Points paper
02/13/2019	Rui Shi	OptML Student
	Theoretical results in Online Learning paper
02/06/2019	Majid Jahani	OptML Student
	Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice paper

2018

12/05/2018	Mohammad Pirhooshyaran	OptML Student
	Adaptive Cubic Regularization paper
11/14/2018	Liyuan Cao	OptML Student
	Natural Evolutionary Strategies paper
11/07/2018	Minhan Li	OptML Student
	Stochastic Quasi Newton Methods paper
10/31/2018	Frank E. Curtis	OptML Faculty
	Theory of BFGS paper
10/24/2018	Suyun Liu	OptML Student
	Complexity of gradient descent for multiobjective optimization paper
10/17/2018	Sarper Aydin	OptML Student
	A derivative-free trust-region algorithm for the optimization of functions smoothed via Gaussian convolution using adaptive multiple importance sampling paper
10/10/2018	Albert Berahas	OptML Postdoc
	Adaptive Sampling Strategies for Stochastic Optimization paper
09/26/2018	Baoyu Zhou	OptML Student
	How to Escape Saddle Points Efficiently paper
09/19/2018	Mertcan Yetkin	OptML Student
	ADMM and Accelerated ADMM as Continuous Dynamical Systems paper
09/05/2018	Rui Shi	OptML Student
	Introduction to Gradient Boosting Models book
05/02/2018	Quoc Tran-Dinh	visitor
	Generalized Self-Concordant Functions: A Recipe for Newton-Type Methods paper
04/18/2018	Mohammad Pirhooshyaran	OptML Student
	Stochastic cubic regularization for fast nonconvex optimization paper
04/11/2018	Nicolas Loizou	Visiting Student from University of Edinburgh
	Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods paper
04/04/2018	Rui Shi	OptML Student
	Stochastic Trust Region Algorithm
03/28/2018	Lam Nguyen	OptML Student
	Stability of stochastic gradient descent paper
03/08/2018	Majid Jahani	OptML Student
	Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy
02/28/2018	Majid Jahani	OptML Student
	Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy paper
02/21/2018	Katya Scheinberg	OptML Faculty
	Global convergence rate analysis of unconstrained optimization methods based on probabilistic models paper
02/14/2018	Courtney Paquette	OptML PostDoc Researcher
	Local search for nonsmooth and nonconvex problems paper
02/07/2018	Mohammadreza Samadi	OptML student
	Recent Updates on Subsampled Newton methods
01/31/2018	All members
	Group discussion in the new office

2017

12/03/2017	Mohammadreza Nazari	OptML student
	Policy gradients in application to reinforcement learning
11/17/2017	Albert Berahas	visitor
	A Multi-Batch L-BFGS Method for Machine Learning
11/10/2017	Lam Nguyen	OptML student
	When do stochastic gradient algorithms work well for training deep neural networks
11/03/2017	Francesco Orabona	visitor
	Coin Betting for Backprop without Learning Rates and More
10/20/2017	Chenxin Ma	OptML student
	Introduction to natural gradient descent II
10/13/2017	Rui Shi	OptML student
	Different measures on convergence results
10/06/2017	Chenxin Ma	OptML student
	Introduction to natural gradient descent I
09/22/2017	Courtney Paquette	visitor
	Acceleration for Gradient-Based Non-Convex Optimization
09/15/2017	Xi He	OptML student
	TensorFlow tutorial II
09/08/2017	Xi He	OptML student
	TensorFlow tutorial I
05/02/2017	Matt Menickelly	OptML student
	On graduated optimization for stochastic non-convex problems
04/18/2017	Majid Jahani	OptML student
	Finite-sum Composition Optimization via Variance Reduced Gradient Descent
04/11/2017	Mertcan Yetkin	OptML student
	SVM for solving optimal control
04/04/2017	Liyuan Cao	OptML student
	k-Components: A Method for Analyzing Data in Radial Shape
03/28/2017	Mohammadreza Samadi	OptML student
	Gradient Descent with Momentum
03/21/2017	Jie Liu	OptML student
	SARAH: part II
03/07/2017	Lam Nguyen	OptML student
	SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
02/28/2017	Hiva Ghanbari	OptML student
	A Globally Convergent Inexact Newton Method for Systems of Monotone Equations
02/21/2017	Chenxin Ma	OptML student
	Underestimated Sequence Part II
02/14/2017	Xi He	OptML student
	Sub-Sampled Exact and Inexact Newton Methods
02/07/2017	Wei Guo	OptML student
	Barzilai-Borwein Step Size for Stochastic Gradient Descent paper
02/01/2017	Majid Jahani	OptML student
	Underestimated Sequence
11/29/2016	Matt Menickelly	OptML student
	Sparsity-Constrained Gaussian Graphical Models for Anomaly Detection
11/22/2016	Jie Liu	OptML student
	Fast stochastic methods for nonsmooth nonconvex optimization paper
11/01/2016	Mohammadreza Samadi	OptML student
	Proof technique in non-convex optimization
10/25/2016	Rui Shi	OptML student
	Robust Stochastic Approximation Approach to Stochastic Programming
10/18/2016	N/A	OptML student
	Discussions with Mark Schmidt
10/04/2016	Kürşat Kemikli	OptML student
	A limited memory quasi-Newton trust-region method for box constrained optimizatio
09/27/2016	Hiva Ghanbari	OptML student
	A Universal Catalyst for First-Order Optimization paper
09/20/2016	Chenxin Ma	OptML student
	An optimal first order method based on optimal quadratic averaging paper
09/13/2016	Wei Guo	OptML student
	Handling Nonpositive Curvature in a Limited Memory Steepest Descent Method
08/30/2016	Xi He	OptML student
	Second order methods in training deep neural network
05/05/2016	Martin Takáč	OptML faculty
	Theano And GPU computing in Python: Code
03/31/2016	Mohammadreza Samadi	OptML student
	Efficient Trust Region Subproblem Solvers
03/24/2016	Chenxin Ma	OptML student
	Second Order Stochastic Optimization in Linear Time
03/10/2016	Matt Menickelly	OptML student
	Random Sampling in Stochastic Optimization
03/03/2016	Xi He	OptML student
	Intro to Deep Neural Networks
02/25/2016	Milad Siami	OptML guest
	System Performance Measures for Noisy Consensus Networks
02/18/2016	Jiawei Zhang	OptML guest
	Deep Neural Network Learning Artistic Style
02/11/2016	Milad Siami	OptML guest
	System Performance Measures for Noisy Consensus Networks
02/04/2016	Jie Liu	OptML student
	Hybrid Optimization Methods in AC Optimal Power Flow
01/28/2016	Frank E. Curtis	OptML faculty
	Theory of Stochastic Gradient Methods

2015

12/02/2015	Hiva Ghanbari	OptML student
12/02/2015	Mohammadreza Samadi	OptML student
11/25/2015	Xi He	OptML student
11/11/2015	Francesco Orabona	Yahoo! Labs
10/28/2015	Chenxin Ma	OptML student
10/21/2015	Jeffrey Larson	Argonne National Laboratory
10/21/2015	Matt Menickelly	OptML student
10/14/2015	Wei Guo	OptML student
10/07/2015	Jie Liu	OptML student
09/30/2015	Hiva Ghanbari	OptML student
09/30/2015	Mohammadreza Samadi	OptML student
09/16/2015	Rui Shi	OptML student
09/16/2015	Matt Menickelly	OptML student
09/09/2015	Xi He	OptML student
09/09/2015	Chenxin Ma	OptML student
09/02/2015	Wei Guo	OptML student
09/02/2015	Jie Liu	OptML student
08/26/2015	Hiva Ghanbari	OptML student
08/26/2015	Mohammadreza Samadi	OptML student
08/19/2015	Xi He	OptML student

Reading Seminars (2017 and earlier)

2017

Date and Location	Speaker	Paper or Topic
2017/09/06, 2:30pm, Mohler 121	Xi He	Escape of saddle points for non-convex optimization
2017/04/11, 2:00pm, Mohler 121	Xi He	Unified convergence analysis of stochastic momentum methods for convex and non-convex optimization
2017/04/04, 2:00pm, Mohler 121	Chenxin Ma	Memory and Communication Efficient Distributed Stochastic Optimization with Minibatch Prox
2017/02/01, 2:30pm, Mohler 121	Lam Nguyen	His new research with Jie Liu
2017/01/17, 3:15pm, Mohler 375	Lam Nguyen	Stochastic Variance Reduction for Nonconvex Optimization
2017/01/10, 3:15pm, Mohler 375	Xi He	Symmetry-Breaking Convergence Analysis of Certain Two-layered Neural Networks with ReLU nonlinearity

Fall 2016

Date and Location	Speaker	Paper or Topic
2016/12/20, 2:30pm, Mohler 375	Chenxin Ma	Deep Learning without Poor Local Minima
2016/10/04, 2:30pm, Mohler 375	Xi He	On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
2016/10/04, 2:30pm, Mohler 375	Chenxin Ma	Gradient Descent Learns Linear Dynamical Systems
2016/09/27, 2:30pm, Mohler 375	Jie Liu	Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method
2016/09/27, 2:30pm, Mohler 375	Xi He	Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints
2016/08/31, 9:00am, Mohler 375	Chenxin Ma	Worst-case Complexity of Cyclic Coordinate Descent: $O(n^2)$ Gap with Randomized Version

Spring 2016

Date and Location	Speaker	Paper or Topic
2016/02/16, 9:00am, Mohler 375	Chenxin Ma Xi He	Second Order Stochastic Optimization in Linear Time Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
2016/03/21, 9:00am, Mohler 375	Xi He Chenxin Ma	RSG: Beating SGD without Smoothness and/or Strong Convexity Gradient Descent Converges to Minimizers
2016/03/28, 9:00am, Mohler 375	Jie Liu	Stochastic subGradient Methods with Linear Convergence for Polyhedral Convex Optimization
2016/04/12, 9:00am, Mohler 375	Jie Liu Xi He	Efficiency of accelerated coordinate descent method on structured optimization problems Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
2016/04/21, 8:30am, Mohler 375	Jie Liu	Variance Reduction for Faster Non-Convex Optimization
2016/04/28, 9:00am, Mohler 375	Chenxin Ma	Distributed Stochastic Variance Reduced Gradient Methods and A Lower Bound for Communication Complexity
2016/05/10, 9:00am, Mohler 375	Xi He	Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex

Fall 2015

Date and Location	Speaker	Paper or Topic
2015/08/28, 4:11pm, Mohler 451	Xi He	Communication Efficient Distributed Optimization using an Approximate Newton-type Method by Ohad Shamir, Nathan Srebro and Tong Zhang
2015/09/04, 4:11pm, Mohler 451	Jie Liu	Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields by Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton and Anoop Sarkar
2015/09/11, 4:11pm, Mohler 451	Xi He Chenxin Ma	Accelerated Stochastic Gradient Descent for Minimizing Finite Sums by Atsushi Nitanda DiSCO: Distributed Optimization for Self-Concordant Empirical Loss by Yuchen Zhang, Lin Xiao
2015/09/18, 10:00pm, Mohler 375	Jie Liu	CRF – tutorial
2015/09/25, 4:11pm, Mohler 451	Xi He Chenxin Ma	A Coordinate Descent Primal-Dual Algorithm with Large Step Size and Possibly Non Separable Functions by Olivier Fercoq and Pascal Bianchi Partitioning Data on Features or Samples in Communication-Efficient Distributed Optimization?
2015/12/22, 12:00pm, Mohler 375	Chenxin Ma Jie Liu	Sparse Linear Programming via Primal and Dual Augmented Coordinate Descent Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems
2015/12/27, 12:00pm, Mohler 375	Jie Liu Chenxin Ma	Scaling Up Distributed Stochastic Gradient Descent Using Variance Reduction

Spring 2015

Date and Location	Speaker	Paper or Topic
2015/04/17, 3:00pm, Mohler 375	Martin Takáč	SDCA without Duality by Shai Shalev-Shwartz Complexity Bounds for Primal-dual Methods Minimizing the Model of Objective Function by Yurii Nesterov
2015/04/24, 3:00pm, Mohler 375	Xi He	PASSCoDe: Parallel ASynchronous Stochastic dual Co-ordinate Descent by Cho-Jui Hsieh, Hsiang-Fu Yu and Inderjit S. Dhillon
2015/05/01, 3:00pm, Mohler 375	Jie Liu	Iteration Complexity of Feasible Descent Methods for Convex Optimization by Po-Wei Wang and Chih-Jen Lin Linear convergence of first order methods under weak nondegeneracy assumptions for convex programming by I. Necoara