Research: Parameter-Free Machine Learning
I focus on a very simple but often ignored topic: how to make machine learning algorithms really automatic? How to remove the constant need for a human (PhD student?) to hand tune all the knobs of an algorithm till it finally works?
Most of the time you find theoretical papers that discuss about "optimal tuning", or applications papers presenting models with 5-6 parameters, and in both cases no procedure on how to tune these parameters is presented.
I try to go beyond the usual "grad student descent" approach of the practictioners and the "doubling trick" of the theoreticians. I design theoretical algorithms that are natively parameter-free and are also useful in the real world.
In the following there is a sample of my contributions in this line of research.
Parameter-Free Backpropagation
- F. Orabona and T. Tommasi. Training Deep Networks without Learning Rates Through Coin Betting, 2017 [PDF] [CODE]
Parameter-Free Convex Prediction and Optimization
- A. Cutkosky and F. Orabona. Black-Box Reductions for Parameter-free Online Learning in Banach Spaces, 2018 [PDF]
- F. Orabona and D. Pal. Coin Betting and Parameter-Free Online Learning. NIPS 2016 [PDF]
- F. Orabona, and D. Pal. Scale-Free Algorithms for Online Linear Optimization. ALT 2015 [PDF]
Parameter-Free Non-Parametric Classification and Regression
- F. Orabona. Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning. NIPS 2014 [PDF (version with proofs)]
- S. Kpotufe, and F. Orabona. Regression-tree Tuning in a Streaming Setting. NIPS 2013 [PDF (version with proofs)] [CODE (linear version)]
Parameter-Free Transfer Learning