Francesco Orabona

Publications

Tutorials

Ashok Cutkosky and I gave a tutorial on Parameter-free Online Optimization at ICML 2020.
Tatiana Tommasi and I gave a tutorial on Domain Adaptation and Transfer Learning at ECCV 2014.

Online Learning/Stochastic Optimization

K. Chen and F. Orabona. Generalized Implicit Follow-The-Regularized-Leader. ICML 2023 [PDF]
A. Cutkosky, H. Mehta, and F. Orabona. Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion. ICML 2023 [PDF]
K. Chen and F. Orabona. Implicit Interpretation of Importance Weight Aware Updates. Workshop on Duality Principles for Modern Machine Learning at ICML 2023 [PDF]
M. Crawshaw, M. Liu, F. Orabona, W. Zhang, and Z. Zhuang. Robustness to Unbounded Smoothness of Generalized SignSGD. NeurIPS 2022 [PDF]
Z. Zhuang, M. Liu, A. Cutkosky, and F. Orabona. Understanding AdamW through Proximal Methods and Scale-Freeness. TMLR 2022 [PDF]
K. Chen, J. Langford, and F. Orabona. Better Parameter-free Stochastic Optimization with ODE Updates for Coin-Betting. AAAI 2022 [PDF]
K. Chen, A. Cutkosky and F. Orabona. Implicit Parameter-free Online Learning with Truncated Linear Models. ALT 2022 [PDF]
X. Li, M. Liu, and F. Orabona. On the Last Iterate Convergence of Momentum Methods. ALT 2022 [PDF]
M. Liu and F. Orabona. On the Initialization for Convex-Concave Min-max Problems. ALT 2022 [PDF]
J. Negrea, B. Bilodeau, N. Campolongo, F. Orabona, and Daniel M. Roy. Minimax Optimal Quantile and Semi-Adversarial Regret via Root-Logarithmic Regularizers. NeurIPS 2021 [PDF]
X. Li, Z. Zhuang, and F. Orabona. A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance. ICML 2021 [PDF] [CODE]
G. Flaspohler, F. Orabona, J. Cohen, S. Mouatadid, M. Oprescu, P. Orenstein, and L. Mackey. Online Learning with Optimism and Delay. ICML 2021 [PDF] [CODE]
N. Cesa-Bianchi and F. Orabona. Online Learning Algorithms. Annual Review of Statistics and Its Application 2021 [PDF]
N. Campolongo and F. Orabona. Temporal Variability in Implicit Online Learning. NeurIPS 2020 [PDF]
X. Li and F. Orabona. A High Probability Analysis of Adaptive SGD with Momentum. Workshop on Beyond First Order Methods in ML Systems at ICML 2020 [PDF]
A. Cutkosky and F. Orabona. Momentum-Based Variance Reduction in Non-Convex SGD. NeurIPS 2019 [PDF]
K.-S. Jun and F. Orabona. Parameter-free Online Convex Optimization with Sub-Exponential Noise. COLT 2019 [PDF]

Conference version: K.-S. Jun and F. Orabona. Parameter-Free Locally Differentially Private Stochastic Subgradient Descent, In NeurIPS Workshop on Privacy in Machine Learning 2019 [PDF]

Z. Zhuang, A. Cutkosky, and F. Orabona. Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization. In ICML 2019 [PDF]
X. Li and F. Orabona. On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes. In AISTATS 2019 [PDF]
A. Cutkosky and F. Orabona. Black-Box Reductions for Parameter-free Online Learning in Banach Spaces. In COLT 2018 [PDF]
F. Orabona and D. Pal. Scale-free Online Learning, Theoretical Computer Science, 716. 2018 [PDF]

Conference version: F. Orabona, and D. Pal. Scale-Free Algorithms for Online Linear Optimization. In ALT 2015 [PDF]

F. Orabona and T. Tommasi. Training Deep Networks without Learning Rates Through Coin Betting, NeurIPS 2017 [PDF] [CODE]
K.-S. Jun, F. Orabona, R. Willett, S. Wright. Online Learning for Changing Environments using Coin Betting, Electronic Journal of Statistics [PDF]

Conference version: K.-S. Jun, F. Orabona, R. Willett, S. Wright. Improved Strongly Adaptive Online Learning using Coin Betting, In AISTATS 2017 [PDF]

F. Orabona and D. Pal. Coin Betting and Parameter-Free Online Learning. In NeurIPS 2016 [PDF]

Workshop version: F. Orabona, and D. Pal. Parameter-Free Convex Learning through Coin Betting. In AutoML 2016 Workshop at ICML 2016 [PDF] [Poster]

A. Gonen, F. Orabona, and S. Shalev-Shwartz. Solving Ridge Regression using Sketched Preconditioned SVRG. In ICML 2016 [PDF]
R. De Rosa, F. Orabona, and N. Cesa-Bianchi. The ABACOC Algorithm: a Novel Approach for Nonparametric Classification of Data Streams. In ICDM 2015 [PDF (long version)]
F. Orabona, K. Crammer, and N. Cesa-Bianchi. A Generalized Online Mirror Descent with Applications to Classification and Regression. Machine Learning Journal, pages 1-25, Dec, 2014 [PDF]
H. B. McMahan and F. Orabona. Unconstrained Online Linear Learning in Hilbert Spaces:
Minimax Algorithms and Normal Approximations. In Conference on Learning Theory (COLT), 2014 [PDF]
F. Orabona. Dimension-free Exponentiated Gradient. In NeurIPS 2013 [PDF (version with proofs)]
S. Kpotufe, and F. Orabona. Regression-tree Tuning in a Streaming Setting. NeurIPS 2013 [PDF (version with proofs)]
F. Orabona, Cesa-Bianchi, C. Gentile. Beyond Logarithmic Bounds in Online Learning. In AISTATS 2012 [PDF]
F. Orabona, L. Jie, and B. Caputo. Multi Kernel Learning with Online-Batch Optimization. In Journal of Machine Learning Research, 13(Feb):165-191, 2012 [PDF] [CODE]

Conference version: F. Orabona, L. Jie, and B. Caputo. Online-Batch Strongly Convex Multi Kernel Learning. CVPR 2010 [PDF] [TALK]

F. Orabona, and K. Crammer. New Adaptive Algorithms for Online Classification. NeurIPS 2011 [PDF]
F. Orabona, and L. Jie. Ultra-Fast Optimization Algorithm for Sparse Multi Kernel Learning. ICML 2011 [PDF] [CODE]
F. Orabona, C. Castellini, B. Caputo, L. Jie, and G. Sandini. On-line Independent Support Vector Machines. Pattern Recognition, 43(4), 1402-1412, 2010 [PDF] [CODE]

Conference version: F. Orabona, C. Castellini, B. Caputo, J. Luo, and G. Sandini. Indoor place recognition using online independent support vector machines. BMVC 2007 [PDF]

L. Jie, F. Orabona, M. Fornoni, B. Caputo, and N. Cesa-Bianchi. OM-2: An Online Multi-class Multi-kernel Learning Algorithm. In Proc. of the 4th IEEE Online Learning for Computer Vision Workshop (in CVPR10), San Francisco, CA, June 2010 [PDF (corrected)] [Errata] [CODE]
F. Orabona, J. Keshet, and B. Caputo. Bounded Kernel-Based Online Learning. Journal of Machine Learning Research, 10(Nov):2643-2666, 2009 [PDF] [CODE]

Conference version: F. Orabona, J. Keshet, and B. Caputo. The Projectron: a bounded kernel-based Perceptron. ICML July 2008 [PDF] [TALK]

M. M. Ullah, F. Orabona, B. Caputo. You Live, You Learn, You Forget: Continuous Learning of Visual Places with a Forgetting Mechanism. IROS 2009
L. Jie, F. Orabona, and B. Caputo. An online framework for learning novel concepts over multiple cues. ACCV 2009 [PDF] [CODE]

Deep Learning Theory

A. Meterez, A. Joudaki, F. Orabona, A. Immer, G. Rätsch, and H. Daneshmand. Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion. ICLR 2024 [PDF]

Statistical Learning Theory

I. Kuzborskij, K.-S. Jun, Y. Wu, K. Jang, and F. Orabona. Better-than-KL PAC-Bayes Bounds. COLT 2024 [PDF]
K.-S. Jun, A. Cutkosky, and F. Orabona. Kernel Truncated Randomized Ridge Regression: Optimal Rates and Low Noise Acceleration. NeurIPS 2019 [PDF]
F. Orabona. Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning. NeurIPS 2014 [PDF (version with proofs)] [CODE (linear version)]

Learning with Partial Information

A. Beygelzimer, F. Orabona, and C. Zhang. Efficient Online Bandit Multiclass Learning with \tilde{O}(\sqrt{T}) Regret. ICML 2017 [PDF]
C. Gentile and F. Orabona. On Multilabel Classification and Ranking with Bandit Feedback. In Journal of Machine Learning Research, 15(Jul):2451-2487, 2014 [PDF]

Conference version: C. Gentile and F. Orabona. On Multilabel Classification and Ranking with Partial Feedback. NeurIPS 2012 [PDF] [APPENDIX]

L. Jie, and F. Orabona. Learning from Candidate Labeling Sets. NeurIPS 2011 [PDF] [CODE]
F. Orabona and N. Cesa-Bianchi. Better Algorithms for Selective Sampling. ICML 2011 [PDF] [CODE] [TALK]
N. Cesa-Bianchi, C. Gentile and F. Orabona. Robust Bounds for Classification via Selective Sampling. ICML 2009 [PDF] [CODE] [TALK]

Transfer Learning

I. Kuzborskij and F. Orabona. Fast Rates by Transferring from Auxiliary Hypotheses, Machine Learning, 2016 [PDF]
I. Kuzborskij, F. Orabona, and B. Caputo. Scalable Greedy Algorithms for Transfer Learning. Computer Vision and Image Understanding, 156, 2016 [PDF]

Conference version: I. Kuzborskij, F. Orabona, and B. Caputo. Transfer Learning through Greedy Subset Selection. ICIAP 2015 (Best Paper Award) [PDF] [CODE]

T. Tommasi, F. Orabona, and B. Caputo. Learning Categories from Few Examples with Multi Model Knowledge Transfer. In IEEE Trans. on Pattern Analysis and Machine Intelligence, 36(5), 2014 [PDF]

Conference version: T. Tommasi, F. Orabona, and B. Caputo. Safety in Numbers: Learning Categories from Few Examples with Multi Model Knowledge Transfer. CVPR 2010 [PDF] [CODE]

I. Kuzborskij, and F. Orabona. Stability and Hypothesis Transfer Learning. ICML 2013 [PDF] [Errata]
I. Kuzborskij, F. Orabona, and B. Caputo. From N to N+1: Multiclass Transfer Incremental Learning. CVPR 2013 [PDF] [APPENDIX] [CODE]
T. Tommasi, F. Orabona, C. Castellini, and B. Caputo. Improving Control of Dexterous Hand Prostheses Using Adaptive Learning. IEEE Trans. on Robotics, vol.29, no.1, pp.207-219, Feb. 2013 [PDF]

Conference version: F. Orabona, C. Castellini, B. Caputo, A. E. Fiorilla, and G. Sandini. Model adaptation with least-squares SVM for adaptive hand prosthetics. ICRA 2009 [PDF]

T. Tommasi, F. Orabona, M. Kaboli, and B. Caputo. Leveraging over Prior Knowledge for Online Learning of Visual Categories. BMVC 2012 [PDF]

Locally Linear Learning

M. Fornoni, B. Caputo, and F. Orabona. Multiclass Latent Locally Linear Support Vector Machines. ACML 2013 [PDF]

Concentration Inequalities

F. Orabona and K.-S. Jun. Tight Concentrations and Confidence Sequences from the Regret of Universal Portfolio. IEEE Trans. on Information Theory, 2024 [PDF]
K. Jang, K.-S. Jun, I. Kuzborskij, and F. Orabona. Tighter PAC-Bayes Bounds Through Coin-Betting. COLT 2023 [PDF]
T. Hazan, F. Orabona, A. D. Sarwate, S. Maji, and T. Jaakkola. High Dimensional Inference with Random Maximum A-Posteriori Perturbations. IEEE Trans. on Information Theory, 2019 [PDF]

Conference version:

F. Orabona, T. Hazan, A. D. Sarwate, and T. Jaakkola. On Measure Concentration of Random Maximum A-Posteriori Perturbations. ICML 2014 [PDF (version with proofs)]

Computer Vision

E. Grossmann, J. A. Gaspar, and F. Orabona. Discrete camera calibration from pixel streams. Computer Vision and Image Understanding, special issue on Omnidirectional Vision, Camera Networks and Non-conventional Cameras, 114(2), 198-209, 2010 [PDF]

Conference versions:

E. Grossmann, J. A. Gaspar, and F. Orabona. Calibration from statistical properties of the visual world. ECCV 2008 [PDF]

E. Grossmann, F. Orabona, and J. Gaspar. Discrete camera calibration from the information distance between pixel streams. In Proc. of the 7th Workshop on Omnidirectional Vision (in ICCV07), Rio de Janeiro, Brazil, October 2007 [PDF]

M. Vurro, G. Baselli, F. Orabona, and G. Sandini. Simulation and assessment of bioinspired visual processing system for epi-retinal prostheses. In Proc. of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, New York City, USA, September 2006

Medical Image Classification

T. Tommasi and F. Orabona. Idiap on medical image classification. In W. B. Croft, H. Müller, P. Clough, T. Deselaers, and B. Caputo, editors, ImageCLEF, volume 32 of The Information Retrieval Series, pages 453-465. Springer Berlin Heidelberg, 2010
T. Tommasi, F. Orabona, and B. Caputo. An SVM confidence-based approach to medical image annotation. In Evaluating Systems for Multi-lingual and Multimodal Information Access, volume 5706 of Lecture Notes in Computer Science, 696-703, 2009 (Winner of the Medical automatic annotation task of ImageCLEF 2008)
T. Tommasi, F. Orabona, and B. Caputo. Discriminative cue integration for medical image annotation. Pattern Recognition Letters, 19(15):1996-2002, 2008 (Winner of the Medical automatic annotation task of ImageCLEF 2007)

Conference version: T. Tommasi, F. Orabona, and B. Caputo. Cue integration for medical image annotation. In Advances in Multilingual and Multimodal Information Retrieval, volume 5152 of Lecture Notes in Computer Science, pages 577-584, 2008

Robotics/Cognitive Systems

F. Orabona, B. Caputo, A. Fillbrandt, and F. W. Ohl. A Theoretical Framework for Transfer of Knowledge Across Modalities in Artificial and Biological Systems. In Proc. of the 8th International Conference on Development and Learning (ICDL), June 2009
J. Anemueller, J.-H. Bach, B. Caputo, L. Jie, F. Ohl, F. Orabona, R. Vogel, D. Weinshall, and A. Zweig. Biologically motivated audio-visual cue integration for object categorization. In Proc. of the International Conference on Cognitive Systems (ICCS), University of Karlsruhe, Germany, April 2008
F. Orabona, G. Metta, and G. Sandini. A proto-object based visual attention model. In L. Paletta and E. Rome, editors, Attention in Cognitive Systems, volume 4840 of Lecture Notes in Artificial Intelligence, pages 198-215, 2007 [PDF]

Conference versions:
F. Orabona, G. Metta, and G. Sandini. Learning association fields from natural images. In Proc. of the 5th Workshop on Perceptual Organization in Computer Vision (in CVPR06), Washington, DC, USA, June 2006. IEEE Computer Society [PDF]
F. Orabona, G. Metta, and G. Sandini. Object-based visual attention: a model for a behaving robot. In Proc. of the 3rd International Workshop on Attention and Performance in Computer Vision (WAPCV) (in CVPR05), Washington, DC, USA, June 2005. IEEE Computer Society (Best Paper Award) [PDF]

C. Castellini, F. Orabona, G. Metta, and G. Sandini. Internal models of reaching and grasping. Advanced Robotics, 21(13):1545-1564, 2007 [PDF]
L. Natale, F. Orabona, G. Metta, and G. Sandini. Sensorimotor coordination in a "baby" robot: learning about objects through grasping. In C. von Hofsten and K. Rosander, editors, Progress in Brain Research, From Action to Cognition, volume 164. Elsevier, 2007
A. Pasquali, C. Castellini, V. Gaillard, F. Orabona, G. Metta, and A. Cleeremans. Developmental learning in non-markovian processes: premises of a biologically plausible cognitive architecture. In 11th Annual Meeting of the Association for the Scientific Study of Consciousness, 2007 (only abstract)
L. Natale, F. Orabona, F. Berton, G. Metta, and G. Sandini. From sensorimotor development to object perception. In Proc. of the 5th IEEE-RAS International Conference on Humanoid Robots, December 2005 [PDF]
L. Natale, F. Orabona, G. Metta, and G. Sandini. Exploring the world through grasping: a developmental approach. In Proc. of the 6th IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA), June 2005 [PDF]

Open Problems

F. Orabona, D. Pal. Open Problem: Parameter-Free and Scale-Free Online Algorithms, Proc. of COLT 2016 [PDF]

Preprints and Technical Reports

F. Orabona. Normalized Gradients for All. 2023 [PDF]
F. Orabona and D. Pal. Parameter-free Stochastic Optimization of Variationally Coherent Functions. 2021 [PDF]
F. Orabona. A Modern Introduction to Online Learning. 2019 [PDF]
F. Orabona and D. Pal. Optimal Non-Asymptotic Lower Bound on the Minimax Regret of Learning with Expert Advice. 2015 [PDF]
F. Orabona. A Simple Expression for Mill's Ratio of the Student's t-Distribution. 2015 [PDF]
F. Orabona, A. Argyriou, N. Srebro. PRISMA: PRoximal Iterative SMoothing Algorithm. 2012 [PDF]