Orbit Regularization

25 November 2014

Renato Negrinho IT

We propose a general framework for regularization based on group-induced majorization. In this framework, a group is defined to act on the parameter space and an orbit is fixed; to control complexity, the model parameters are confined to the convex hull of this orbit (the orbitope). We recover several well-known regularizers as particular cases, and reveal a connection between the hyperoctahedral group and the recently proposed sorted l1-norm. We derive the properties a group must satisfy for being amenable to optimization with conditional and projected gradient algorithms. Finally, we suggest a continuation strategy for orbit exploration, presenting simulation results for the symmetric and hyperoctahedral groups.



Renato Negrinho received a M.Sc degree in Electrical and Computer Engineering from Instituto Superior Técnico, Portugal, in 2013. He currently holds a research scholarship and is working on natural language processing and machine learning problems under the supervision of André Martins at Priberam. He is interested in machine learning, optimization and the application of mathematics in general to solve difficult problems in science.