IRNN: Iteratively Reweighted Nuclear Norm for Nonconvex Nonsmooth Low-rank Minimization

Introduction

The nuclear norm is widely used as a convex surrogate of the rank function in compressive sensing for low rank matrix recovery with its applications in image recovery and signal processing. However, solving the nuclear norm based relaxed convex problem usually leads to a suboptimal solution of the original rank minimization problem. In this paper, we propose to use a family of nonconvex surrogates of -norm on the singular values of a matrix to approximate the rank function. This leads to a nonconvex nonsmooth minimization problem. Then we propose to solve the problem by Iteratively Reweighted Nuclear Norm (IRNN) algorithm. IRNN iteratively solves a Weighted Singular Value Thresholding (WSVT) problem, which has a closed form solution due to the special properties of the nonconvex surrogate functions. We also extend IRNN to solve the nonconvex problem with two or more blocks of variables. In theory, we prove that IRNN decreases the objective function value monotonically, and any limit point is a stationary point. Extensive experiments on both synthesized data and real images demonstrate that IRNN enhances the low rank matrix recovery compared with state-of- the-art convex algorithms.

IRNN solves the following nonconvex nonsmooth low-rank minimization problem

where is continuous, concave and monotonically increasing on , and has Lipschitz continuous gradient.

Our work [1] is the first one which considers such a general nonconvex nonsmooth low-rank minimization problem. It is further extended to the journal version [2]. We also propose a faster solver by using generalized singular value thresholding in [3] for solving the same problem.

The nonconvex function is the nonconvex surrogate of the -norm. Some known examples are given in Table 1 and Figure 1 below. By applying on the singular values, becomes to the nonconvex surrogate function of matrix rank. See the manifolds of some nonconvex surrogates in Figure 2 below.

References

1. Generalized Nonconvex Nonsmooth Low-Rank Minimization, Canyi Lu, Jinhui Tang, Shuicheng Yan, Zhouchen Lin, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2014: 4130-4137
2. Nonconvex Nonsmooth Low-Rank Minimization via Iteratively Reweighted Nuclear Norm, Canyi Lu, Jinhui Tang, Shuicheng Yan, Zhouchen Lin, IEEE Transactions on Image Processing (TIP). 2016
3. Generalized Singular Value Thresholding, Canyi Lu, Changbo Zhu, Chunyan Xu, Shuicheng Yan and Zhouchen Lin, AAAI Conference on Artificial Intelligence (AAAI), 2015