Distributed iterative thresholding for ℓ0/ℓ1-regularized linear inverse problems

Chiara Ravazzi, Sophie Marie Fosson, Enrico Magli

IEEE Transactions on Information Theory, vol. 61, no. 4, pp. 2081-2100, April 2015.


The ℓ0/ℓ1-regularized least-squares approach is used to deal with linear inverse problems under sparsity constraints, which arise in mathematical and engineering fields. In particular, multi-agent models have recently emerged in this context to describe diverse kinds of networked systems, ranging from medical databases to wireless sensor networks. In this paper, we study methods for solving ℓ0/ℓ1-regularized least-squares problems in such multi-agent systems. We propose a novel class of distributed protocols based on iterative thresholding and input driven consensus techniques, which are well-suited to work in network when the communication to a central processing unit is not allowed. Estimation is performed by the agents themselves, which typically consist of devices with limited computational capabilities. This motivates us to develop low-complexity and low-memory algorithms that are feasible in real applications. Our main result is a rigorous proof of the convergence of these methods in regular networks. We introduce suitable distributed, regularized, least-squares functionals and we prove that our algorithms reach their minima, using results from dynamical systems theory. Furthermore, we propose numerical comparisons with the alternating direction method of multipliers and the distributed subgradient methods, in terms of performance, complexity, and memory usage. We conclude that our techniques are preferable for their good memory-accuracy tradeoff.

Additional material

This paper is available as open access and can be downloaded as PDF.

Click on an item to open a preview, then on (top-left) to download it.