Compressed Sensing

Sparse Signal Recovery in the Presence of Intra-Vector and Inter-Vector Correlation

by Zhilin Zhang

Bhaskar D. Rao, Zhilin Zhang, Yuzhe Jin

Invited review paper of 2012 International Conference on Signal Processing and Communications (SPCOM 2012)

This work discusses the problem of sparse signal recovery when there is correlation among the values of non-zero... more This work discusses the problem of sparse signal recovery when there is correlation among the values of non-zero entries. We examine intra-vector correlation in the context of the block sparse model and inter-vector correlation in the context of the multiple measurement vector model, as well as their combination. Algorithms based on the sparse Bayesian learning are presented and the benefits of incorporating correlation at the algorithm level are discussed. The impact of correlation on the limits of support recovery is also discussed highlighting the different impact intra-vector and inter-vector correlations have on such limits.

Low Energy Wireless Body-Area Networks for Fetal ECG Telemonitoring via the Framework of Block Sparse Bayesian Learning

by Zhilin Zhang

Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Bhaskar D. Rao
Submitted to IEEE Transaction on Biomedical Engineering, Feb. 2012

Code and data can be found in the first author's homepage: https://sites.google.com/site/researchbyzhang/bsbl

Fetal ECG (FECG) telemonitoring is an important branch in telemedicine. The design of a telemonitoring system via a... more Fetal ECG (FECG) telemonitoring is an important branch in telemedicine. The design of a telemonitoring system via a low-power wireless body-area network for ambulatory use is highly desirable. As an emerging technique, compressed sensing (CS) shows great promise in compressing data with low power consumption. However, due to some specific characteristics of FECG recordings such as non-sparsity and strong noise contamination, current CS algorithms generally fail in this application.

In this work we utilize the block sparse Bayesian learning (bSBL) framework, a recently developed framework solving the CS problems. To illustrate the ability of the bSBL methods, we apply it to two representative FECG datasets. In one dataset the fetal heartbeat signals are visible, while in the other dataset are barely visible. The experiment results show that the bSBL framework is capable of compressing FECG raw recordings and successfully reconstructing them. These successes rely on two unique features of the bSBL framework; one is the ability to reconstruct less-sparse but structured signals, and the other is the ability to learn and exploit correlation structure of signals to improve performance. These two abilities of the framework greatly enhance the potential use of bSBL in telemonitoring of other physiological signals.

Sparse Bayesian Multi-Task Learning for Predicting Cognitive Outcomes from Neuroimaging Measures in Alzheimer's Disease

by Zhilin Zhang

Jing Wan, Zhilin Zhang, Jingwen Yan, Taiyong Li, Bhaskar D. Rao, Shiaofen Fang, Sungeun Kim, Shannon Risacher, Andrew Saykin, Li Shen, to appear in CVPR 2012

Alzheimer’s disease (AD) is the most common form of dementia that causes progressive impairment of memory and other... more Alzheimer’s disease (AD) is the most common form of dementia that causes progressive impairment of memory and other cognitive functions. Multivariate regression models have been studied in AD for revealing relationships between neuroimaging measures and cognitive scores to understand how structural changes in brain can influence cognitive status. Existing regression methods, however, do not explicitly model dependence relation among multiple scores derived from a single cognitive test. It has been found that such dependence can deteriorate the performance of these methods. To overcome this limitation, we propose an efficient sparse Bayesian multi-task learning algorithm, which adaptively learns and exploits the dependence to achieve improved prediction performance. The proposed algorithm is applied to a real world neuroimaging study in AD to predict cognitive performance using MRI scans. The effectiveness of the proposed algorithm is demonstrated by its superior prediction performance over multiple state-of-the-art competing methods and accurate identification of compact sets of cognition-relevant imaging biomarkers that are consistent with prior knowledge.

Compressive binary search

by Mark Davenport

Co-authored with E. Arias-Castro (Preprint, February 2012)

In this paper we consider the problem of locating a nonzero entry in a high-dimensional vector from possibly adaptive... more In this paper we consider the problem of locating a nonzero entry in a high-dimensional vector from possibly adaptive linear measurements. We consider a recursive bisection method which e dub the compressive binary search and show that it improves on what any nonadaptive method can achieve. We establish a non-asymptotic bound that applies to all methods, regardless of their computational complexity. Combined, these results show that the compressive binary search is within a double logarithmic factor of the optimal performance.

Recovery of Block Sparse Signals Using the Framework of Block Sparse Bayesian Learning

by Zhilin Zhang

By Zhilin Zhang, Bhaskar D. Rao
Accepted by International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012)

In this paper we study the recovery of block sparse signals and extend conventional approaches in two important... more In this paper we study the recovery of block sparse signals and extend conventional approaches in two important directions; one is learning and exploiting intra-block correlation, and the other is generalizing signals’ block structure such that the block partition is not needed to be known for recovery. We propose two algorithms based on the framework of block sparse Bayesian learning (bSBL). One algorithm, directly derived from the framework, requires a priori knowledge of the block partition. Another algorithm, derived from an expanded bSBL framework using the generalization method, can be used when the block partition is unknown. Experiments show that they have superior performance to state-of-the-art algorithms.

Extension of SBL Algorithms for the Recovery of Block Sparse Signals with Intra-Block Correlation

by Zhilin Zhang

by Zhilin Zhang and Bhaskar D. Rao

http://dsp.ucsd.edu/~zhilin/BSBL.html

We examine the recovery of block sparse signals and extend the framework in two important directions; one by... more We examine the recovery of block sparse signals and extend the framework in two important directions; one by exploiting intra-block correlation and the other by generalizing the block structure. We propose two families of algorithms based on the framework of block sparse Bayesian learning (bSBL). One family, directly derived from the bSBL framework, requires knowledge of the block partition. Another family, derived from an expanded bSBL framework, is based on a weaker assumption about the a priori information of the block structure, and can be used in the cases when block partition, block size, block-sparsity are all unknown. Using these algorithms we show that exploiting intra-block correlation is very helpful to improve recovery performance. These algorithms also shed light on how to modify existing algorithms or design new ones to exploit such correlation for improved performance.

On the fundamental limits of adaptive sensing

by Mark Davenport

Co-authored with E. Arias-Castro and E.J. Candès (Preprint, November 2011)

Suppose we can sequentially acquire arbitrary linear measurements of an n-dimensional vector x resulting in the linear... more Suppose we can sequentially acquire arbitrary linear measurements of an n-dimensional vector x resulting in the linear model y = A x + z, where z represents measurement noise. If the signal is known to be sparse, one would expect the following folk theorem to be true: choosing an adaptive strategy which cleverly selects the next row of A based on what has been previously observed should do far better than a nonadaptive strategy which sets the rows of A ahead of time, thus not trying to learn anything about the signal in between observations. This paper shows that the folk theorem is false. We prove that the advantages offered by clever adaptive strategies and sophisticated estimation procedures--no matter how intractable--over classical compressed acquisition/recovery schemes are, in general, minimal.

Compressive Echelle spectroscopy

by Mark Davenport

Co-authored with L. Xu, M.A. Turner, T. Sun, and K.F. Kelly (Proc. Unconventional Imaging and Wavefront Sensing VII at SPIE Optics & Photonics, San Diego, California, August 2011.)

Building on the mathematical breakthroughs of compressive sensing (CS), we developed a 2D spectrometer system that... more Building on the mathematical breakthroughs of compressive sensing (CS), we developed a 2D spectrometer system that incorporates a spatial light modulator and a single detector. For some wavelengths outside the visible spectrum, when it is too expensive to produce the large detector arrays, this scheme gives us a better solution by using only one pixel. Combining this system with the “smashed filter” technique, we hope to create an efficient IR gas sensor. We performed Matlab simulations to evaluate the effectiveness of the smashed filter for gas tracing.

An l1 algorithm for underdetermined systems and applications

by Carlos Ramirez

In this work, we consider a homotopic principle for solving large-scale and dense ℓ1 underdetermined problems and its... more In this work, we consider a homotopic principle for solving large-scale and dense ℓ1 underdetermined problems and its applications. The idea consists of obtaining the solution of the problem by solving a sequence of linear equality constrained multiquadric problems that depends on a regularization parameter that converges to zero. The procedure generates a central path that converges to a point on the solution set of the ℓ1-underdetermined problem. This allows us to mimic the path-following methodology for primal-dual interior-point methods. We present a numerical experimentation showing the capability and effectiveness of our algorithm for recovering sparse signals, and its applications to MRI compressed sensing, seismic reflection and speech separation problems.

Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval

by Anoop Cherian

Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval

by Anoop Cherian

Clarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery

by Zhilin Zhang

by Zhilin Zhang, Bhaskar D. Rao
Technical Report, University of California, San Diego, September, 2011

Sparse Bayesian learning (SBL) is an important family of algorithms for sparse signal recovery and compressed sensing.... more Sparse Bayesian learning (SBL) is an important family of algorithms for sparse signal recovery and compressed sensing. It has shown superior recovery performance in challenging practical problems, such as highly underdetermined inverse problems, recovering signals with less sparsity, recovering signals based on highly coherent measuring/sensing/dictionary matrices, and recovering signals with rich structure. However, its advantages are smeared in current literature due to some misunderstandings on the parameters of SBL and incorrect parameter settings in algorithm comparison and practical use. This work clarifies some important issues, and serves as guidance for correctly using SBL.

Analysis of orthogonal matching pursuit using the restricted isometry property

by Mark Davenport

Co-authored with M.B. Wakin. (IEEE Trans. on Information Theory, 56(9) pp. 4395-4401, September 2010.)

Orthogonal matching pursuit (OMP) is the canonical greedy algorithm for sparse approximation. In this paper we... more Orthogonal matching pursuit (OMP) is the canonical greedy algorithm for sparse approximation. In this paper we demonstrate that the restricted isometry property (RIP) can be used for a very straightforward analysis of OMP. Our main conclusion is that the RIP of order K+1 (with isometry constant δ 1 / (3 K^(1/2))) is sufficient for OMP to exactly recover any K-sparse signal. The analysis relies on simple and intuitive observations about OMP and matrices which satisfy the RIP. For restricted classes of K-sparse signals (those that are highly compressible), a relaxed bound on the isometry constant is also established. A deeper understanding of OMP may benefit the analysis of greedy algorithms in general. To demonstrate this, we also briefly revisit the analysis of the regularized OMP (ROMP) algorithm.

Random observations on random observations: Sparse signal acquisition and processing

by Mark Davenport

Ph.D. Thesis, Rice University, August 2010. (Winner of 2011 Ralph Budd Award from Rice University for best thesis in the School of Engineering.)

In recent years, signal processing has come under mounting pressure to accommodate the increasingly high-dimensional... more In recent years, signal processing has come under mounting pressure to accommodate the increasingly high-dimensional raw data generated by modern sensing systems. Despite extraordinary advances in computational power, processing the
signals produced in application areas such as imaging, video, remote surveillance, spectroscopy, and genomic data analysis continues to pose a tremendous challenge. Fortunately, in many cases these high-dimensional signals contain relatively little information compared to their ambient dimensionality. For example, signals can often be well-approximated as a sparse linear combination of elements from a known basis or dictionary.

Traditionally, sparse models have been exploited only after acquisition, typically for tasks such as compression. Recently, however, the applications of sparsity have greatly expanded with the emergence of compressive sensing, a new approach to data acquisition that directly exploits sparsity in order to acquire analog signals more efficiently via a small set of more general, often randomized, linear measurements. If properly chosen, the number of measurements can be much smaller than the number of Nyquist-rate samples. A common theme in this research is the use of randomness in signal acquisition, inspiring the design of hardware systems that directly implement random measurement protocols.

This thesis builds on the field of compressive sensing and illustrates how sparsity can be exploited to design efficient signal processing algorithms at all stages of the information processing pipeline, with a particular focus on the manner in which randomness can be exploited to design new kinds of acquisition systems for sparse signals. Our key contributions include: (i) exploration and analysis of the appropriate properties for a sparse signal acquisition system; (ii) insight into the useful properties of random measurement schemes; (iii) analysis of an important family of algorithms for recovering sparse signals from random measurements; (iv) exploration of the impact of noise, both structured and unstructured, in the context of random measurements; and (v) algorithms that process random measurements to directly extract higher-level information or solve inference problems without resorting to full-scale signal recovery, reducing both the cost of signal acquisition and the complexity of the post-acquisition processing.

Signal processing with compressive measurements

by Mark Davenport

Co-authored with P.T. Boufounos, M.B. Wakin, and R.G. Baraniuk. (IEEE J. of Selected Topics in Signal Processing, 4(2) pp. 445-460, April 2010.)

The recently introduced theory of compressive sensing enables the recovery of sparse or compressible signals from a... more The recently introduced theory of compressive sensing enables the recovery of sparse or compressible signals from a small set of nonadaptive, linear measurements. If properly chosen, the number of measurements can be much smaller than the number of Nyquist-rate samples. Interestingly, it has been shown that random projections are a near-optimal measurement scheme. This has inspired the design of hardware systems that directly implement random measurement protocols. However, despite the intense focus of the community on signal recovery, many (if not most) signal processing problems do not require full signal recovery. In this paper, we take some first steps in the direction of solving inference problems-such as detection, classification, or estimation-and filtering problems using only compressive measurements and without ever reconstructing the signals involved. We provide theoretical bounds along with experimental results.

Texas Hold'Em algorithms for distributed compressive sensing

by Mark Davenport

Co-authored with S.R. Schnelle, J.N. Laska, C. Hegde, M.F. Duarte, and R.G. Baraniuk. (Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Dallas, Texas, March 2010.)

This paper develops a new class of algorithms for signal recovery in the distributed compressive sensing (DCS)... more This paper develops a new class of algorithms for signal recovery in the distributed compressive sensing (DCS) framework. DCS exploits both intra-signal and inter-signal correlations through the concept of joint sparsity to further reduce the number of measurements required for recovery. DCS is well-suited for sensor network applications due to its universality, computational asymmetry, tolerance to quantization and noise, and robustness to measurement loss. In this paper we propose recovery algorithms for the sparse common and innovation joint sparsity model. Our approach leads to a class of efficient algorithms, the Texas Hold ’Em algorithms, which are scalable both in terms of communication bandwidth and computational complexity.

A simple proof that random matrices are democratic

by Mark Davenport

Co-authored withJ.N. Laska, P.T. Boufounos, and R.G. Baraniuk. (Rice University ECE Technical Report TREE 0906, November 2009.)

The recently introduced theory of compressive sensing (CS) enables the reconstruction of sparse or compressible... more The recently introduced theory of compressive sensing (CS) enables the reconstruction of sparse or compressible signals from a small set of nonadaptive, linear measurements. If properly chosen, the number of measurements can be significantly smaller than the ambient dimension of the signal and yet preserve the significant signal information. Interestingly, it can be shown that random measurement schemes provide a near-optimal encoding in terms of the required number of measurements. In this report, we explore another relatively unexplored, though often alluded to, advantage of using random matrices to acquire CS measurements. Specifically, we show that random matrices are democratic, meaning that each measurement carries roughly the same amount of signal information. We demonstrate that by slightly increasing the number of measurements, the system is robust to the loss of a small number of arbitrary measurements. In addition, we draw connections to oversampling and demonstrate stability from the loss of significantly more measurements.

Exact signal recovery from sparsely corrupted measurements through the pursuit of justice

by Mark Davenport

Co-authored with J.N. Laska and R.G. Baraniuk. (Proc. 43rd Asilomar Conference on Signals, Systems and Computers, Pacific Grove, California, November 2009.)

Compressive sensing provides a framework for recovering sparse signals of length N from M << N measurements. If... more Compressive sensing provides a framework for recovering sparse signals of length N from M << N measurements. If the measurements contain noise bounded by ɛ, then standard algorithms recover sparse signals with error at most Cɛ. However, these algorithms perform suboptimally when the measurement noise is also sparse. This can occur in practice due to shot noise, malfunctioning hardware, transmission errors, or narrowband interference. We demonstrate that a simple algorithm, which we dub Justice Pursuit (JP), can achieve exact recovery from measurements corrupted with sparse noise. The algorithm handles unbounded errors, has no input parameters, and is easily implemented via standard recovery techniques.

Application of compressive sensing to the design of wideband signal acquisition receivers

by Mark Davenport

Co-authored with J.R. Treichler and R.G. Baraniuk. (Proc. 6th U.S. / Australia Joint Workshop on Defense Applications of Signal Processing (DASP), Lihue, Hawaii, September 2009.)

Compressive sensing (CS) exploits the sparsity present in many signals to reduce the number of measurements needed for... more Compressive sensing (CS) exploits the sparsity present in many signals to reduce the number of measurements needed for digital acquisition. With this reduction would come, in theory, commensurate reductions in the size, weight, power consumption, and/or monetary cost of both signal sensors and any associated communication links. This paper examines the use of CS in environments where the input signal takes the form of a sparse combination of narrowband signals of unknown frequencies that appear anywhere in a broad spectral band. We formulate the problem statement for such a receiver and establish a reasonable set of requirements that a receiver should meet to be practically useful. The performance of a CS receiver for this application is then evaluated in two ways: using the applicable (and still evolving) CS theory and using a set of computer simulations carefully constructed to compare the CS receiver against the performance expected from a conventional implementation. This sets the stage for work in a sequel that will use these results to produce comparisons of the size, weight, and power consumption of a CS receiver against an exemplar of a conventional design.

Compressive domain interference cancellation

by Mark Davenport

Co-authored with P.T. Boufounos and R.G. Baraniuk. (Proc. Workshop on Signal Processing with Adaptive Sparse Structured Representations (SPARS), Saint-Malo, France, April 2009.)

In this paper we consider the scenario where a compressive sensing system acquires a signal of interest corrupted by... more In this paper we consider the scenario where a compressive sensing system acquires a signal of interest corrupted by an interfering signal. Under mild sparsity and orthogonality conditions on the signal and interference, we demonstrate that it is possible to efficiently filter out the interference from the compressive measurements in a manner that preserves our ability to recover the signal of interest. Specifically, we develop a filtering method that nulls out the interference while maintaining the restricted isometry property (RIP) on the set of potential signals of interest. The construction operates completely in the compressive domain and has computational complexity that is polynomial in the number of measurements.