Approximation Theory

Compressive binary search

by Mark Davenport

Co-authored with E. Arias-Castro (Preprint, February 2012)

In this paper we consider the problem of locating a nonzero entry in a high-dimensional vector from possibly adaptive... more In this paper we consider the problem of locating a nonzero entry in a high-dimensional vector from possibly adaptive linear measurements. We consider a recursive bisection method which e dub the compressive binary search and show that it improves on what any nonadaptive method can achieve. We establish a non-asymptotic bound that applies to all methods, regardless of their computational complexity. Combined, these results show that the compressive binary search is within a double logarithmic factor of the optimal performance.

On the fundamental limits of adaptive sensing

by Mark Davenport

Co-authored with E. Arias-Castro and E.J. Candès (Preprint, November 2011)

Suppose we can sequentially acquire arbitrary linear measurements of an n-dimensional vector x resulting in the linear... more Suppose we can sequentially acquire arbitrary linear measurements of an n-dimensional vector x resulting in the linear model y = A x + z, where z represents measurement noise. If the signal is known to be sparse, one would expect the following folk theorem to be true: choosing an adaptive strategy which cleverly selects the next row of A based on what has been previously observed should do far better than a nonadaptive strategy which sets the rows of A ahead of time, thus not trying to learn anything about the signal in between observations. This paper shows that the folk theorem is false. We prove that the advantages offered by clever adaptive strategies and sophisticated estimation procedures--no matter how intractable--over classical compressed acquisition/recovery schemes are, in general, minimal.

On the stability and accuracy of least squares approximations

by Mark Davenport

Co-authored with A. Cohen and D. Leviatan (Preprint, November 2011)

We consider the problem of reconstructing an unknown function f on a domain X from samples of f at n randomly chosen... more We consider the problem of reconstructing an unknown function f on a domain X from samples of f at n randomly chosen points with respect to a given measure ρ. Given a sequence of linear spaces (V_m)m>0 with dim(V_m) = m <= n, we study the least squares approximations from the spaces V_m. It is well known that such approximations can be inaccurate when m is too close to n, even when the samples are noiseless. Our main result provides a criterion on m that describes the needed amount of regularization to ensure that the least squares method is stable and that its accuracy, measured in L2(X;ρ), is comparable to the best approximation error of f by elements from V_m. We illustrate this criterion for various approximation schemes, such as trigonometric polynomials, with ρ being the uniform measure, and algebraic polynomials, with ρ being either the uniform or Chebyshev measure. For such examples we also prove similar stability results using deterministic samples that are equispaced with respect to these measures.

Analysis of orthogonal matching pursuit using the restricted isometry property

by Mark Davenport

Co-authored with M.B. Wakin. (IEEE Trans. on Information Theory, 56(9) pp. 4395-4401, September 2010.)

Orthogonal matching pursuit (OMP) is the canonical greedy algorithm for sparse approximation. In this paper we... more Orthogonal matching pursuit (OMP) is the canonical greedy algorithm for sparse approximation. In this paper we demonstrate that the restricted isometry property (RIP) can be used for a very straightforward analysis of OMP. Our main conclusion is that the RIP of order K+1 (with isometry constant δ 1 / (3 K^(1/2))) is sufficient for OMP to exactly recover any K-sparse signal. The analysis relies on simple and intuitive observations about OMP and matrices which satisfy the RIP. For restricted classes of K-sparse signals (those that are highly compressible), a relaxed bound on the isometry constant is also established. A deeper understanding of OMP may benefit the analysis of greedy algorithms in general. To demonstrate this, we also briefly revisit the analysis of the regularized OMP (ROMP) algorithm.

A wideband compressive radio receiver

by Mark Davenport

Co-authored with S.R. Schnelle, J.P. Slavinsky, R.G. Baraniuk, M.B. Wakin, and P.T. Boufounos. (Proc. Military Communications Conference (MILCOM), San Jose, California, October 2010.)

Compressive sensing (CS) is an alternative to Shannon/Nyquist sampling for the acquisition of sparse or compressible... more Compressive sensing (CS) is an alternative to Shannon/Nyquist sampling for the acquisition of sparse or compressible signals. Instead of taking periodic samples, CS measures inner products with M random vectors, where M is much smaller than the number of Nyquist-rate samples. The implications of CS are promising for many applications and enable the design of new kinds of analog-to-digital converters, imaging systems, and sensor networks. In this paper, we propose and study a wideband compressive radio receiver (WCRR) architecture that can efficiently acquire and track FM and other narrowband signals that live within a wide frequency bandwidth. The receiver operates below the Nyquist rate and has much lower complexity than either a traditional sampling system or CS recovery system. Our methods differ from most standard approaches to the problem of CS recovery in that we do not assume that the signals of interest are confined to a discrete set of frequencies, and we do not rely on traditional recovery methods such as ℓ_1-minimization. Instead, we develop a simple detection system that identifies the support of the narrowband FM signals and then applies compressive filtering techniques based on discrete prolate spheroidal sequences to cancel interference and isolate the signals. Lastly, a compressive phase-locked loop (PLL) directly recovers the FM message signals.

Reconstruction and cancellation of sampled multiband signals using discrete prolate spheroidal sequences

by Mark Davenport

Co-authored with M.B. Wakin. (Proc. Workshop on Signal Processing with Adaptive Sparse Structured Representations (SPARS), Edinburgh, Scotland, June 2011.)

There remains a significant gap between the discrete, finite-dimensional compressive sensing (CS) framework and the... more There remains a significant gap between the discrete, finite-dimensional compressive sensing (CS) framework and the problem of acquiring a continuous-time signal. In this talk, we will discuss how sparse representations for multiband signals can be incorporated into the CS framework through the use of Discrete Prolate Spheroidal Sequences (DPSS’s). DPSS’s form a highly efficient basis for sampled bandlimited functions; by modulating and merging DPSS bases, one obtains a sparse representation for sampled multiband signals. We will discuss the use of DPSS bases for both signal recovery and the cancellation of strong narrowband interferers from compressive samples.

Dynamic range and compressive sensing acquisition receivers

by Mark Davenport

Co-authored with J.R. Treichler, J.N. Laska, and R.G. Baraniuk. (Proc. 7th U.S. / Australia Joint Workshop on Defense Applications of Signal Processing (DASP), Coolum, Australia, July 2011.)

Compressive sensing (CS) exploits the sparsity present in many signal environments to reduce the number of... more Compressive sensing (CS) exploits the sparsity present in many signal environments to reduce the number of measurements needed for digital acquisition and processing. We have previously introduced the concept and feasibility of using CS techniques to build a wideband signal acquisition systems. This paper extends that work to examine such a receiver’s performance as a function of several key design parameters. In particular we show that that the system noise figure is predictably degraded as the subsampling implicit in CS is made more aggressive. Conversely we show that the dynamic range of a CS-based system can be substantially improved as the subsampling factor grows. The ability to control these aspects of performance provides an engineer new degrees of freedom in the design of wideband acquisition systems. A specific practical example, a multi-collector emitter geolocation system, is included to illustrate that point.

The pros and cons of compressive sensing for wideband signal acquisition: Noise folding vs. dynamic range

by Mark Davenport

Co-authored wtih J.N. Laska, J.R. Treichler, and R.G. Baraniuk. (Preprint. April 2011)

Compressive sensing (CS) exploits the sparsity present in many common signals to reduce the number of measurements... more Compressive sensing (CS) exploits the sparsity present in many common signals to reduce the number of measurements needed for digital acquisition. With this reduction would come, in theory, commensurate reductions in the size, weight, power consumption, and/or monetary cost of both signal sensors and any associated communication links. This paper examines the use of CS in the design of a wideband radio receiver in a noisy environment. We formulate the problem statement for such a receiver and establish a reasonable set of requirements that a receiver should meet to be practically useful. We then evaluate the performance of a CS-based receiver in two ways: via a theoretical analysis of the expected performance, with a particular emphasis on noise and dynamic range, and via simulations that compare the CS receiver against the performance expected from a conventional implementation. On the one hand, we show that CS-based systems that aim to reduce the number of acquired measurements are somewhat sensitive to signal noise, exhibiting a 3dB SNR loss per octave of subsampling which parallels the classic noise-folding phenomenon. On the other hand, we demonstrate that since they sample at a lower rate, CS-based systems can potentially attain a significantly larger dynamic range. Hence, we conclude that while a CS-based system has inherent limitations that do impose some restrictions on its potential applications, it also has attributes that make it highly desirable in a number of important practical settings.

Democracy In action: Quantization, saturation, and compressive sensing

by Mark Davenport

Co-authored with J.N. Laska, P.T. Boufounos, and R.G. Baraniuk. (Applied and Computational Harmonic Analysis, 31(3) pp. 429-443, November 2011.)

Recent theoretical developments in the area of compressive sensing (CS) have the potential to significantly extend the... more Recent theoretical developments in the area of compressive sensing (CS) have the potential to significantly extend the capabilities of digital data acquisition systems such as analog-to-digital converters and digital imagers in certain applications. To date, most of the CS literature has been devoted to studying the recovery of sparse signals from a small number of linear measurements. In this paper, we study more practical CS systems where the measurements are quantized to a finite number of bits; in such systems some of the measurements typically saturate, causing significant nonlinearity and potentially unbounded errors. We develop two general approaches to sparse signal recovery in the face of saturation error. The first approach merely rejects saturated measurements; the second approach factors them into a conventional CS recovery algorithm via convex consistency constraints. To prove that both approaches are capable of stable signal recovery, we exploit the heretofore relatively unexplored property that many CS measurement systems are democratic, in that each measurement carries roughly the same amount of information about the signal being acquired. A series of computational experiments indicate that the signal acquisition error is minimized when a significant fraction of the CS measurements is allowed to saturate (10–30% in our experiments). This challenges the conventional wisdom of both conventional sampling and CS.

How well can we estimate a sparse vector?

by Mark Davenport

Co-authored with E.J. Candès. (Preprint. April 2011)

The estimation of a sparse vector in the linear model is a fundamental problem in signal processing, statistics, and... more The estimation of a sparse vector in the linear model is a fundamental problem in signal processing, statistics, and compressive sensing. This paper establishes a lower bound on the mean-squared error, which holds regardless of the sensing/design matrix being used and regardless of the estimation procedure. This lower bound very nearly matches the known upper bound one gets by taking a random projection of the sparse vector followed by an ℓ_1 estimation procedure such as the Dantzig selector. In this sense, compressive sensing techniques cannot essentially be improved.

Compressive sensing of analog signals using discrete prolate spheroidal sequences

by Mark Davenport

Co-authored with M.B. Wakin. (Preprint. September 2011)

Compressive sensing (CS) has recently emerged as a framework for efficiently capturing signals that are sparse or... more Compressive sensing (CS) has recently emerged as a framework for efficiently capturing signals that are sparse or compressible in an appropriate basis. While often motivated as an alternative to Nyquist-rate sampling, there remains a gap between the discrete, finite-dimensional CS framework and the problem of acquiring a continuous-time signal. In this paper, we attempt to bridge this gap by exploiting the Discrete Prolate Spheroidal Sequences (DPSS's), a collection of functions that trace back to the seminal work by Slepian, Landau, and Pollack on the effects of time-limiting and bandlimiting operations. DPSS's form a highly efficient basis for sampled bandlimited functions; by modulating and merging DPSS bases, we obtain a dictionary that offers high-quality sparse approximations for most sampled multiband signals. This multiband modulated DPSS dictionary can be readily incorporated into the CS framework. We provide theoretical guarantees and practical insight into the use of this dictionary for recovery of sampled multiband signals from compressive measurements.

Chromatic derivatives, chromatic expansions and associated spaces

by Aleksandar Ignjatovic

Approximation of unbounded functions with Linear Positive Operators

by RKS Rathore

Linear combinations of linear positive operators and generating relations in special functions

by RKS Rathore

Extended two-dimensional differential transform method and its application on nonlinear PDEs with proportional delay

by Reza Abazari

Int. J. Comput. Math, (2011), 88(8) (2011), 1749-1762.

In this work, we successfully extended two-dimensional differential transform method (DTM) and their reduced form... more In this work, we successfully extended two-dimensional differential transform method (DTM) and their reduced form (RDTM), by presenting and proving some theorems, to obtain the solution of partial differential equations (PDEs) with proportional delay in t and shrinking in x. Theorems are presented in the most general form to cover a wide range of PDEs, being linear or nonlinear and constant or variable coefficient. In order to show the power and robustness of the present methods and to illustrate the pertinent features of related theorems, some examples are presented.

Analyzing Postaliasing and Smoothing Effects of Non-separable Reconstruction Schemes on the BCC Lattice

by Balázs Domonkos

Prefiltered gradient reconstruction for volume rendering

by Balázs Domonkos

3D Frequency-Domain Analysis of Non-Separable Reconstruction Schemes by using Direct Volume Rendering

by Balázs Domonkos

DC-splines: Revisiting the trilinear interpolation on the bodycentered cubic lattice

by Balázs Domonkos