Sparse Data Representation: The Role of Redundancy in Data Processing

Book excerpt: The last decade has seen a dramatic increase in computational power and sensor ubiquity, as well as an ever increasing demand for finer resolution in both scientific and geometric modeling. This has led to the creation of enormously large data sets with exquisite detail. However, these data sets will be useful only if we can process them efficiently, whether it be for storage, transmission, visual display, fast on-line graphical query, correlation, or registration against data from other modalities. Raw data sets are typically inaccessible and need to be transformed to more efficient representations for further processing. Several competing issues emerge. Sparsity is essential for efficient transmission, storage, and computation. Multiscale representations are critical to extract features at desired scales. Implementation in silicon leads to new issues of robustness in the face of computational error and imprecise circuit implementation. An emerging technology to address these issues utilizes redundant representations. High oversampling followed by coarse quantization is the preferred method for analog to digital conversion of signals. Sparse representation of images using redundant families of waveforms is effectively utilized in feature extraction and denoising. These redundant families can be frames, dictionaries, or libraries of bases. On the other hand, there is, at present, no compelling theory to explain the advantages of redundancy in image and signal processing. This program will convene leading experts from data representation into two workshops to describe the current understanding of the benefits of redundancy and to set forward a program for further research.