Due date: Tuesday, 17 24 September 2019
In class we have discussed many techniques for restoring images that have been degraded by noise and/or blur, from the “classical” averaging filters and Wiener filtering to more recent methods. In this assignment you will experiment with these techniques on some images with synthetic and real degradations.
Here are some test images you can use for your results (click for full size):
You may use built-in library routines for PSF construction, convolution, median filtering, and the Fourier transform.
Basic denoising
Add Gaussian noise to each test image to create a noticeably noisy image. Then explore the use of mean filters and median filters to reduce the noise: for each type of filter, try different filter sizes, compute the PSNR, and choose the filter size manually (or by some loop) to approximately maximize the PSNR. Then, do the same with salt-and-pepper noise instead of Gaussian noise. Report your observations, including the noisy and maximum-PSNR images and their corresponding PSNRs, and discuss what you can conclude about the effectiveness of the two filters in different cases.
Edge-preserving smoothing
Implement one of the edge-preserving smoothing methods discussed in class:
Demonstrate results on the same noisy images from the previous part.
Deblurring
Blur the clean image with a disk-shaped point spread function \(h(x,y)\), i.e. constant inside a disk and zero outside, and normalized so the coefficients sum to 1. Add Gaussian noise \(\eta(x,y)\) of some chosen variance. Compute the frequency-domain transfer function \(H(u,v)\) of the PSF using an FFT. Calculate by hand the expected noise spectrum \(|N(u,v)|\) using the orthogonality property (\(\sum |N(u,v)|^2 \propto \sum |\eta(x,y)|^2\)) and the known variance of the added noise, and verify that it agrees with the computed FFT of \(\eta\).
Then perform deblurring using the Wiener filter. For this, you will need to provide an estimate of the desired image’s spectrum \(|F(u,v)|\). Experiment with different choices: (i) use the ground-truth spectrum from the original image; (ii) use \(|F(u,v)| =\) const; (iii) use \(|F(u,v)| \propto (\sqrt{u^2+v^2})^\alpha\) for some manually chosen \(\alpha\) that best fits the ground-truth spectrum.
Real-world image restoration
Here are some real-world images suffering from unknown noise and blur. Your task is to estimate the noise distribution and the point spread function purely from the given image, and then perform deblurring to restore the image as well as possible. This may be quite challenging! Even if you have a great estimate of the PSF, you will probably not get an extremely clean and sharp image since the blur is quite large. Do as well as you can.
You are not required to have a perfectly automated solution. Some manual effort is expected, for example: estimating the noise variance by locating a uniform region or by trial and error; finding an image of a point light source to estimate the PSF; or manually choosing an appropriate PSF model (e.g. a uniform disk) and fitting its parameters to features in the image.
Show results on at least two of these images, or at least one image from these and another blurry image you have taken yourself. Discuss your process in detail in your report.
Extra credit
How can the edge-preserving property of modern denoising methods be incorporated in a deblurring method? One way is to replace the Laplacian criterion used in constrained least-squares filtering with one that penalizes noise but not sharp edges, such as the total variation (TV). This TV-regularized deconvolution approach results in a nonsmooth optimization problem, which can nonetheless be solved effectively with an appropriate algorithm such as ADMM. See Gordon Wetzstein’s slides and notes for more details.
Implement this or any other technique you can find for edge-preserving deblurring, and show the results on some challenging examples.
Make sure your code works on the General Computing Lab machines, which have Python2+OpenCV and Matlab installed. If you are programming in a compiled language, include a Makefile or script which compiles the code without user input. In any case, your code should make it easy to run any of the different tasks on any user-provided image.
Submit your code along with a report describing your work and any relevant mathematical derivations (e.g. calculating \(|N(u,v)|\) from \(\sigma_\eta^2\)) and implementation choices (e.g. estimating \(|F(u,v)|\)). Explain how you estimated the noise and point spread function in the real-world restoration task. Include images demonstrating the results of each task of the assignment. You will be graded on both your working code and the comprehensiveness of the report.