Image Inpainting
Assignment 3 – Digital Image Processing (CSL783)

Abhinav Golas – 2003CS50466
S. Arun Nair – 2003CS50222

Introduction:

The motivation of developing such a system is to remove certain objects from the image while maintaining the structure and texture of the image. The paper we are attempting to implement is “Image Inpainting” by Bertalmio, Sapiro, Caselles, and Ballester. This paper attempts to interpolate and fill the regions on the basis of the structural information contained in the neighbourhood of the region specified.

Algorithm:

The basic procedure of the paper attempts to fill in the pixels of the image along the isophote lines, i.e. The lines of equal intensity. So features like lines, edges are of high value. The algorithm attempts to do this by filling the information along the normal to the current region boundary. The information is obtained from a smoothness operator, as we would like to the propagation to be smooth. This information is projected along the normal by taking a simple dot product. For the smoothness operator, they use the LaPlacian operator.

Mathematical basis:

The algorithm proceeds iteratively by using a Taylor series like expansion of the process:

Iⁿ⁺¹(i,j) = Iⁿ(i,j) + delta_t * Iⁿ_t(i,j) for all (i,j) belonging to region gamma

where deltat is the step value,
gamma is the region to be filled in
Iⁿ is the approximation of the result at the n^th step

The algorithm is said to have converged when Iⁿ⁺¹ = Iⁿ.

The paper describes the process to calculate the Iⁿ_t matrix. In addition, it also specifies that this iteration is to be interleaved with some iterations of anisotropic diffusion. Anisotropic diffusion uses the same iterative equation given above but with a different value of the Iⁿ_t matrix.

For anisotropic diffusion:

Iⁿ_t(x,y,t) = g_e(x,y) * K(x,y,t) * |delta_I(x,y,t)| for all (x,y) belonging to gamma^e

where g_e(x,y) = 0 for (x,y) on the boundary of delta_gamma^e, = 1 inside
gamma^e is a dilation with a ball of radius e
K is the euclidean curvature of the isophotes of I, the image

For the iterative filling process, the paper describes the equations for getting the Int matrix as well. However, since they are sufficiently complicated to write in this form, we refrain from writing those here.

Implementation details:

We implemented this paper in C++ using the OpenCV library for image processing. The anisotropic diffusion code is written in MATLAB because it is inherently numerically unstable, and thus requires proper coding. The MATLAB code was taken from an implementation on the web, as the paper does not provide sufficient details on the implementation and handling the instabilities.

Problems:

The algorithm is inherently unstable numerically, as it involves the normalisation of the gradient and Laplacian, which are prone to small values for majority of the image. We have tried our level best to reduce these artifacts, but have had very limited success.

Results:

Original image:

Mask:

Resulting process: