Dealing with Blurring

Created

May 25, 2023

Created by

Neo Yin

Status

In Progress

What is blurring, technically speaking?

Technically speaking, the operation of blurring is the application of a

blurring kernel

via convolution to a reference image — to create a less sharp, and smoother look.

‣

A better question which I find very intriguing.

However, there is a different question of naturally occurring blurriness — perhaps due to an ill-focused camera, squinting your eyes, myopia, dirty camera lens, etc. Just why the very notion of

blurring kernel

is a good modelling choice for these naturally occurring image distortions, and what kernel should we choose and how we justify the choice of this kernel?

‣

How is the blurring of an image measured?

The less interesting question is when you are the one blurring the image. The choice of the kernel (how centred it is) dictates the amount of blurring. The ensuing methods are agnostic toward how the blurring has occurred.
Gradient-based metrics:

Frequency-based metrics such as the
Fourier Transform or Power Spectral Density (PSD)
of the image can reveal the energy distribution across different spatial frequencies. Blur typically reduces high-frequency energy, so metrics like the
Blur Metric (BLUR)
,
Spectral Slope (SS)
or
Mean Square Error in Frequency Domain (MSEF)
can quantify the blurriness. The idea here is that high-frequency spectrum can be a good measurement of sharpness. Analogous approach can be (and probably should be) applied using
wavelets
instead of
fourier transform
, since Fourier approximation of constant functions is very slow, and most natural images have a lot of constant areas.
If you have the original reference image, and the blurred image, you can use image quality metrics such as
Structural Similarity Index (SSIM)
to measure the distortion.
A general problem with these metrics to think about is just how standardized is the measurement across different images types and different kinds of objects in the image.
A great resource that I found:

www.csl.cornell.edu

‣

How to measure a computer vision model’s robustness toward blurring?

The immediate approach is to probe what is the minimal amount of sacrifice to image quality metrics, through blurring data augmentations, (between a reference image and blurred image) to cause a misclassification.

The best way to find blur-robustness metrics is to look at the blurring-robustness metrics and measurements used in papers that talk about techniques promoting blur-robustness.

‣

What are methods in deep computer vision that promote model robustness toward blurred and blurring images?

Data augmentation:

The simplest and most widely applicable method is to apply blurring to the training examples — either/both during the pre-training stage or the task-specific training stage.

Adversarial training:

We can train two networks at the same time. The first network is a image classifier, latent representation encoder, etc., and the second network is a adversary that tries to learn the optimal blurring method that tries to mess with the first network — for instance, by learning the best blurring kernel — regularized by fidelity between the blurred image and the original, to prevent
model collapse
.

Blind Deblurring:

Blind deblurring consists of feeding images through sharpening/deblurring algorithm, either fixed or learned, before feeding them through the classification model.

‣

We might not want to use these below because it sounds like they involve changing the architecture to specifically deal with blurring, and might be overkill for what we are interested in with much sacrifice to other things we care about such as predictive performance and latent space structures:

Blur-invariant architecture:
Blur-invariant Losses: