Suppose we have an image represented by a tensor X ∈ Rr c × ×3, with Xi,j,1 being the red intensity at row i and column j, Xi,j,2 giving the green intensity and Xi,j,3 giving the blue intensity.
Then the contrast of the entire image is given.
Global contrast normalization (GCN) aims to prevent images from having varying amounts of contrast by subtracting the mean from each image, then rescaling it so that the standard deviation across its pixels is equal to some constant s.
This approach is complicated by the fact that no scaling factor can change the contrast of a zero-contrast image (one whose pixels all have equal intensity).
Images with very low but non-zero contrast often have little information content.
Dividing by the true standard deviation usually accomplishes nothing more than amplifying sensor noise or compression artifacts in such cases.
This motivates introducing a small, positive regularization parameter λ to bias the estimate of the standard deviation.
Alternately, one can constrain the denominator to be at least . Given an input image X, GCN produces an output image X, defined such that
Datasets consisting of large images cropped to interesting objects are unlikely to contain any images with nearly constant intensity.
In these cases, it is safe to practically ignore the small denominator problem by setting λ = 0 and avoid division by 0 in extremely rare cases by setting to an extremely low value like 10−8.
This is the approach used by ( ) on the CIFAR-10 Goodfellow et al.
2013a dataset. Small images cropped randomly are more likely to have nearly constant intensity, making aggressive regularization more useful.
( ) used Coates et al. 2011 λ = 0 and = 10 on small, randomly selected patches drawn from CIFAR-10.