With the advent of CHDK (Canon Hackers' Development Kit), many inexpensive point&shoot Canon cameras can be turned into versatile photographic instruments, with capabilities that far exceed other cameras in their price range, e.g bracketing, taking raw pictures, shooting indefinite time lapses.

When saving images in the DNG raw format (digital negative), CHDK also offers the possibility to remove bad pixels. However, these bad pixels seem to be particularly volatile. I have documented wide variations of number and position of bad pixels in a variety of situations. Although there is as yet no explanation for why bad pixels should vary in such a striking manner, determining how they vary might provide strategies for improving picture quality by removing them.

Hardware and Software

The camera used in this report is a Canon PowerShot SX120 IS, purchased in August 2010.
The version of CHDK used was as close to the currently compiled version as practicable, but had no discernible influence on the production of bad pixels.
The DNG version employed was 1.1. There have been reports of version 1.3 producing bad pixels which cannot be removed.
Raw DNG images were processed primarily by DCRAW in the current version. (UFRaw has also been used, but does not have the option of removing bad pixels. However, the same bad pixels turned up when this program was used.)

List of bad pixels produced by CHDK's bad pixel function

At various times, the badpixel.bin files have been saved separately and are available as follows:

Legend for comparisons: =: unchanged; +: new; -: removed.

Even this table shows a counterintuitive decrease in the number of bad pixels over time, which on its own requires an explanation.

Problems begin

Over time a number of images have come to my attention that show serious problems with bad pixels. Below are excerpts from pictures in an exposure series. The overexposed image, but also the underexposed image, show many more bad pixels than could possibly be contained in the small files from above:

DNG Images, extreme exposures of an exposure bracket
ISO 200, f 6.3, M mode
1/20 s, 2013/03/01 09:35:58	1/1250 s, 2013/03/01 09:35:40

These two pictures are part of an exposure bracket with 5 pictures of intermediate exposure intervening. Once the bracket is set going the camera is not moved or adjusted in any way.

However, the camera's processing of its raw images show that the bad pixels have been successfully compensated for (images offset by the displacement between JPG and DNG images):

JPG Images, extreme exposures of an exposure bracket
Images correspond to those above

A method for determining bad pixels in exposure bracket series

The real extent of the variability of bad pixel distribution only comes to light when many such exposure bracket series can be compared with one another.

A set of 36 exposure bracket series, each series consisting of 7 images at an exposure range of 6 EV and taken within a two month period (February-March 2013), was examined for bad pixels. Raw images were processed by DCRAW with no .badpixel file present in the current or lower directories, and wavelet denoising set at 0. A bad pixel was judged to be any pixel in the first of the series (underexposed image, -3 EV) that had exactly the same R, G and B values as the pixel in the same position in the last of the series (overexposed image, +3 EV). The number of bad pixels in an image pair varied widely:

Also quite extraordinary is the reliability at which any one pixel appeared to be bad. The next table shows how many pixels appeared to be bad in how many pairs of images:

Among the unusual phenomena in this table are:

1. A total of about 143,000 pixels at some time or another were deemed to be bad pixels. This represents about 1.2% of all the pixels in the sensor (12 MB, 3652 x 2758 pixels for DNG images);
2. There is a solitary peak of 940 bad pixels which appear in just over a third of the image pairs;
3. This peak appears to have no or very little error associated with it;
4. One particular bad pixel which was already immediately obvious in many of the images shot, whether brackets or not, was the Bright red pixel, which was found to be bad in 35 of 36 pairs;
5. No pixel was bad in all 36 pairs examined.

Sudden Appearance of Bad Pixels in Time-Lapse Series

It has been found by examining time-lapse series carefully, that whole sets of bad pixels turn up spontaneously during the course of the time-lapse. The following excerpt of four shots taking during a sunset series shows ordinary sensor noise in the first two shots, followed by bad pixels quite distinguishable from the noise in the shot immediately following. Image excerpts are from the areas identical to those in the previous images. Interestingly, these appear initially to be almost identical to the bad pixels in those images. After a few minutes, however, picture quality deteriorates further as new bad pixels suddenly appear (and remain until the end of the series).

Stepwise appearance of bad pixels
All images taken at ISO 80, f 2.8 in P mode
1/30 s, 2013/02/28 20:22:48	1/30 s, 2013/02/28 20:23:18	1/25 s, 2013/02/28 20:23:48	1/6 s, 2013/02/28 20:30:18

Questions

This poses a number of questions that I hope to find answers to, among others:

• How does CHDK determine what pixels are bad?
• Are the algorithms that CHDK uses to convert the really raw data (bits and bytes) into DNG files sufficiently accurate?
• Has anyone noticed this happening on their camera as well? Can you provide further examples?
• What correlations are there between the pixels determined to be bad by CHDK and those determined to be bad by the method described above?
• What correlations exist between bad pixels determined by the above method in different image pairs?

Note on this work in progress

This file will be updated as new information is obtained. If you have any information to contribute, or questions about data that might be useful in understanding these issues, please contact me here. A forum for discussing questions around this issue can be found here.