MATRIX VISION - mvBlueCOUGAR-X/-XD Technical Documentation
Correcting image errors of a sensor

Due to random process deviations, technical limitations of the sensors, etc. there are different reasons that image sensors have image errors. MATRIX VISION provides several procedures to correct these errors, by default these are host-based calculations.

However the mvBlueCOUGAR-X for example also supports a camera-based Flat-Field Correction, which saves dozens of % CPU load and lowers latency.

Provided image corrections procedures are

  1. Defective Pixels Correction,
  2. Dark Current Correction, and
  3. Flat-Field Correction.
Note
If you execute all correction procedures, you have to keep this order. All gray value settings of the corrections below assume an 8-bit image.
Figure 1: Host-based image corrections

The path "Setting -> Base -> ImageProcessing -> ..." indicates that these corrections are host-based corrections.

Before starting consider the following hints:

  • To correct the complete image, you have to make sure no user defined AOI has been selected: Right-click "Restore Default" on the devices AOI parameters Width and Height or "Setting -> Base -> Camera -> GenICam -> Image Format Control" using the GenICam interface layout.
  • You have several options to save the correction data. The chapter Storing and restoring settings describes the different ways.
See also
There is a white paper about image error corrections with extended information available on our website: http://www.matrix-vision.com/files/mv11/Glossary/art_image_errors_sensors_en.pdf

Defective Pixels Correction

Due to random process deviations, not all pixels in an image sensor array will react in the same way to a given light condition. These variations are known as blemishes or defective pixels.

There are three types of defective pixels:

  1. leaky pixel (in the dark)
    which indicates pixels that produce a higher read out code than the average
  2. hot pixel (in standard light conditions)
    which indicates pixels that produce a higher non-proportional read out code when temperatures are rising
  3. cold pixel (in standard light conditions)
    which indicates pixels that produce a lower read out code than average when the sensor is exposed (e.g. caused by dust particles on the sensor)
Note
Please use either an Mono or RAW Bayer image format when detecting defective pixel data in the image.

To correct the defective pixels various substitution methods exist:

  1. "Replace 3x1 average"
    which substitutes the detected defective pixels with the average value from the left and right neighboring pixel (3x1)
  2. "Replace 3x3 median"
    which substitutes the detected defective pixels with the median value calculated from the nearest neighboring in a 3 by 3 region
  3. "Replace 3x3 Filtered Data Averaged"
    which substitutes and treats the detected defective pixels as if they have been processed with a 3 by 3 filter algorithm before reaching this filter
    Only recommended for devices which do not offer a defective pixel compensation; packed RGB or packed YUV444 data is needed. See enumeration value dpfmReplaceDefectivePixelAfter3x3Filter in the corresponding API manual for additional details about this algorithm and when and why it is needed

Correcting leaky pixels

To correct leaky pixels the following steps are necessary:

  1. Set gain ("Setting -> Base -> Camera -> GenICam -> Analog Control -> Gain = 0 dB") and exposure time "Setting -> Base -> Camera -> GenICam -> Acquisition Control -> ExposureTime = 360 msec" to the given operating conditions
    The total number of defective pixels found in the array depend on the gain and the exposure time.
  2. Black out the lens completely
  3. Set the (Filter-) "Mode = Calibrate leaky pixel"
  4. Snap an image (e.g. by pressing Acquire in wxPropView with "Acquisition Mode = SingleFrame")
  5. To activate the correction, choose one of the substitution methods mentioned above
  6. Save the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"
    (Settings can be saved in the Windows registry or in a file)
Note
After having re-started the camera you have to reload the capture settings!

The filter checks:

Pixel > LeakyPixelDeviation_ADCLimit // (default value: 50)

All pixels above this value are considered as leaky pixel.

Correcting hot pixels

Note
With "Mode = Calibrate Hot And Cold Pixel" you can execute both corrections at the same time.

To correct hot pixels the following steps are necessary:

  1. You will need a uniform sensor illumination approx. 50 - 70 % saturation (which means an average gray value between 128 and 180)
  2. Set the (Filter-) "Mode = Calibrate Hot Pixel"
  3. Snap an image (e.g. by pressing Acquire in wxPropView with "Acquisition Mode = SingleFrame")
  4. To activate the correction, choose one of the substitution methods mentioned above
  5. Save the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"
    (Settings can be saved in the Windows registry or in a file)
Note
After having re-started the camera you have to reload the capture settings!

The filter checks:

Pixel > T[hot] // (default value: 15 %)

// T[hot] = deviation of the average gray value

Correcting cold pixels

Note
With "Mode = Calibrate Hot And Cold Pixel" you can execute both corrections at the same time.

To correct cold pixels the following steps are necessary:

  1. You will need a uniform sensor illumination approx. 50 - 70 % saturation (which means an average gray value between 128 and 180)
  2. Set the (Filter-) "Mode = Calibrate cold pixel" (Figure 2)
  3. Snap an image (e.g. by pressing Acquire in wxPropView with "Acquisition Mode = SingleFrame")
  4. To activate the correction, choose one of the substitution methods mentioned above
  5. Save the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"
    (Settings can be saved in the Windows registry or in a file)
Note
After having re-started the camera you have to reload the capture settings!

The filter checks:

Pixel < T[cold] // (default value: 15 %)

// T[cold] = deviation of the average gray value

All pixels below this value have a dynamic below normal behavior.

Figure 2: Image corrections: DefectivePixelsFilter
Note
Repeating the defective pixel corrections will accumulate the correction data which leads to a higher value in "DefectivePixelsFound". If you want to reset the correction data or repeat the correction process you have to set the filter mode to "Reset Calibration Data". In oder to limit the amount of defective pixels detected the "DefectivePixelsMaxDetectionCount" property can be used.

Storing pixel data on the device

To save and load the defective pixel data, appropriate functions are available:

  • int mvDefectivePixelDataLoad( void )
  • int mvDefectivePixelDataSave( void )

The section "Setting -> Base -> ImageProcessing -> DefectivePixelsFilter" was also extended (see Figure 2a). First, the DefectivePixelsFound indicates the number of found defective pixels. The coordinates are available through the properties DefectivePixelOffsetX and DefectivePixelOffsetY now. In addition to that it is possible to edit, add and delete these values manually (via right-click on the "DefectivePixelOffset" and select "Append Value" or "Delete Last Value"). Second, with the function

  • int mvDefectivePixelReadFromDevice( void )
  • int mvDefectivePixelWriteToDevice( void )

you can exchange the data from the filter with the camera and vice versa

Figure 2a: Image corrections: DefectivePixelsFilter (since driver version 2.17.1 and firmware version 2.12.406)

Just right-click on mvDefectivePixelWriteToDevice and click on "Execute" to write the data to the camera (and hand over the data to the Storing pixel data on the device). To permanently store the data inside the cameras non-volatile memory afterwards mvDefectivePixelDataSave must be called as well!

Figure 2b: Defective pixel data are written to the camera (since driver version 2.17.1 and firmware version 2.12.406)

While opening the camera, the camera will load the defective pixel data from the camera. If there are pixels in the filter available (via calibration), nevertheless you can load the values from the camera. In this case the values will be merged with the existing ones. I.e., new ones are added and duplicates are removed.

Dark Current Correction

Dark current is a characteristic of image sensors, which means, that image sensors also deliver signals in total darkness by warmness, for example, which creates charge carriers spontaneously. This signal overlays the image information. Dark current depends on two circumstances:

  1. Exposure time
    The longer the exposure, the greater the dark current part. I.e. using long exposure times, the dark current itself could lead to an overexposed sensor chip
  2. Temperature
    By cooling the sensor chips the dark current production can be highly dropped (approx. every 6 °C the dark current is cut in half)

Correcting Dark Current

The dark current correction is a pixel wise correction where the dark current correction image removes the dark current from the original image. To get a better result it is necessary to snap the original and the dark current images with the same exposure time and temperature.

Note
Dark current snaps generally show noise.

To correct the dark current pixels following steps are necessary:

  1. Black out the lens completely
  2. Set exposure time according to the application
  3. Set the number of image for calibration in "Setting -> Base -> ImageProcessing -> DarkCurrentFilter -> CalibrationImageCount" (Figure 3).
  4. Set "Setting -> Base -> ImageProcessing -> DarkCurrentFilter -> Mode" to "Calibrate" (Figure 3)
  5. Snap an image ("Acquire" with "Acquisition Mode = SingleFrame")
  6. Finally, you have to activate the correction: Set "Setting -> Base -> ImageProcessing -> DarkCurrentFilter -> Mode" to "On"
  7. Save the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"
    (Settings can be saved in the Windows registry or in a file)

The filter snaps a number of images and averages the dark current images to one correction image.

Note
After having re-started the camera you have to reload the capture settings vice versa.
Figure 3: Image corrections: CalibrationImageCount
Figure 4: Image corrections: Calibrate
Figure 5: Image corrections: Dark current

Flat-Field Correction

Each pixel of a sensor chip is a single detector with its own properties. Particularly, this pertains to the sensitivity as the case may be the spectral sensitivity. To solve this problem (including lens and illumination variations), a plain and equally "colored" calibration plate (e.g. white or gray) as a flat-field is snapped, which will be used to correct the original image. Between flat-field correction and the future application you must not change the optic. To reduce errors while doing the flat-field correction, a saturation between 50 % and 75 % of the flat-field in the histogram is convenient.

Note
Flat-field correction can also be used as a destructive watermark and works for all f-stops.

To make a flat field correction following steps are necessary:

  1. You need a plain and equally "colored" calibration plate (e.g. white or gray)
  2. No single pixel may be saturated - that's why we recommend to set the maximum gray level in the brightest area to max. 75% of the gray scale (i.e., to gray values below 190 when using 8-bit values)
  3. Choose a BayerXY in "Setting -> Base -> Camera -> GenICam -> Image Format Control -> PixelFormat".
  4. Set the (Filter-) "Mode = Calibrate" (Figure 6)
  5. Start a Live snap ("Acquire" with "Acquisition Mode = Continuous")
  6. Finally, you have to activate the correction: Set the (Filter-) "Mode = On"
  7. Save the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"
    (Settings can be saved in the Windows registry or in a file)
Note
After having re-started the camera you have to reload the capture settings vice versa.

The filter snaps a number of images (according to the value of the CalibrationImageCount, e.g. 5) and averages the flat-field images to one correction image.

Figure 6: Image corrections: Host-based flat field correction

Host-based Flat-Field Correction With Calibration AOI

In some cases it might be necessary to use just a specific area within the camera's field of view to calculate the correction values. In this case just a specific AOI will be used to calculate the correction factor.

You can set the "host-based flat field correction" in the following way:

  1. All necessary setting can be found under "ImageProcessing"-> "FlatfieldFilter".
  2. Stop "Continuous" acquisition mode.
  3. Set "CalibrationImageCount" to, for example, 5.
  4. Set "Mode" to "Calibrate".
  5. Set "CalibrationAoiMode" to "UseAoi".
  6. Set the properties ("X, Y and W, H") appeared under "CalibrationAOI" to the desired AOI.
  7. Start "Continuous" acquisition mode.
  8. Finally, you have to activate the correction: Set the "Mode" to "On".
Figure 7: Image corrections: Host-based flat field correction with calibration AOI

Host-based Flat-Field Correction With Correction AOI

In some cases it might be necessary to correct just a specific area in the camera's filed of view. In this case the correction values are only applied to a specific area. For the rest of the image, the correction factor will be just 1.0.

You can set the "host-based flat field correction" in the following way:

  1. All necessary setting can be found under "ImageProcessing" -> "FlatfieldFilter".
  2. Stop "Continuous" acquisition mode.
  3. Set "CalibrationImageCount" to, for example, 5.
  4. Set "Mode" to "Calibrate".
  5. Start "Continuous" acquisition mode.
  6. Now, you have to activate the correction: Set the "Mode" to "On".
  7. Set "CorrectionAOIMode" to "UseAoi".
  8. Finally use the properties ("X, Y and W, H") which appeared under "CorrectionAOI" to configure the desired AOI.
Figure 8: Image corrections: Host-based flat field correction with correction AOI

Camera-based Flat-Field Correction

The camera-based Flat-Field Correction feature supports full AOI and 14 bit to 14 bit correction (12 bit coefficients.). This enables a pixel to pixel correction and saves dozens of % CPU load and lowers latency. To reduce noise averaging of a number of "mv Flat-Field Correction Calibration Images" is possible. A correction image in the camera is then calculated. This may take some time / number of images, however, the camera blinks green. One correction image can be stored for all user settings.

The camera-based Flat-Field Correction is independent of the offset and uses a run-mode trigger (e.g. external trigger).

You can set the "camera-based flat field correction" in the following way:

  1. Set "mv Flat-Field Correction Calibration Image Count" to, for example, 5.
  2. This will average 5 images before calculating the FFC factors to reduce impact of noise.
  3. Stop "Continuous" acquisition mode, then right click on "int mvFFCCalibrate()" -> "Execute".
  4. Finally, you have to activate the correction: Set the "mv Flat-Field Correction Enable = 1".

Depending on the sensor, this need some time, because the data is stored in the internal flash memory (yellow LED lights up).

Figure 9: wxPropView - settings

Example

Figure 10: wxPropView - inhomogeneous light / pixel histogram
Figure 11: wxPropView - inhomogeneous light / pixel histogram (horizontal)
Figure 12: wxPropView - compensated light / pixel histogram
Figure 13: wxPropView - compensated light / pixel histogram (horizontal)