Precise radiometric alignment

Producing consistent HDR images of scenes with moving objects and varying illumination.

HDR image of a scene with moving subject: ghost artefacts and blending failure on the left, fine blending on the right.

A typical HDR merging artefacts versus precisely blended image.

Color values in an image are related to image irradiance by a nonlinear function. Radiometric alignment is a process of estimating this nonlinear function.

A precise radiometric alignment is desirable in image fusion tasks. Especially when a number of fusion techniques are applied in parallel. I.e. when merging a stack of exposure-bracketed images and concurrently processing the scene to remove duplicates of moving objects.

PhotoAcute Studio implements some complex algorithms to achieve precise radiometric alignment. Why? Can't we just use the values supplied by the camera built-in EXIF fields?

The usage of EXIF values (exposure time, apperture, ISO boost) is similar to linear alignment and might be enough in some cases. I.e for the typical RAW image the pixel intensities are aligned like this (the data is obtained from a real scene):

RAW image pixel intensity relations (5 images):
the intensity in the composite image vs. the intensity in the input images

For the same scene, same camera, but with camera built-in JPEG output the result is:

JPEG image pixel intensity relations (5 images):
the intensity in the composite image vs. the intensity in the input images

The relation is clearly non-linear. That happens due to a number of reasons:

In-camera applied tone curve;

In-camera applied contrast and brightness settings.

Even for a RAW data the EXIF-supplied alignment might be inprecise due to:

Non-stationarity of the illumination, for example:
- moving clouds in the sky can cause both changes in brightness and change in ratio of direct vs ambient illumination;
- moving of the incadescent illumination source or moving of large reflective areas;

The raw sensor values are not necessarely perfectly linear;

Not all the required EXIF data is always available (i.e. ISO setting may be reported as 'auto').

PhotoAcute Studio versions 2.88 and earlier used quadratic approximation to perform radimetric alignment. The problems with quadratic approximation are:

Radiomatric alignment curve should always be monotonic. This should be watched in case of quadratic fitting, which can create extremums in the curve;

Possible noise amplification if there is even a slight misalignment in shadows.

Versions 2.89 and above are using a new proprietary method that is much more precise and does not suffer from quadratic approximation issues. This method uses EXIF supplied data as a first approximation (if available) and builds an approximation function using photographed scene pixels to get precise result.

The difficulties the scene-based radiometric alignment algorithm is tackling with are:

Noise;

Possible geometric misalignment of source images;

Clipped areas;

Lack of certain luminocity ranges in the scene;

Moving objects may interfere;

Changes in focus (in Focus-Stacking tasks).

As a demonstration of complexity of scene-based radiometric alignment analysis, here is an example of per-pixel raw input of radiometric relations between two shots of the same scene at a different exposures and with some moving objects in it:

Per-pixel radiometric relations between two snapshots (normalized):
The intensity relation is different for the same intensity value in different areas of the picture.

Real-life example

Six exposure-bracketed images, handheld, leaves moving in the wind.


HDR image merged with moving objects handling, without radiometric alignment	HDR image merged with moving objects handling, WITH radiometric alignment


HDR image merged with moving objects handling, without radiometric alignment	HDR image merged with moving objects handling, WITH radiometric alignment


HDR image merged with moving objects handling, without radiometric alignment	HDR image merged with moving objects handling, WITH radiometric alignment