Photometric stereo registration of a moving object

Typically, the scene or object undergoing photometric stereo registration is stationary. However, you can use the PhotometricStereo step with images taken of a moving object (for example, on a conveyor).

In the case of a moving object, the source images must be aligned before you can run the PhotometricStereo step. You can align the images in 1 of 2 ways. The first method requires a start and an end image of your scene taken with all lights on, and finding a model in both images. These 2 images serve as anchors for locating the object in the intermediate images, which are the photometric stereo source images taken with directional lighting. The second method finds a model within each source image (taken with directional lighting) to locate the moving object from one frame to the next. You can use a PatternMatching step, ModelFinder step, or other Finder step (such as the RectangleFinder step) to find the model occurrence, which must be visible in each source image.

Once you have acquired the start/end images (only for the first method) and the photometric stereo source images, apply the following:

1. Define a model. To do so, use a PatternMatching step or Finder step. This is the model to find in the start/end images (first method) or the model to find in all source images (second method).

2. Find the model. If using the first method, find the model in the start/end images. If using the second method, find the model in all source images.

3. Calculate the model's displacement per photometric stereo image. Note that the object is assumed to have moved consistently. That is, the displacement in X, Y, and any angular or rotational displacement is constant.

4. Align all the photometric stereo source images such that the model is positioned in the same location for each image. You can use the ImageProcessing step's Translate operation if the motion is in a straight line. Otherwise, use the ImageCorrection step.

5. Run the PhotometricStereo step.

The key step for a successful photometric stereo operation on a moving object is source image alignment. The defined model must be visible in the start and end images (first method) or it must be visible in all source images (second method). In addition, the object's displacement must be small, relative to the lighting setup, so that distinct directional lighting is maintained between each image. A small displacement also minimizes perspective distortion and parallax errors.

For an illustration of photometric stereo registration on a moving object, see the Photometric Stereo With Motion example project.