180 Project 1

Overview

This project aimed to automatically produce color images from Sergei Mikhailovich Prokudin-Gorskii's digitized glass plate negatives. These negatives contain three separate exposures taken through red, green, and blue filters, capturing the Russian Empire in the early 20th century. Our goal was to align these exposures and combine them into a single color image with minimal visual artifacts.

Approach

Small images: Exhaustive search

For small images, I implemented an exhaustive search algorithm using the euclidean distance (L2 norm) as the error metric. This approach involved shifting the green and red channels over a range of possible offsets relative to the blue channel, which remained fixed. For each offset, the L2 distance between the overlapping regions was calculated. The offset that yielded the minimum error was selected as the best alignment.

While effective for small images, this method had limitations. The computational cost increased significantly with larger search ranges, making it impractical for bigger images or when a wider range of potential offsets needed to be considered. Despite these constraints, the exhaustive search proved to be robust and accurate for aligning the color channels in smaller images from the Prokudin-Gorskii collection, producing visually pleasing results with minimal artifacts.

Large images: Gaussian Pyramids with edge detection

For the larger images, I utilised a Gaussian pyramid approach in conjunction with edge detection.

I created a Gaussian pyramid by recursively applying a Gaussian filter to the image and downsampling it. We then calculate the best shift at the top level of the pyramid, and use this as a base for the next layer. By repeating this at every layer until we get to our original image we do not have to calculate as big of a window at any single level.

Takeaways

I think my techniques worked quite well. The only image that I think could be aligned better is the "lady" image. Which I believe is because of its blurry edges. I would like to improve the white balance because as you can see in the full list of examples, some images it improves, some it makes unnaturally dark.

Bells and Whistles

Edge Detection

To improve alignment accuracy on bigger images I applied a sobel kernel to the image to get isolate the edges in the x and y direction. This improves the accuracy of the alignment as the edges, rather than all the pixels in the image.

Original Image

Shift: Green: (41, 9), Red: (35, -291)

Image after Sobel kernel convolution

Final image Gaussian Pyramid with edge detection

Shift: Green: (49, 23), Red: (107, 40)

Localized Histogram Equalisation

Process: I got the histogram for each of the 3 channels of the image, and then applied histogram equalization by mapping the cumulative distribution function (CDF) of the histogram to the pixel values. This was done for each of the 3 channels.

I then performed this technique in localized regions across the image to avoid darkening or lightening the image too much overall.

As you can see the added contrast makes detail in the image more visible, in the global histogram equalisation, and even more so with the localized histogram equalisation. You can also see that the global histogram equalisation makes the image look unnatural.

Before Histogram Equalisation

Image after GLOBAL Histogram Equalisation

Image after LOCALIZED Histogram Equalisation

Automatic White Balance

Process: I first calculated the illumant using the average pixel value of the aligned image. Next, I computed scaling factors for each color channel (R, G, B) that would shift the estimated illuminant towards pure white (equal values in all channels). These scaling factors were then applied to the entire image, effectively removing the color cast. To preserve the overall brightness of the image, we normalized the scaling factors to ensure that the brightest channel remained unchanged.

Before white balance

After white balance

The illuminant

Final Thoughts

Over the summer I learned how to edit photos using software like Adobe Lightroom. This project was particularly interesting to me because I got to see how a lot of their features are implemented in code.

The next things that I would like to implement are automatic cropping (so we can avoid the chromatic aberration at the edges of the image) and color grading tools for personalizing the image.