HW4: Automatic Colorization
CSE590: Computational Photography

Due Date: 11:59pm on Sunday, Oct. 30, 2011

Overview

In this assignment you will automatically "colorize" face images using a large data base of face images. The algorithm will procceed as follows. Given a grayscale query image, match that image to a large training set of color images. Transfer the color from matched image(s) to the query image.

Baseline Implementation

1. Compute an image descriptor for the query face.
2. Compute image descriptors for the training set of face images.
   
3. Find the training image that is most similar to the query.
4. Transfer color from the matched training image to the query image.

Face image data is online here. Images in the testset directory are your grayscale query images. Images in the trainset directory are your large data base of color training images. Your task is to compute colorizations for each query image using the training data base.

Some parameters you should explore in your assignment are:

1. Choice of image descriptor (e.g. tiny image, gist, local feature based, etc)
2. Choice of image similarity measure (e.g. ssd, normalized correlation)
3. Choice of color space for transfer (e.g. HSV, LaB, etc)

Additional details can be found in the "80 million tiny images" reference

Enhancements

You may observe some artifacts in your resulting colorized images. You should explore enhancements to the basic colorization algorithm to deal with these artifacts. There is no single answer here and you will be graded on your ideas for enhancements and quality of your results.

Some possible ideas might be: using information from a set of similar data base images to colorize your image, transfering color from parts of matching images rather than the complete image, estimating better feature based alignments prior to colorization, merging the results with other work on colorization (e.g. "Colorization Using Optimization - code available), performing colorization on entire people (whole bodies rather than just faces) or other categories (e.g. landscapes or product images).

Deliverables

To turn in your assignment, email your commented code, readme, and a web page showing results to sbu590@gmail.com. If your images are too large, you may use links to an online version, but please send small versions via email!

Use both words and images to show us what you've done. Please:

• Include colorized results for each test image.
• Explain the algorithm you implemented, including the different choices of parameters that you tried.
• Explain the enhancements you implemented with details and example results for each enhancement.
• Explain where the algorithms worked, where they didn't and why.

Scoring

The core assignment is worth 100 points, as follows:

• 40 points for correct implementation of colorization algorithm and resulting colorized images.
• 20 points for exploration of parameters described above.
• 40 points for your proposed and implemented enhancements.