Assignment 4: Image Warping (or fun with frames)

Due Tues, April 6 (150 points)

Overview

In this assignment you will experiment with computing and applying image transformations using matlab. Part 1 will ask you to merge parts of two images into a single seamless output using warping. Part 2 will ask you to merge an image with a video into a single output video using warping.

Part 1

Images by THEfunkyman on flickr.com

Example input to your method will be two file names denoting a target image (left), and source image (center). Output will be something like the result image (right) where part of the source image has been warped to fit within the frame in the target image.

Part 1.1 (20 pts)

Implement a wrapper function, wrapper(targetfilename,sourcefilename,resultfilename), that takes three file names as input: targetfilename specifies the target image (left), sourcefilename specifies the source image (center), resultfilename specifies where to save the result (right).

This function should:

Read in the target and source images.
Ask the user to click on 4 points in each image. For the target image this specifies the rectangle where you want to insert (ie the user might click on the four corners of the frame in the target image). For the source image this specifies what you want to insert (ie the user should click on 4 points denoting what rectangular piece to insert).
Make these user specified points integers so that they can be used to index into images.
Call the frameit function described next.

Useful matlab functions for this part are imread, ginput, floor, ceil, round. As always use help in matlab to get descriptions of each function (or look them up on google).

Part 1.2 (65 pts)

Implement a function, frameit(targetimg,sourceimg,targetx,targety,sourcex,sourcey,resultfilename) that takes as input the images read in by wrapper.m, the x, and y locations of the points clicked on by the user for the source and target images, and the file name for where to save the resulting merged image.

This function should:

Compute the best affine transform between the source and target points.
Extract the piece of the source image that the user selected.
Transform this piece according to the computed affine transform.
Insert this transformed image into the target image in the rectangle specified by the user
Display the resulting merged image.
Save the resulting image as resultfilename.

Useful matlab functions for this part are cp2tform, imtransform, imresize, min, max. As always use help in matlab to get descriptions of each function (or look them up on google).

Part 1.3 (15 pts)

Start with the example images I have provided: Target Image, Source Image to test out your code. Now, find or capture your own images, and play around with making interesting combinations. You should run your code on at least 3 different pairs of images.

Part 2

Part 2.1 (50 pts) - Embedding an image into your video

	+		=
Input Video - frames.avi - Click to play. Uncompressed version available here		Selected frame that you want to embed, and selected rectangular region		Result video with embedded image - Click to play. Uncompressed version available here

Load in your test video, frames2.avi.gz is an avi file containing a test movie and can be read in using aviread in matlab. Alternatively I have also provided a matfile containing the frames, frames.mat and can be read in using load in matlab.

First, choose the video frame that you want to embed (imagesc and pause are useful for viewing the frames of a video). Next, select the rectangular region of this video frame that you want to embed (source points). Make sure the part you select is perfectly rectangular otherwise your result video will have visual artifacts. For every video frame after your selected video frame, click on the four corners of the cardboard frame (target points). There have been colored dots placed at the corners to aid you in good localization. I would suggest saving these locations so that you only have to click on them once.

For every video frame after your selected video frame, compute the transform between the target points and source points. Transform the source region accordingly, and embed it in the target region. Here you will just need to apply your frameit.m function from Part 1!

Save the resulting video as result.avi. You can save an avi by creating an avifile object then repeatedly adding the video frames you have generated.

Extra Credit - Embedding a video into your video

	+		=
Input Video - frames.avi - Click to play. Uncompressed version available here		Video that you want to embed - You will add this into the Input Video, offset in time. Click to play. Uncompressed version available here		Result video with embedded video - Click to play. Uncompressed version available here

Instead of inserting a single static image into a set of frames, you can insert a different image into each frame, thereby embedding a video into a video. For every video frame after your selected video frame, insert a transformed frame from earlier in the video. For example if you are at video frame 120 and want your time offset to be -30 frames, then transform video frame 90 and embed it into the result video frame. You can either transform the whole image (as I have done in this example), or transform your hand selected regions, producing different effects.

Save the resulting video as extracredit.avi. You can save an avi by creating an avifile object then repeatedly adding the video frames you have generated.

What to turn in

Your code for Parts 1 and 2.
A wepage called results.html showing your resulting images (source, target, result) triples from Part 1, and your result video with embedded image (result.avi) from Part 2. If your video is too large for email you may post your web page online and send me a link.