CSE 591: Recognizing People, Objects, and Actions

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Lectures: Tues/Thurs 11:20-12:40pm Rm N310, Soc Behav Sci
Office Hours: Tues/Thurs 12:40-1:40pm, and by appointment
Course Webpage: http://tamaraberg.com/teaching/Fall_09


*Announcements*

Dec 2 - No office hours tomorrow due to visiting speakers. Instead I will hold additional office hours for last minute project questions: Friday 2:45-3:45, and Monday 3-5pm.

Oct 6 - Due date for HW2 has been extended to Sunday, Oct 11.

Oct 5 - Class on Thurs, Oct 8 will be held in 1306 CS so we can use the internet to form our debate presentations.

Sept 29 - Reminder HW1 is due today. HW2 is now online.

Sept 19 - The due date for Homework 1 has been extended to Sept 29.

  • Visualize the top 20 eigenfaces and produce a plot showing the computed eigenvalues in sorted order.

    Sept 15 - Homework 1 is online.

    Sept 11 - I have put up the tentative paper presentation schedule (and shifted a few dates around). Please look this over and find your presentation day. If you are registered for the class and do not have a presentation scheduled please email me today otherwise I will assign you a paper.

    Sept 8 - The deadline for HW0 has been extended to Tues, Sept 15.

    Sept 7 - We will have a matlab help session on Tues, Sept 8 at 5pm in room 1204. Please come by and we can help with matlab or homework questions.

    Sept 5 - I have received feedback from a few students that reading 2 papers per class will take a lot of time. Instead you may select one of the assigned papers to read and summarize for each lecture.

    Sept 4 - Homework 0 is online. I have also started a discussion board on blackboard where you can ask/answer questions about the homework or matlab.


    • Introduction

    Recognition is one of the core pursuits of computer vision. In recognition one attempts to attach semantics to visual data such as images or video. Object recognition is an important subtopic where one builds models to recognize object categories or instances. Other subtopics include: activity recognition -- building descriptions of what people are doing from visual data, face recognition -- attaching identities to pictures or video of faces, and detection -- localizing all instances of a particular category in an image. This course will look at both historical and current methods for recognizing objects, people, actions, and scenes in images and video. Students will have a chance to define their own problems and work on solutions through a course project.


    Topics
    • Objects - single instance or category based
    • People - faces, pedestrians, pose, and actions
    • Scenes - whole image features, recognition in context, surfaces
    • Recognition by the human visual system
    • Recognition using vision + other modalities

    Tentative Schedule
    DateTopic Readings Presenter & SlidesAssignments
    Sept 1Intro & Overview of Course-Tamara - SlidesGet access to matlab, do a tutorial.
    Sept 3Computer Vision Review Vision is getting easier every dayTamara - SlidesGet access to matlab, do a tutorial. HW0 out
    Sept 8Bag of feature models - Discriminative Object Recognition from Local Scale-Invariant Features,
    Learning Globally-Consistent Local Distance Functions for Shape-Based Classification    		Tamara - Slides-
    Sept 10Bag of feature models - GenerativeVisual Categorization with Bags of Keypoints,
    Discovering Objects and Their Location in Images    		Tamara - Slides-
    Sept 15Spatial ModelsObject Class Recognition by Unsupervised Scale-Invariant Learning,
    Shape Matching and Object Recognition Using Low Distortion Correspondence    		Vicente - Slides, Tamara - SlidesHW0 due. HW1 out.
    Sept 17Face DetectionA Statistical Method for 3D Object Detection Applied to Faces and Cars,
    Robust Real-Time Face Detection    		 Aravinda - Slides, Debaleena - Slides-
    Sept 22Face RecognitionFace Recognition using Eigenfaces,
    Face Recognition Based on Fitting a 3D Morphable Model    		 Anupam - Slides, Kiwon - Slides-
    Sept 24Recognition by the human visual systemFace recognition by humans: 20 results Tamara - Slides, Bharti - Slides-
    Sept 29Correction day - no class- -HW1 due, HW2 out
    Oct 1Recognition by the human visual systemThe Role of Top-down and Bottom-up Processes in Guiding Eye Movements during Visual Search Guest Lecture - Greg Zelinsky-
    Oct 6Recognition by the human visual system (cont)- See slides from Sept 24-
    Oct 8Categories pro/con, group discussionPrinciples of Categorization,
    Categories (Aristotle),
    100 years of Psychology of Concepts     		In class group discussionsHW2 due. See me in office hours to discuss projects.
    Oct 13Categories pro/con, group debate- Group presentationsSee me in office hours to discuss projects.
    Oct 15Project ProposalsIn class presentations whole classPrepare 5-10 minute presentation
    Oct 20Recognizing AttributesAttribute and Simile Classifiers for Face VerificationGuest Lecture - Alex Berg - Slides-
    Oct 22Intro to People & Actions-Tamara - Slides-
    Oct 27Pedestrian DetectionPedestrian Detection in Crowded Scenes,
    Histograms of Oriented Gradients for Human Detection    		Yifan - Slides,
    Jose - Slides    		-
    Oct 29Pose Estimation in imagesRecovering Human Body Configurations: Combining Segmentation and Recognition,
    Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations    		Visruth - Slides,
    Thanadit - Slides    		 -
    Nov 3Project Update PresentationsIn class presentations whole classPrepare 5-10 minute presentation
    Oct 5Pose Estimation in images (cont)-- -
    Nov 10Action RecognitionRecognizing Action at a Distance,
    Learning Realistic Human Actions from Movies    		Piyush - Slides,
    Jonathan - Slides    		-
    Nov 12ScenesOn the semantics of a glance at a scene,
    Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories    		 Tamara,
    Xufeng - Slides    		-
    Nov 17Recognition in ContextObjects in Context(Xufeng cont),
    Hiep - Slides    		-
    Nov 19Project Update PresentationsIn class presentations whole classPrepare 5-10 minute presentation
    Nov 24Recognizing surfacesRecovering Surface Layout from an Image,
    Parsing Images of Architectural Scenes    		Taj - Slides,
    Sagnik - Slides    		-
    Nov 26Thanksgiving - no class---
    Dec 1Pictures & Other Meta-DataIm2GPS,
    Estimating Age, Gender and Identity using First Name Priors    		Muhammed - Slides,
    Ritwik - Slides    		-
    Dec 3Guest Lecture - Fernando de la Torre*1310 Computer Science*--
    Dec 8Final Project PresentationsIn class presentations Sagnik-Debaleena, Bharti, Yifan-Jose-Ritwik-Fatih, Anupam, Xufeng-Aravinda
    		In class presentations
    Dec 10Final Project PresentationsIn class presentations Visruth-Piyush, Vicente, Hiep, Jonathan, Thanadit, TajIn class presentations
    Dec 15-- -Final Project Write-Up Due via email


    Grading
     There will be 3-4 short homeworks during the first month and a half of the course to get students aquainted with computer vision and recognition. Over the final two months of the course students will develop and present a project related to recognition. Students will also be responsible for leading one class paper discussion. One paragraph summaries of each paper will be due before the start of class.

    Grading will consist of: Assignments (30%), Project (40%), Paper presentation (10%), Paper summaries (10%), Participation (10%).

    No prior experience in computer vision is required to take this course. Homeworks should be done individually, but projects may be done in groups. Homeworks will be completed in matlab.

    Submit all paper summaries, homeworks, and project presentations to: cse591@gmail.com


    Useful links

    Matlab
     Student Matlab licenses can be purchased from mathworks for $99 - Link.
    Matlab tutorial by Hany Farid and Eero Simoncelli - Link
    A more comprehensive Matlab tutorial by David Griffiths - Link

    Data
     Label Me - Link
    Tiny Images - Link
    Code for downloading Flickr images - Link

    Computing Features
     SIFT features - Link
    Scale Invariant Interest Points - Link
    Affine Covariant Regions - Link
    Shape Contexts - Link
    Gist - Link

    Other Useful Software
     Various Code from INRIA - Link
    Various Code from Oxford - Link
    Various useful machine learning tools - Link

    Reference Books
     Forsyth, David A., and Ponce, J. Computer Vision: A Modern Approach, Prentice Hall, 2003.
    Hartley, R. and Zisserman, A. Multiple View Geometry in Computer Vision, Academic Press, 2002.
    Stephen E Palmer, Vision Science: Photons to Phenomenology, MIT Press, 1999.