CSE/ISE 364: Advanced Multimedia
Lectures: T/Th 3:50-5:10pm, Rm 2205 Computer Science
Course Webpage: http://tamaraberg.com/teaching/Spring_10/

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Office Hours: Tuesdays/Thursdays 5:10-6:10pm and by appointment



Topics will include:
  • Text
  • Sound
  • Images
  • Video
  • Tagging & Annotation
  • Social Media
  • Location Information
  • Speech Processing
  • Recommendation systems
The focus of this course will be on multimedia as accessed via the web. For each type of digital media studied we will discuss fundamentals including storage, compression, and use. We will also study various algorithms for organizing, retrieving, and manipulating digital media.

Assignments
Assignment 1 Due Feb 18
Assignment 2 Due March 2
Assignment 3 Due March 11
Assignment 4 Due April 6


Schedule:
DateTopicReadings
Jan 26Introduction - Slides-
Jan 28Text basics & Web document retrieval- SlidesThe Anatomy of a Search Engine
Feb 2Text modeling and classification - Slides, Demo Code-
Feb 4Using Matlab for multimedia - Tutorial (from Clarkson University), String Manipulation Exercise (from David Griffiths) An Extended Matlab Reference
Assignment 1 out
Feb 9Useful Matlab functionality - Some string code: text.m-
Feb 11Document clustering - Slides
Feb 16Intro to sound & digitization - Slides, Demo1, Demo2Chapter 6 of Textbook
Feb 18Sound analysis - SlidesAssignment 2 out
Feb 23Sound analysis & applications - See slides for Feb 18 - Filtering Demo"Content-Based Music Information Retrieval"
Feb 25Guest Lecture - Margaret Schedel - Digital Music-
March 2Intro to Computer Vision - Slides-
March 4Images, cameras, & color - SlidesAssignment 3 out
Chapter 4 of textbook
March 9Image Content Analysis - Slides, Demos-
March 11Image Blending & Compositing - SlidesA Multiresolution Spline With Application to Image Mosaics, Peter J. Burt and Edward H. Adelson
March 16Guest Lecture - Klaus Mueller - Visualization-
March 18Image Warping & Morphing - SlidesAssignment 4 out
March 23Image retrieval - Slides, DemoChapter 18 of textbook.
March 25Image retrieval (cont) - SlidesPlease come to office hours to discuss project ideas.
March 30Spring Break-
April 2Spring Break-
April 6Meta data & Tagging - SlidesPlease come to office hours to discuss project ideas.
April 8Project Proposal PresentationsPlease prepare a 5 minute presentation describing your project proposal.
April 13Location Information - Slides-
April 15Social Media-
April 20Project Update Presentations-
April 22Project Help day-
April 27Collaborative Filtering-
April 29Words & Pictures-
May 4Final Project PresentationsJoe, Kevin, Bia, Dmitriy, Kaustubh & Paul, Colin, Mike
May 6Final Project PresentationsTimothy, Gajendra, Jonathan, Carlos, Greg & Marcin, Nicolas, Luis
May 11Final Project Write-Up due8 page document including abstract, introduction (motivation), method, results, & figures


Grading:
 This course will focus on developing a hands on understanding of various types of multi-media. There will be 4 programming assignments related to the course topics. Students will also be responsible for defining and developing a project related to multi-media over the course of the semester, including a project proposal, status update, and final project presentation. A project write-up will serve as your final exam.

Late homeworks will be accepted with a 10% reduction in value per day late.

Grading will consist of 55% assignments, 35% project, 10% participation.

Prerequisites:
 You do not need to have taken CSE/ISE 334. Students are expected to be proficient in programming and the basics of digital media, but I will provide most necessary background for the course as we go. Come talk to me if you have any questions!


Useful links:
Matlab
 Matlab tutorial by Hany Farid and Eero Simoncelli - Link
A more comprehensive Matlab tutorial by David Griffiths - Link
Matlab Answers from MIT - Link
Online Mathworks Matlab documentationLink
Lots of Matlab tutorialsLink

Book
 Fundamentals of Multimedia, Ze-Nian Li, and Mark S. Drew.

Other useful reference books
 Artificial Intelligence: A Modern Approach, Russel and Norvig.
Computer Vision: A Modern Approach, Forsyth and Ponce.
Foundations of Statistical Natural Language Processing, Christopher D. Manning, and Hinrich Schutze.