Event-specific Image Importance

When creating a photo album of an event, people typically select a few important images to keep or share. Modeling this selection process will assist automatic photo selection and album summarization. In this project, we show that the selection of important images is consistent among different viewers, and that this selection process is related to the event type of the album. We introduce the concept of event-specific image importance. We propose a Convolutional Neural Network (CNN) based method to predict the image importance score of a given event album, using a novel rank loss function and a progressive training scheme. Results demonstrate that our method significantly outperforms various baseline methods.

Automatic organization of personal photos is a problem with many real world applications, and can be divided into two main tasks: recognizing the event type of the photo collection, and selecting interesting images from the collection. After we have the model to predict event-specific image importance, we attempt to simultaneously solve the two tasks: album-wise event recognition and image- wise importance prediction. We propose a hybrid system consisting of three parts: A siamese network-based event-specific image importance prediction, a Convolutional Neural Network (CNN) that recognizes the event type, and a Long Short-Term Memory (LSTM)-based sequence level event recognizer. We propose an iterative updating procedure for event type and image importance score prediction. We experimentally verified that image importance score prediction and event type recognition can each help the performance of the other.

CUFED Dataset

The CUration of Flickr Events Dataset (CUFED) dataset is an event curation dataset containing 1883 albums. Each album describes an event, and the event type of albums are from 23 most common events in our daily life, ranging from Wedding to Nature Trip. The size of albums varies between 30 and 100 images.

For each album, the event type of the album is voted by 12 participants. For some ambiguous albums, there are multiple event types.

For each album, images in the album are rated by 5 participants by their importance/interestingness score based on the content and event type of the album. The score is the "event-specific image importance" of the image.