Data Mining

Original Paper: http://i.stanford.edu/~julian/pdfs/eccv2012.pdf

This paper tries to predict tags, labels or recommendations for images based on social network meta data of the images, i.e., the connection between images, like images uploaded to same groups. The task is a binary labeling problem using structure learning techniques

Challenge is to find how to effectively use/model social network data and predict based on it.

Modeling is done as follows:

we propose a graphical model that treats image classication as a problem of simultaneously predicting binary labels for a network of photos. Nodes represent images, and edges represent relationships between images. Our intuition that images sharing common properties are likely to share labels allows us to exploit techniques from supermodular optimization, allowing us to efficiently make binary predictions on all images simultaneously

Labels range from subjective to objective. Social network metadata provides context that is not inherent in the image. So, the model proposed in this paper outperforms SVM.

Following is a gist of dataset:

The photo itself
Photo data, including the photo’s title, description, location, timestamp, viewcount, upload date, etc.
User information, including the uploader’s name, username, location, their network of contacts, etc.
Photo tags, and the user who provided each tag
Groups to which the image was submitted (only the uploader can submit a photo to a group)
Collections (or sets) in which the photo was included (users create collections from their own photos)
Galleries in which the photo was included (a single user creates a gallery only from other users’ photos)
Comment threads for each photo

Images sharing common tags are likely to share image labels and similarly other properties as well.
Features vectors represent presence or absence of each tag.
Memory is important as feature vectors are stored in memory.

For Nodes, 1000 most popular words / tags / groups features are used by feature reduction.
For edges, indicators for no. of tags, groups, collections etc are used.

Performance is reported from previous papers using, image tags alone, SVM etc. They are compared using Mean Average Precision and Balanced Error Rate.