Methods for Sentiment Analysis and Social Media Popularity of Crowdsourced Visual Contents

Abstract

This thesis collects all the research work done by the PhD candidate in the Joint Open Lab for Wireless Applications in multi-deVice Ecosystems (JOL WAVE Catania) of TIM Telecom Italia, which granted his doctoral fellowship. The crowdsourcing paradigm opens new opportunities to understand various aspects of people's interactions, preferences and behaviors. In this thesis we investigated methods aimed to infer people's reaction toward visual contents, under different headings. We first focus on the task of understanding how people visit a place (e.g., a cultural heritage site) and infer what catch most their attention and interest, by means of the analysis of the shared photos taken by the users themselves. Then, addressing the issues related to the noisy text associated to images, we defined a method for Image Popularity Prediction, considering an alternative source of text automatically extracted from the visual content. We first highlight the drawbacks of the text used in most of the state of the art methods, and then experimentally compared the two sources of text. Starting from the analysis of the state of the art in image popularity prediction, we observed that a time-aware approach is needed, as the temporal normalization commonly employed in literature makes two contents published at different times incomparable. For this reason we introduced a new task, named Image Popularity Dynamics Prediction, which aims to predict the evolution of the engagement scores of a photo over a period of 30 days from its upload. To challenge the problem, we introduce a large scale dataset of 20K photos whose engagement scores have been tracked for 30 days. Moreover, we presented an approach that is able to perform the prediction at time zero. Furthermore, we investigated methods for scene popularity estimation, from a set of videos taken by people attending a public event. This involved the definition of methods for unsupervised video segmentation and scene clustering, able to work in both mobile and wearable domains. The methods have been developed considering unconstrained scenarios without any prior on the input videos. In appendix, we also report some additional results and the pseudocode of the developed algorithms.