Multi-Modal Recommender Systems: Towards Addressing Sparsity, Comparability, and Explainability

Hoang, Tuan, Aghiles, and Hady will be delivering a tutorial at the ACM Web Conference 2023 conference that will take place in May 2023 in Austin, Texas.

The slides and the hands-on materials can be found here.

A video of the tutorial can be found here.

Abstract

Web applications frequently feature a recommender system to help users discover items (e.g., products, content articles) of interest. This tutorial focuses on multi-modality, i.e., the use of side information such as text, images, or graphs to augment the preference data. In particular, we cover several important aspects of multi-modality. First is how models rely on the auxiliary modality to address the sparsity of preference observations in order to better bridge users and items. These models are typically designed along modality lines, which we cover comprehensively. Second is how to manage comparison and cross-utilization of multi-modal models. The former is concerned with streamlining the treatment of models that share the same modality. The latter is concerned with using a model initially designed for one modality with another. Third is how the auxiliary modalities could act as recommendation explanations, as recipients may find textual, visual, or graphical explanations more intuitive. This is a hands-on tutorial, whereby lectures are supplemented with exercises conducted with Cornac, a comparative framework for multimodal recommender systems.

Outline

Multimodal Recommender Systems
Hands-on 1: Cornac framework
Text, Image, and Graph Modalities
Hands-on 2: Multi-modal & Cross-modal
Explainability
Hands-on 3: Explainability
Q & A

Target Audience

Introductory to intermediate. We target both practitioners seeking applicable experience, as well as researchers interested in recent and future research directions in multimodal recommender systems.

Prerequisites

Basic knowledge of Python, machine learning and recommender systems.

Speakers