RecSys 2019 - Recommendation in Multi-Stakeholder Environments (RMSE) and 7th International Workshop on News Recommendation and Analytics (INRA 2019) in RecSys 2019

Once that you’re in a conference, the first thing that you do is certainly go to the main talks and see the presentations of big companies, look for the big cases, hang out with authors of great papers stating the SOTA and so on. 

This is the safest path and probably most of those cases will have press releases discussed in media or subject in blogposts and you can have access to this before almost everyone. 

However, one thing that I think is very underestimated in conferences are the workshops.

My favorite definition of what a workshop is comes from Oxford Dictionary that is “a meeting of people to discuss and/or perform practical work in a subject or activity”.

For me, workshops are the best blend between the conference format - that contains the peer-review and trusteeship from the chair – in a smaller format that you can go for a specific but subjacent subject and have direct contact with the authors. In those places, there’s some information that is not available for the greater public.

I’ll talk today about two workshops that I attended in RecSys 2019 that is Recommendation in Multi-Stakeholder Environments (RMSE) and 7th International Workshop on News Recommendation and Analytics (INRA 2019).

But first I’ll explain why I attended those two workshops in RecSys. 

First of all, I decided to attend RMSE because here at MyHammer, we’re dealing with several challenges regarding the recommendations in a marketplace. We need to take into consideration not only a single platform user but several different users that not only interact with each other, but we have all the dynamics of a job marketplace. We have complex competition dynamics and seeing the proceedings. I saw that I could learn tons of ways to apply this knowledge in the company.

For the INRA as this specific topic doesn’t have much in common with job recommendation, I decided to attend because some papers are talking about some very relevant aspects that have a big intersection with our use case like giving recommendations for non-logged users, contextual multi-armed bandits, content-representation and strategies to use word embeddings.   

For the matter of clarity, I’ll include the official descriptions of those workshops:

7th International Workshop on News Recommendation and Analytics (INRA 2019)

This workshop primarily addresses news recommender systems and analytics. The news ecosystem engulfs a variety of actors including publishers, journalists, and readers. The news may originate in large media companies or digital social networks. INRA aims to connect researchers, media companies, and practitioners to exchange ideas about creating and maintaining a reliable and sustainable environment for digital news production and consumption.

Topics of interests for this workshop include but are not limited to:

• News Recommendation
• News Analytics
• Ethical Aspects of News Recommendation

For the RMSE: Recommendation in Multi-Stakeholder Environments we have the following description:

One of the most essential aspects of any recommender system is personalization – how well the recommendations delivered suit the user’s interests. However, in many real world applications, there are other stakeholders whose needs and interests should be taken into account. In multisided e-commerce platforms, such as auction sites, there are parties on both sides of the recommendation transaction whose perspectives should be considered. There are also contexts in which the recommender system itself also has certain objectives that should be incorporated into the recommendation generation. Problems like long-tail promotion, fairness-aware recommendation, and profit maximization are all examples of objectives that may be important in different applications. In such multistakeholder environments, the recommender system will need to balance the (possibly conflicting) interests of different parties.

This workshop will encourage submissions that address the challenges of producing recommendations in multistakeholder settings, including but not limited to the following topics:

• The requirements of different multistakeholder applications such as:

  • Recommendation in multisided platforms

  • Fairness-aware recommendation

  • Multi-objective optimization in Recommendation

  • Value-aware recommendation in commercial settings

  • Reciprocal recommendation

• Algorithms for multistakeholder recommendation including multi-objective optimization, re-ranking and others
• Evaluation of multistakeholder recommendation systems
• User experience considerations in multistakeholder recommendation including ethics, transparency, and interfaces for different stakeholders.

RecSys is one of the best conferences for Recommendation Systems because it’s an excellent blend between industry and academics, where in one side in academia we have a fast paced rhythm of research in scenarios more complex than ever before and for industry most of the companies are moving forward those new methods in battle-tested environments where we have not only sterile benchmarks, but recommender systems applied in real live data. 

Below I’ll highlight some interesting papers and some quick notes about them. I strongly suggest the read on the full papers because this is the SOTA in terms of research and industrial applications in recommender systems.

7th International Workshop on News Recommendation and Analytics (INRA 2019)

• On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems, Gabriel De Souza P. Moreira, Dietmar Jannach and Adilson Marques Da Cunha. Abstract: News recommender systems are designed to surface relevant information for online readers by personalizing their user experiences. A particular problem in that context is that online readers are often anonymous, which means that this personalization can only be based on the last few recorded interactions with the user, a setting named session-based recommendation. Another particularity of the news domain is that constantly fresh articles are published, which should be immediately considered for recommendation. To deal with this item cold-start problem, it is important to consider the actual content of items when recommending. Hybrid approaches are therefore often considered as the method of choice in such settings. In this work, we analyze the importance of considering content information in a hybrid neural news recommender system. We contrast content-aware and content-agnostic techniques and also explore the effects of using different content encodings. Experiments on two public datasets confirm the importance of adopting a hybrid approach. Furthermore, we show that the choice of the content encoding can have an impact on the resulting performance.

• Defining a Meaningful Baseline for News Recommender Systems, Benjamin Kille and Andreas Lommatzsch.Abstract: The analysis of images in the context of recommender systems is a challenging research topic. NewsREEL Multimedia enables researchers to study new algorithms with a large dataset. The dataset comprises news items and the number of impressions as a proxy for interestingness. Each news article comes with textual and image features. This paper presents data characteristics and baseline prediction models. We discuss the performance of these predictors and explain the detected patterns.

• Trend-responsive user segmentation enabling traceable publishing insights. A case study of a real-world large-scale news recommendation system, Joanna Misztal-Radecka, Dominik Rusiecki, Michał Żmuda and Artur Bujak. Abstract: The traditional offline approaches are no longer sufficient for building modern recommender systems in domains such as online news services, mainly due to the high dynamics of environment changes and necessity to operate on a large scale with high data sparsity. The ability to balance exploration with exploitation makes the multi-armed bandits an efficient alternative to the conventional methods, and a robust user segmentation plays a crucial role in providing the context for such online recommendation algorithms. In this work, we present an unsupervised and trend-responsive method for segmenting users according to their semantic interests, which has been integrated with a real-world system for large-scale news recommendations. The results of an online A/B test show significant improvements compared to a global-optimization algorithm on several services with different characteristics. Based on the experimental results as well as the exploration of segments descriptions and trend dynamics, we propose extensions to this approach that address particular real-world challenges for different use-cases. Moreover, we describe a method of generating traceable publishing insights facilitating the creation of content that serves the diversity of all users needs.

RMSE: Recommendation in Multi-Stakeholder Environments

• Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness (Himan Abdollahpouri and Robin Burke). Abstract: There is growing research interest in recommendation as a multistakeholder problem, one where the interests of multiple parties should be taken into account. This category subsumes some existing well-established areas of recommendation research including reciprocal and group recommendation, but a detailed taxonomy of different classes of multi-stakeholder recommender systems is still lacking. Fairness-aware recommendation has also grown as a research area, but its close connection with multi-stakeholder recommendation is not always recognized. In this paper, we define the most commonly observed classes of multi-stakeholder recommender systems and discuss how different fairness concerns may come into play in such systems.

• Simple Objectives Work Better (Joaquin Delgado, Samuel Lind, Carl Radecke and Satish Konijeti). Abstract: Groupon is a dynamic two-sided marketplace where millions of deals organized in three different lines of businesses or verticals: Local, Goods and Getaways, using various taxonomies, are matched with customers’ demand across 15 countries around the world. Customers discover deals by directly entering the search query or browsing on the mobile or desktop devices. Relevance is Groupon’s homegrown search and recommendation engine, tasked to find the best deals for its users while ensuring the business objectives are also met at the same time. Hence the objective function is designed to calibrate the score to meet the needs of multiple stakeholders. Currently, the function is comprised of multiple weighted factors that are combined to satisfy the needs of the respective stakeholders in the multi-objective scorer, a key component of Groupon’s ranking pipeline. The purpose of this paper is to describe various techniques explored by Groupon’s Relevance team to improve various parts of Search and Ranking algorithms specifically related to the multi-objective scorer. It is for research only, and it does not reflect the views, plans, policy or practices of Groupon. The main contributions of this paper are in the areas of factorization of the different abstract objectives and the simplification of the objective function to capture the essence of short, mid and long term benefits while preserv

• Recommender Systems Fairness Evaluation via Generalized Cross Entropy (Yashar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogin Kouki and Tommaso Di Noia). Abstract: Fairness in recommender systems has been considered with respect to sensitive attributes of users (e.g., gender, race) or items (e.g., revenue in a multistakeholder setting). Regardless, the concept has been commonly interpreted as some form of equality – i.e., the degree to which the system is meeting the information needs of all its users in an equal sense. In this paper, we argue that fairness in recommender systems does not necessarily imply equality, but instead it should consider a distribution of resources based on merits and needs.__
  We present a probabilistic framework based on generalized cross entropy to evaluate fairness of recommender systems under this perspective, where we show that the proposed framework is flexible and explanatory by allowing to incorporate domain knowledge (through an ideal fair distribution) that can help to understand which item or user aspects a recommendation algorithm is over- or under-representing. Results on two real-world datasets show the merits of the proposed evaluation framework both in terms of user and item fairness.

• The Unfairness of Popularity Bias in Recommendation (Himan Abdollahpouri, Masoud Mansoury, Robin Burke and Bamshad Mobasher). Abstract: Recommender systems are known to suffer from the popularity bias problem: popular (i.e. frequently rated) items get a lot of exposure while less popular ones are under-represented in the recommendations. Research in this area has been mainly focusing on finding ways to tackle this issue by increasing the number of recommended long-tail items or otherwise the overall catalog coverage. In this paper, however, we look at this problem from the users’ perspective: we want to see how popularity bias causes the recommendations to deviate from what the user expects to get from the recommender system. We define three different groups of users according to their interest in popular items (Niche, Diverse and Blockbuster-focused) and show the impact of popularity bias on the users in each group. Our experimental results on a movie dataset show that in many recommendation algorithms the recommendations the users get are extremely concentrated on popular items even if a user is interested in long-tail and non-popular items showing an extreme bias disparity.

• Bias Disparity in Recommendation Systems (Virginia Tsintzou, Evaggelia Pitoura and Panayiotis Tsaparas).Abstract: Recommender systems have been applied successfully in a number of different domains, such as, entertainment, commerce, and employment. Their success lies in their ability to exploit the collective behavior of users in order to deliver highly targeted, personalized recommendations. Given that recommenders learn from user preferences, they incorporate different biases that users exhibit in the input data. More importantly, there are cases where recommenders may amplify such biases, leading to the phenomenon of bias disparity. In this short paper, we present a preliminary experimental study on synthetic data, where we investigate different conditions under which a recommender exhibits bias disparity, and the long-term effect of recommendations on data bias. We also consider a simple re-ranking algorithm for reducing bias disparity, and present some observations for data disparity on real data.

• Joint Optimization of Profit and Relevance for Recommendation Systems in E-commerce (Raphael Louca, Moumita Bhattacharya, Diane Hu and Liangjie Hong). Abstract: Traditionally, recommender systems for e-commerce platforms are designed to optimize for relevance (e.g., purchase or click probability). Although such recommendations typically align with users’ interests, they may not necessarily generate the highest profit for the platform. In this paper, we propose a novel revenue model which jointly optimizes both for probability of purchase and profit. The model is tested on a recommendation module at Etsy.com, a two-sided marketplace for buyers and sellers. Notably, optimizing for profit, in addition to purchase probability, benefits not only the platform but also the sellers. We show that the proposed model outperforms several baselines by increasing offline metrics associated with both relevance and profit.

• A Multistakeholder Recommender Systems Algorithm for Allocating Sponsored Recommendations (Edward Malthouse, Khadija Ali Vakeel, Yasaman Kamyab Hessary, Robin Burke and Morana Fuduric). Abstract:__Retailing and social media platforms recommend two types of items to their users: sponsored items that generate ad revenue and non-sponsored ones that do not. The platform selects sponsored items to maximize ad revenue, often through some form of programmatic auction, and non-sponsored items to maximize user utility with a recommender system (RS). We develop a multiobjective binary integer programming model to allocate sponsored recommendations considering a dual objective of maximizing ad revenue and user utility. We propose an algorithm to solve it in a computationally efficient way. Our method can be applied as a form of post processing to an existing RS, making it widely applicable. We apply the model to data from an online grocery retailer and show that user utility for the recommended items can be improved while reducing ad revenue by a small amount. This multiobjective approach, which unifies programmatic advertising and RS, opens a new frontier for advertising and RS research and we therefore provide an extended discussion of future research topics.

Final Remarks

I think that for every practitioner or researcher engineer involved in Recommendation Systems, RecSys is a great conference to attend. There’s a great overlap with academia and industry where the first one pushes forward in terms of new methods, algorithms, and a reflexive attitude for important themes like bias and fairness; and the second one applies those methods using engineering and presents some results on battle tested applications using contextual bandits, click prediction and the combination between domain heuristics with optimization method in machine learning.