Serendipity-Oriented Recommender System with Dynamic Unexpectedness Prediction

Serendipity-oriented Recommender System with Dynamic Unexpectedness Prediction Yu Tokutake, Kazushi Okamoto The University of Electro-Communications 2023.10.02 IEEE SMC 2023 1 / 28

Recommender System (RS) Suggest items that match the user's preferences from a large number of items Evaluation Recommendation accuracy in many RS studies e.g. Precision, Recall, MRR, NDCG 2023.10.02 IEEE SMC 2023 2 / 28

Filter-bubble in RS Cause: Pursuit of high recommendation accuracy Result User boredom Lower satisfaction 2023.10.02 IEEE SMC 2023 3 / 28

Serendipity One of the beyond-accuracy metrics to solve filter-bubble No standard definition in RS Use some common components 2023.10.02 IEEE SMC 2023 4 / 28

Problem Unexpectedness Score Degree at the time of recommendation Use user profiles (e.g. user ratings and browsing records) Previous Studies 2023.10.02 IEEE SMC 2023 5 / 28

Our Solution Assume The unexpectedness score changes dynamically Calculate for each time window separated by a fixed period Concept 2023.10.02 IEEE SMC 2023 6 / 28

Related Works - Serendipity-oriented RS Classification of Algorithms [Kotkov+, 2020] Type Overview Previous Studies Reranking Sort accuracy-oriented recommendation list SOG [Kotkov+, 2020] Modification Modify accuracy-oriented RS UAUM [Zheng+, 2015] New New approach for improving serendipity SDNet [Ziarani+, 2021] Acceptance of Serendipity [Maccatrozzo+, 2017][Li+, 2019] Some users may prefer accuracy-oriented recommendations → Serendipitous recommendation to them may reduce user satisfaction 2023.10.02 IEEE SMC 2023 7 / 28

Related Works - RS using Time-series model HNATS [Yang+, 2021]: Calculate item popularity for each time window Four models were compared Double Exponential Smoothing (DES) Suitable for predicting data without seasonal but with level and trend Parameter Optimize: Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method → DES showed the best recommendation accuracy 2023.10.02 IEEE SMC 2023 8 / 28

Preliminary Relevance Items of interest to users (who give high ratings) Do not consider temporal changes Unexpectedness Items not popular, but different from existing user preferences Do not have seasonal, but have level and trend 2023.10.02 IEEE SMC 2023 9 / 28

Proposed System - Overview Reranking algorithm Apply to any accuracy-oriented RS and time-series prediction model 2023.10.02 IEEE SMC 2023 10 / 28

Identification Candidates Objective Create candidates : top- for the user items with highest predicted rating Method Accuracy-oriented RS Matrix Factorization (MF) Factorization Machines (FM) 2023.10.02 IEEE SMC 2023 11 / 28

Scores for reranking 2023.10.02 IEEE SMC 2023 12 / 28

Relevance Score Normalized predicted rating of accuracy-oriented RS Score Definition : the predicted rating of user for item : the maximum rating that users can give to items 2023.10.02 IEEE SMC 2023 13 / 28

Unexpectedness Score - Definition Weighted sum of between item unpopularity and differences from user preferences Score Definition : Proportion of users who did not rate in : Content difference between item in and 2023.10.02 IEEE SMC 2023 rated 14 / 28

Unexpectedness Score - Prediction Predict the value in from past scores Adopted the time-series prediction model in HNATS using the DES and BFGS methods Model Objective Function 2023.10.02 IEEE SMC 2023 15 / 28

The Risk of Pure Genre Diversity as an Acceptance Parameter Previous studies use genre diversity Possibly influenced by genre bias! 2023.10.02 IEEE SMC 2023 16 / 28

Acceptance Parameter of Unexpectedness Multiple genre diversity by the ratio of outside the main genre Parameter Definition : Set of genres, : Set of genres for items user : Rating history of user , : rated whose genre is In the example on the previous page, 2023.10.02 IEEE SMC 2023 17 / 28

Serendipity Score and Reranking Score Definition Reranking 1. Calculate of candidates 2. Sort them in descending order 3. Top- is the final list 2023.10.02 IEEE SMC 2023 18 / 28

Research Question RQ1: How does the performance of the proposed system change compared to before reranking or without using the acceptance parameter? RQ2: Does the proposed system (DES) accurately predict unexpectedness score in the recommendation time? 2023.10.02 IEEE SMC 2023 19 / 28

Experiment - Dataset MovieLens 100k (ml-100k), 1M (ml-1m) Dataset of movie rating Rating range : 0.5 ~ 5 Same settings as HNATS The number of time window In ml-1m, used ratings whose timestamp < 974,403,356 2023.10.02 IEEE SMC 2023 20 / 28

Experiment - Baseline MF, FM: Accuracy-oriented RS before ReRanking (No RR) RR (w/o param): The proposed system calculating serendipity score without using the acceptance parameter 2023.10.02 IEEE SMC 2023 21 / 28

Recommendation Performance - Evaluation Metrics Name Recall NDCG Unexp Ser 2023.10.02 Type Accuracy Overview The degree to which the prediction reproduces the correct answer Accuracy Normalized gain of the item's position (ranking) in the recommendation list Unexpected Serendipitous Proportion of items in the user do not expect when is calculated with the ground truth IEEE SMC 2023 22 / 28

Comparison Accuracy Improve by using the acceptance parameter RR (w/o param) < RR < No RR 2023.10.02 IEEE SMC 2023 23 / 28

Comparison Unexpectedness and Serendipity Serendipity showed the largest value in RR Unexpectedness : No RR < RR < RR (w/o param) Serendipity : No RR < RR (w/o param) < RR 2023.10.02 IEEE SMC 2023 24 / 28

Accuracy of Unexpectedness Score’s Prediction - Settings Metric Compared Method LAST : Use recent value ALL : Treat all history as equal 2023.10.02 IEEE SMC 2023 25 / 28

Accuracy of Unexpectedness Score’s Prediction - Result DES is better able to capture in large time window RSS : LAST < DES < ALL 2023.10.02 IEEE SMC 2023 26 / 28

Conclusion Research Question and Result 1. Impact of reranking and the acceptance parameter on recommendation performance → Improve serendipity by using both 2. Prediction accuracy of unexpectedness score → The usefulness could not be clarified, but the error was smaller for a dataset with a large time window Future Works Applies to other time-series model (e.g. TES) 2023.10.02 IEEE SMC 2023 27 / 28

Thank you! If you have any question, please contact [email protected] 2023.10.02 IEEE SMC 2023 28 / 28