We built a simulator that accounts for the dynamic and interactive nature of recommender systems. Off-the-shelf RL baselines are not able to solve the tasks we propose.
We discussed offline evaluation for RL-based recommender systems and noted that the most common evaluation protocol, i.e., next-item prediction, is unsuited to such approaches.
Learning to recommend slates with RL is hard, because of the combinatorial action-space. Instead of making assumptions on user behavior, we tried taking actions in the latent space of a VAE, which then decodes slates.