论文标题
位置偏见在市场搜索引擎中的测量和应用
Measurement and applications of position bias in a marketplace search engine
论文作者
论文摘要
搜索引擎通过选择和排名结果列表来故意影响用户行为。用户由于其出色的位置和通常是最相关的文档而获得最高结果。搜索引擎排名算法需要确定相关性，同时纳入搜索引擎本身的影响。本文介绍了我们在Thumbtack了解排名的影响的努力，包括随机计划的经验结果。在消费市场的背景下，我们讨论了模型选择，实验设计，偏置计算和机器学习模型适应的实用细节。我们包括一个新的讨论，即对排名偏见如何不仅影响标签，还影响模型特征。随机计划导致改进的模型，动机的内部方案分析并启用了面向用户的方案工具。
Search engines intentionally influence user behavior by picking and ranking the list of results. Users engage with the highest results both because of their prominent placement and because they are typically the most relevant documents. Search engine ranking algorithms need to identify relevance while incorporating the influence of the search engine itself. This paper describes our efforts at Thumbtack to understand the impact of ranking, including the empirical results of a randomization program. In the context of a consumer marketplace we discuss practical details of model choice, experiment design, bias calculation, and machine learning model adaptation. We include a novel discussion of how ranking bias may not only affect labels, but also model features. The randomization program led to improved models, motivated internal scenario analysis, and enabled user-facing scenario tooling.
