- Good for new users to avoid cold-start
- Efficient to compute
- Creates a ‘single source of truth’ (Reddit topic rankings for example)
Summary Statistics
When computing rank for an item based on predictions, we should factor in;
- Confidence
- How confident are we in the prediction? (Think IMDB score)
- Risk Tolerance
- Is it a controversial item? High risk - high reward?
- Is it better to recommend safe options?
- Domain and business considerations
- Age
- System Goals
Damped Mean
Problem:
Low confidence with few ratings.
Solution:
Assume that, without evidence, everything is average.
Formula:
Thoughts: How would this differ to IMDB rating formula?
Confidence Intervals
Not clear if this is under assumptions of normal distribution to calculate or if I’m missing something.
Choice of bound affects risk/confidence;
- Recommend based on lower bound for conservative recommendations.
- Recommend based on upper bound for risky recommendations.
Wilson interval can be used for binomial distributions.
Domain Considerations: Time
Examples of ways to factor this in.
Hacker News formula
Net upvotes polynomially decayed by age (by hour).
Old items scored mainly by vote, newer items gets ‘freshness’ boost.
The alpha factor means popularity matters less the more popular a post is.
The gamma term means an exponential loss of interest for older items.
Multiplied by item penalty terms for types of posts not encouraged by site owner.
More up to date description of the formula available at Medium.
Reddit Algorithm (ca 2010)
Separates between the net upvotes score and a freshness score.
The freshness score uses the sign of the net upvotes, meaning that posts with a negative score gets buried.
Time term is seconds since Reddit epoch. In shortness, is means that age won’t change the rating of individual posts, but that newer posts over time continuously will have higher scores.
More up to date version of the algorithm and explanation at Medium
On Ranking and Confidence
“Wilson score interval”, Wilson’s score interval can be made into “the confidence sort” for binomial distributions. See How not to sort by average rating
Weak Personalisation
Beginning personalisation based on cohorts with different tastes, such as age, gender etc., including non-demographics that may be predictive.
- Find relevant demographics (using data exploration)
- Break down summary statistics by demographics (consider factorial)
- Next step could be to consider a regression model based on the demographics
Comments