Why ranking sits between retrieval and the user
Retrieval gives you a hundred candidates. The ranker decides which twelve to put in front of the user, in what order. The data is simple: positives are real (customer, article) purchases, negatives are random pairs the customer never engaged with. The model learns the difference.
The ranking stage
From retrieved candidates to a sorted list. Same customer features the ranker saw during training power the request at inference.
def compute_ranking_dataset(trans_df, articles_df, customers_df, negative_ratio=10):
df = trans_df.select(["article_id","customer_id"]).join(
customers_df.select(["customer_id","age"]), on="customer_id", how="left"
)
positive_pairs = df.with_columns(pl.lit(1).alias("label"))
n_neg = positive_pairs.height * negative_ratio
article_ids = df.select("article_id").unique().sample(n=n_neg, with_replacement=True, seed=2).get_column("article_id")
customer_ids = df.select("customer_id").sample(n=n_neg, with_replacement=True, seed=3).get_column("customer_id")
ages = df.select(["age"]).sample(n=n_neg, with_replacement=True, seed=4).get_column("age")
negative_pairs = pl.DataFrame({"article_id": article_ids, "customer_id": customer_ids, "age": ages})\
.with_columns(pl.lit(0).alias("label"))
ranking_df = pl.concat([positive_pairs, negative_pairs.select(positive_pairs.columns)])
return ranking_df.join(articles_df.unique(subset=["article_id"]).select(keep_cols), on="article_id", how="left")The training dataset for the ranker. Positives come from transactions. Negatives are sampled from the catalogue.
A real customer browsed a lot more than they bought. A 10-to-1 ratio reflects that imbalance and gives the ranker enough negatives to learn a useful boundary. CatBoost handles the imbalance via scale_pos_weight so the model does not just always predict zero.
Quiz: Quiz
Loading practice…