Metrics¶

Metric¶

class recoder.metrics.Metric(metric_name)[source]¶

A Base class for metrics. All metrics should implement the evaluate function.

Parameters:	metric_name (str) – metric name. useful for representing it as string (printing) and hashing.

evaluate(x, y)[source]¶

Evaluates the recommendations with respect to the items the user interacted with.

Parameters:	x (list) – items recommended for the user y (list) – items the user interacted with
Returns:	metric value
Return type:	float

class recoder.metrics.Recall(k, normalize=True)[source]¶

Computes the recall @ K of the recommended items.

Parameters:	k (int) – the cut position of the recommended list normalize (bool, optional) – if True, normalize the value to 1 (divide by k) if k is less than the number of items in the user interactions, otherwise normalize only by number of items in the user interactions.

evaluate(x, y)[source]¶

Evaluates the recommendations with respect to the items the user interacted with.

Parameters:	x (list) – items recommended for the user y (list) – items the user interacted with
Returns:	metric value
Return type:	float

class recoder.metrics.NDCG(k)[source]¶

Computes the normalized discounted cumulative gain @ K of the recommended items.

Parameters:	k (int) – the cut position of the recommended list

evaluate(x, y)[source]¶

Evaluates the recommendations with respect to the items the user interacted with.

Parameters:	x (list) – items recommended for the user y (list) – items the user interacted with
Returns:	metric value
Return type:	float

class recoder.metrics.AveragePrecision(k, normalize=True)[source]¶

Computes the average precision @ K of the recommended items.

Parameters:	k (int) – the cut position of the recommended list normalize (bool, optional) – if True, normalize the value to 1 (divide by k) if k is less than the number of items the user interacted with, otherwise normalize only by number of items the user interacted with.

evaluate(x, y)[source]¶

Evaluates the recommendations with respect to the items the user interacted with.

Parameters:	x (list) – items recommended for the user y (list) – items the user interacted with
Returns:	metric value
Return type:	float

class recoder.metrics.RecommenderEvaluator(recommender, metrics)[source]¶

Parameters:	recommender (Recommender) – the Recommender to evaluate metrics (list) – list of metrics used to evaluate the recommender

evaluate(eval_dataset, batch_size=1, num_users=None, num_workers=0)[source]¶

Evaluates the recommender with an evaluation dataset.

Parameters:	eval_dataset (RecommendationDataset) – the dataset to use in evaluating the model batch_size (int) – the size of the users batch passed to the recommender num_users (int, optional) – the number of users from the dataset to evaluate on. If None, evaluate on all users num_workers (int, optional) – the number of workers to use on evaluating the recommended items. This is useful if the recommender runs on GPU, so the evaluation can run in parallel.
Returns:	A dict mapping each metric to the list of the metric values on each user in the dataset.
Return type:	dict