Abstract:
The evaluation and selection of recommender systems is a difficult decision making process. This difficulty is partially due to the large diversity of published evaluation criteria in addition to lack of standardized methods of evaluation. As such, a systematic methodology is needed that explicitly considers multiple, possibly conflicting metrics and assists decision makers to evaluate and find the best recommender system among a given set of alternatives. This paper introduces Multi-Criteria Decision Making (MCDM) approach for evaluation of recommender systems. In particular, this paper proposes the use of Data Envelopment Analysis (DEA) approach, as a sub-category of MCDM, in order to solve this problem. Various DEA models are introduced and their applicability are illustrated. A real case of evaluation of recommender systems is used to demonstrate the approach.
Machine summary:
Evaluation of recommender systems: A multi-criteria decision making approach Babak Sohrabi1, Mehdi Toloo2, Ali Moeini3, Soroosh Nalchigar1 1.
As such, a systematic methodology is needed that explicitly considers multiple, possibly conflicting metrics and assists decision makers to evaluate and find the best recommender system among a given set of alternatives.
This paper introduces Multi- Criteria Decision Making (MCDM) approach for evaluation of recommender systems.
Keywords Data envelopment analysis, Evaluation, Metrics, Multi-criteria decision making, Recommender systems.
To address these problems, this paper introduces Multi-Criteria Decision Making (MCDM) approach for evaluation of recommender systems.
DEA is a widely used optimization, based on non-parametric method for efficiency evaluation of a set of similar units, usually referred to as Decision Making Units (DMUs).
In this paper, we introduce and show applications of various DEA models for evaluation of recommender systems in presence of multiple evaluation metrics.
This paper shows how DEA models could assist organizational decision makers to evaluate a set of recommender systems and to find the best system among the given alternatives.
Section 5 illustrates application of various DEA models for evaluation of recommender systems.
(2006) has extended minimax DEA model to identify a single most efficient DMU and used to evaluate layout design of manufacturing systems.
Toloo and Nalchigar (2011) proposed a new DEA model that is able to find the most efficient unit while the data of inputs and outputs of alternatives are imprecise (i.
The main contribution of this paper is to introduce and to illustrate applications of DEA models for evaluation and selection of most efficient recommender systems.
(
, 
, 
)
