Robustness analysis in a TODIM-based multicriteria evaluation model of rental properties

AbstractA new robustness analysis framework is proposed where robustness of a solution in a decision aiding process is measured as the distance from that solution to an expected outcome, chosen by the decision-aiding analyst. The framework is explained by the application of the TODIM method of multicriteria decision aiding to the problem of predicting rental ranges for properties in a Chilean city. Therefore, the robustness concern concentrates on changes in criteria weights as well as in trade-off rates, as they are defined in the method. Two main contributions are introduced: a local robustness measure, defined in terms of a distance among rankings; and a global robustness measure, as an adaptation of the minimax-regret rule to select a global robust solution, i.e. a ranking produced by TODIM.

interact in a decision problem, some reflecting more or less arbitrary choices of the decision analyst (method, modelling technique, qualitative and quantitative evaluation scales) and others concerning external uncertainties (parameter values, evaluation of alternatives over the criteria, preference instabilities and biases). In this sense, it has been proposed that the robustness concern needs to be explicit in a problem such that the robustness analysis is driven by a specific aim (Aissi, Roy 2010).
In a previous work, we have conceived robustness as one of three dimensions of flexibility: it refers to the adjustment capability a system has to adapt to its environment (Pereira, Paulré 2001). The other dimensions are time and effort of adjustment, raising the dynamic nature of flexibility in most domains (Pereira et al. 2011). In this framework, in order to evaluate the adjustment capability, the following aspects need to be identified: -A system, representing the object of analysis; -A period of time where the analysis is to be performed; -The observer, or analyst, who is the person judging about the system, the environment and its properties; -The current state of the system, as defined and assessed by the observer; -The logic of the system, as perceived by the observer, determining the states of the system; -An expected state for the system, as represented by the logic of the system. According to these aspects, the local flexibility of a system is defined as the capability to adjust its current state to an expected state (Pereira, Paulré 2001). Therefore, robustness is the property that an observer assesses from a system exhibiting adjustment among a series of actual and expected states. As a consequence, robustness analysis concerns the activity aiming to evaluate flexibility of a system, when a given environment is considered. In this sense, robustness is different from sensitivity, as it is not concerned with the stability of a specific state of a system when it faces uncertainty factors. Instead, it is about the capability of the system to reach the expected states.
In fact, let us consider a problem situation where a decision d needs to be made (Rosenhead et al. 1972). In addition, suppose that the long-term possible solutions are contained in a set S, but only E S ⊂ outcomes are acceptable or good solutions. If the set of acceptable solutions from d is d S , then robustness is defined as ( ) / ( ) d d R n S n E = , where (.) n is the number of elements in the argument set. Interestingly, what is acceptable depends on "some combination of satisficing criteria" (op.cit., 419 p.), which implies that it needs to be explicitly identified by some actor in the decision process. In our framework, E orresponds to the set the expected states of the process. However, we have argued that robustness metrics may change depending on the way that the observer models the system's logic (Pereira et al. 2011).
The framework described has been applied to the analysis of flexibility in different kinds of systems (Pereira, Paulré 2001;Pereira et al. 2011): inventory management, sequential decision problems, setup flexibility in manufacturing systems, information and telecommunications technology investment, among others. In general, flexibility is conceived as a property revealed in the interaction between a system and its environment, subject to uncertainty conditions.
Here, we introduce this framework in a replication study of an application of the TODIM method to the evaluation of residential properties carried out in Brazil ). The aims of the former study were: (i) obtaining a ranking of properties, based on a number of evaluation criteria and some reference alternatives with known rental values; and (ii) finding rental values for properties where rental was not appraised in advance. The purpose of inserting the reference properties was to establish a range of values for the other properties under analysis. Analysis on a restricted set of parameters was also made in order to assert conclusions. As suggested by these two authors, the final ranking and rental ranges found for residential properties were very precisely established by the application of the TODIM method ). Our purpose with the replication study is evaluating the robustness of a solution produced by TODIM, evaluated as the degree of congruence between such outcome and a reference solution.
Methodologically, the study presented in this article is based on the following phases of a decision aiding process (Bouyssou et al. 2006): -Problem Formulation: The problem formulation is a statement defining the triplet A,V, Γ = Π , where A is a set of actions, in our case rental properties; V is a set of points of view (e.g. dimensions) considered to evaluate elements of A ; and Π is a statement on what would be done with elements of A, in this case the ranking of properties; -Evaluation Model: Formally, an evaluation model is a tuple M A,C,U,R = , where C is a set of criteria, eventually derived from V, allowing the evaluation of elements of A in terms of each criteria; U models the uncertainty regarding the available information in A C × , and R is an aggregation logic defining the way that the information concerning A and C is operated in order to obtain a global conclusion by solving the problem Π. In this case, such logic is provided by the TODIM method. The evaluation model produces a process output, i.e. the ranking of rental properties; -Recommendation: The output of the evaluation model is translated into the decision maker's language, verifying that it is technically sound and deployable in the decision maker's setting and processes. An important issue to take into account is robustness analysis. This analysis consists of investigating "how good will the solution be under different scenarios and combinations of the parameters" (Bouyssou et al. 2006). Robustness analysis is indeed our main focus in this article. The article is organized as follows. In Section 1 the main structure and features of the TODIM method are introduced. Next, in Section 2, the application of TODIM to the evaluation of rental properties is described and the ranking of alternatives used for purposes of the analysis is built. In Section 3, the robustness analysis technique is introduced, where uncertainty regarding weights and implications for robustness are discussed. Conclusions are presented in final section.

The toDiM method
A number of discrete multicriteria methods have been developed and are in current use for solving various complex evaluation and decision making problems (Balezentiene, Kusta 2012;Baležentis et al. 2012;Mulliner et al. 2013;Ginevičius et al. 2008;Vaidogas et al. 2007). However, unlike most other discrete multicriteria decision aiding methods the TODIM method is founded on Prospect Theory (Kahneman, Tversky 1979;Tversky, Kahneman 1992). TODIM is an acronym in Portuguese for Interactive and Multicriteria Decision Making.
The TODIM method with its prospect theoretical foundation first appeared in the literature in the early nineties (Gomes, Lima 1991). Today different extensions of the TODIM method are available that can deal with either fuzzy or hybrid data (Fan et al. 2013;Krohling, Souza 2012;Chen et al. 2010). Applications of TODIM have covered a wide spectrum of situations such as real estate evaluation (Moshkovich et al. 2012(Moshkovich et al. , 2011), business planning (Rangel et al. 2011, energy resource management (Gomes et al. 2010, hospital management (Nobre et al. 1999), and road planning (Gomes, Lima 1991). The fact that TODIM is founded on Prospect Theory is reflected in the shape of its value function. This is essentially the same as the gains/losses function of Cumulative Prospect Theory, where gains and losses are always established with respect to a reference point (Tversky, Kahneman 1992).
Differently from other multicriteria methods, TODIM relies on a value function reproducing the gain and loss attitude of a single decision maker on each criterion, following the prospect theoretical paradigm (Gomes, Lima 1991Nobre et al. 1999;). Next, a global multiattribute value function aggregates measures of gains and losses over all criteria in a problem in the form of an additive difference function, which summarizes arguments in favour and against an alternative when compared to another, similar to what PROMÉTHÉE does (Brans, Mareschal 1990). Such a function is supposed to account for dominance relations among all pairs of alternatives, through a pairwise comparison process. Finally, by normalizing the dominance function the method leads to a global ordering of the alternatives.
Let us consider as the set of alternatives to be ordered and … the set of m quantitative or qualitative criteria. Let us also assume that one of these criteria can be considered as the reference criterion, as explained below. Although TODIM can be used for qualitative as well as quantitative criteria, the evaluation of the alternatives in relation to all criteria is a matrix of evaluation, where the values are all numerical. If needed, verbal scales of qualitative criteria are converted to cardinal ones. Let nm P P =     , where 0,1 nm P ∈     , be the matrix of normalized alternatives' scores against criteria.
TODIM requires the decision-maker to provide inter-criteria information. First, the importance of each criterion c expressed as c w , needs to be fixed. Next, a reference criterion must be defined as the one with the highest importance, let us say r c . Then, the relative weight of each criterion c becomes / cr c r w w w = . Once this information is available, the following utility function is defined: (1) Technological and Economic Development of Economy, 2013, 19(Supplement 1): S176-S190 S179 This is a piecewise function modelling the decision maker's preference. When ic jc P P − is positive, gains are experienced and the concave form of utility denotes the aversion to risk by the decision maker. On the contrary, for negative differences, losses arise and the convex form of utility reflects propensity to risk, accordingly to prospect theory. In such case, ϑ is interpreted as an attenuation factor of losses, which shapes the prospect theoretical value function in the negative part of (1). A net flow may be calculated as a global dominance relation between each pair of alternatives. It is expressed as follows: In TODIM, (2) is aggregated in order to obtain a global value for each alternative, in the following way: Expression (3) represents a normalized global performance for the alternative i A when it is compared against all other alternatives in terms of preferences. Determining i ξ leads to the ordering of alternatives. As a consequence, a complete pre-order is then induced, which also means that no incomparability is allowed and the indifference relation among alternatives is transitive (Roy 1996).

Rental evaluation of residential property
An evaluation of residential property available for rent is described in this section. In the ) study, the TODIM method was used to order the alternatives of residential property for rent in the city of Volta Redonda, in the State of Rio de Janeiro, Brazil. One of the aims of that study was to define the rental values, using information about previously evaluated properties, included in the set of alternatives. In the present article we describe a replication developed during the second semester of 2010 in Talca, a medium size city in Chile, 250 kilometres to the south of Santiago, the capital of Chile. This is an emerging university village surrounded by agriculture and industrial settings. In the last two decades Talca has benefited from special conditions by inviting professionals, technicians and students to move there. In addition, Talca was one of most affected cities by a recent 27 th February 2011 earthquake, destroying a great part of the old city. As a consequence, the city is experiencing an explosive development of properties both for buying and rental, which makes this an interesting place for our study.
Operationally, we are looking for a method evaluating conditions under which an outcome from using TODIM effectively fits an expectation. In this paper, as an illustration procedure, we ask if there will be weights such that the ordering is compatible with a weighted qualified majority. Evidently, this may not be the case in a number of situations, but we present a very simple method for finding ranges of values for weights making TODIM results compatible with a given ranking. Differently from ), we do not explore other kinds of uncertainty mainly assuming that both evaluations on criteria and decision maker's preferences are well defined and modelled. In a future work, we will present a study concerning these aspects.
The analysis is twofold: first, an application of TODIM is developed for eight properties, based on a family of criteria composed by nine conflicting criteria. Second, using the ranking of alternatives (i.e. rental property) in the first phase, two properties, with known rental values, are selected as reference alternatives. The family of criteria used in this case is presented in Table 1. In the original study by , the set of criteria was defined, except 7 c , 8 c and 9 c , which we include here because they are characteristics usually appearing in published advertising of a rental property. In Table 2 evaluation of alternatives according to different criteria are shown; numerical scales are then used, codifying the original ones. We call this structure the rough evaluation matrix, because it still needs to be normalized, as explained above, in order to have a unique scale on which the dominance function (2) may be calculated. In Table 2 the importance of every criterion, c w , is also shown; it can be verified that the most important criterion is 1 c .
A realtor aided in the definition of the first set of values for the relative importance of criteria. Further, this relative importance of criteria was discussed with other specialists in the field of property evaluations. In addition, the eight alternatives were randomly chosen among properties published in local journals; the purpose was to verify in which extent the method correctly predicted the ranking among them. Alternatives were evaluated using the information available on the respective advertisements. For imprecise criteria 1 3 4 8 9 ( , , , , ) c c c c c , alternatives were evaluated in several rounds, until no questioning concerning the relative importance of criteria remained.
By using a value 1 θ = , the final results for this evaluation matrix are presented in Table 3, where a complete pre-order is defined by the global value i ξ . Now, let us suppose that rental values for properties 1 A and 5 A are known in advance. This ranking could therefore be used to establish possible ranges for rental of the remaining alternatives. Actually, in Table 3 the predicted and published rental values are shown, which confirms that TODIM ranks properties following an order near to the order induced by real rental values.

Robustness analysis
Now, let us consider the TODIM method in terms of the described flexibility framework. First of all, we need to distinguish the following features in a specific situation where this method is applied: -Given a specific decision problem situation, a system is implemented in order to aid the decision maker to understand, describe or find some interesting conclusions. In general, the system refers to a decision aiding process covering the Problem Formulation, Evaluation Model and Recommendation phases, described above; -The period of time extends while the decision aiding process remains open for the problem situation; -The system's logic includes every element, and relationship among them, on the decision aiding process phases. In particular, it includes the TODIM method and specificities (decision aiding rationale, preference modelling, preference aggregation features, etc.); -The decision analyst is the person who drives the decision aiding process, implementing the system's phases; -Despite the fact that a state of the system may be identified in every step of the decision aiding process, we may consider a current state as the output of the Evaluation Model phase, in our case, a ranking of alternatives; -In the application of TODIM, some reference alternatives are included in the evaluation model; the purpose is that the preference modelling adjusts as good as possible to real expectations. Thus, let us assume that the expected state corresponds to the real order, obtained from the published rental values shown in Table 3 Notice that the decision analyst could choose other or even several distinct expected states. It is well-known that, given a set A of alternatives, a pre-order on it is representable if it exists a utility function : Barba-Romero 2000). TODIM builds such a function using the information coming from evaluations of alternatives on criteria, the set of weights and the attenuation factor. Therefore, let us assume that the robustness concern in this case refers to the conditions where the expected state (the pre-order induced by rental value of properties) is reproduced by the system. Actually, from Table 3 we see that the expected and current states are not the same because 6 A is displaced from the fourth to sixth position in TODIM's ranking. Thus, our inquiry consists of determining how different the orders are, which naturally refers to some distance metrics among rankings. Two classical approaches have been applied to do this: the Kendall tau and the Spearman ranking index (Chen, Popovich 2002). While the former measures the total number of pairwise inversions among rankings, the latter measures the total displacement of elements from the identity ranking. In this work, we will focus on the former.
Henceforth, let us define e O as the complete pre-order induced on A, reflecting the order of published rental value for properties. Equally, let O be the complete pre-order resulting from the application of the TODIM method. The position of an alternative i A on a given ranking will be denoted as ( ) O i . Therefore, the Kendall tau between e O and O may be expressed as: It is easy to verify that ( ) , 2 e K O O = . However, this measure does not take into account the relative position and relevance of alternatives being inverted. In particular, it would not be the same impact when two alternatives placed below in the ranking are inverted, versus two other placed near the top of the pre-order. Kumar and Vassilvitskii (2010) have proposed a modified version of the Kendall tau, taking into account the position, weight and similarities among elements in a ranking such that inversions do not have the same cost depending on the importance of changes. Although these authors claim that this modified measure encompasses purposes of modern information retrieval systems, we believe that it also provides interesting features in decision analysis where the ranking problem is a very common situation. This new measure is defined as: where: i r represents the relevance of element i; i p is the position weight of swapping this element with an element in the next position in the ranking; and ij D is the dissimilarity between elements i and j. Kumar and Vassilvitskii have demonstrated that this measure is invariant to re-labelling of elements in the ranking, provided that relative weights are preserved. Furthermore, it is very easy to see that it is reduced to the traditional Kendall tau when the relative weights are all set to the unit. In the case of TODIM, we will consider 1, ij D i j = ∀ ≠ . Naturally, relevance of alternatives in the ranking is captured by the ξ value. Though, different functions may be used to represent relevance in terms of this value, we illustrate this point defining 1/ As in Kumar and Vassilvitskii (2010), we define: Therefore, the modified distance value for our case is ( ) This small value, as compared to the usual Kendall tau, could be explained by the fact that the difference between the expected and current permutations is observed in medium-low positions in the ranking. In fact, in these positions the factor i r is very small when compared with the respective values in the first or second place in the pre-order.
Are there other sets of values for the weights of criteria bringing the same solution, i.e. the same distance from an outcome to the expected pre-order? In fact, for the sake of simplicity, we could assume that if TODIM builds a ranking among alternatives, there will be weights such that the ordering is compatible with a weighted qualified majority (Bouyssou et al. 2006), that is, a binary relation  satisfying the property (8), where it is usually accepted that 1 ,1 2   λ∈     and c S is a binary relation defining a weak order.
In practice, TODIM defines such relation in terms of two binary relations: one, an asymmetric, transitive and non-reflexive relation called strict preference, which we may denote by c P ; the other, a reflexive, symmetric and transitive relation called indifference, which we denote by c I . Therefore, i c j i c j i c j

A S A A P A A I A
In consequence, (4) expresses that there is a coalition of criteria strong enough to assert that i A is as good as j A . For instance, let us assume that we have found 2 1 A A  and criteria 1 2 3 6 7 8 9 , , , , , , c c c c c c c are in favor of 2 A , that is, each one verifies either a strict preference or an indifference situation against 1 .
A Thus, we should have 1 3 6 7 8 9 w w w w w w + + + + + . If TODIM was compatible with the weighted qualified majority, then 2 1 2 1 A A ⇒ ξ ξ   . Unfortunately, this relation cannot be proved in every case. So, we use the weighted qualified majority as a generator procedure aiding to find some alternative solutions, i.e. rankings, that are consistent with the expected pre-order.
In that way, the decision maker may require additional restrictions for weights, in order to represent her/his preferences. For instance, she/he could enforce that one criterion k c is more important than another i c , or that a coalition of criteria should not be less important than another. Let  (Mousseau et al. 2001), where 1 2 , C C are two coalitions of criteria and α is a constant. Application of a simple procedure allows finding different sets W preserving the current pre-order, shown in Table 3. In Table 4, weights and the respective modified Kendall distances to the expected pre-order are presented. A global robustness assessment has a wider scope. It compels the decision analyst to define a series of expected states, each one defined as an experiment setting to test a robustness concern in a specific situation. Here, we conceive a situation as a context, a combination of circumstances in a given period of time, where elements are recognized because of the properties and relations they manifest on it (Barwise 1981). In the decision aiding process, a problem situation is a context where actors are involved, and their concerns and resources are identified (Bouyssou et al. 2006). For instance, in our problem a context is established when actors, their priorities, concerns and resources remain defined, jointly with other contextual aspects as age, education, gender, city, professional interests, and geographic features, among others.
In terms of robustness analysis, one should expect that each context bring its own characteristic uncertainty sources. We will assume that, given a context, at least one expected state might be defined. Therefore, the decision analyst may test how close the outcomes of the decision aiding method (e.g. TODIM) are from the different expected states, when uncer-tainty is present. In terms of procedure, it means assessing the local robustness for each one of selected contexts; then, the single evaluations may be aggregated into a global measure. Let Actually, we could select a robust pre-order applying the minimax-regret criterion (Liesiö et al. 2007), a measure that we could define here as the minimum value among the worst-case distance difference between a given pre-order and the best ranking, relative to an expected state (Kouvelis, Yu 1997  . On the contrary, the less a pre-order is present as a robust solution, the more it is penalized. Evidently, the generation of sets e Ω is a critical issue in the proposed framework. However, some features may be mentioned. First, if we accept that a context is a singular identifiable situation, bringing its own uncertainty characteristics, then it makes sense to consider the global robustness analysis as a cross-contexts task. Second, an elementary question is what does count as an expected state. In this paper, we intentionally consider this kind of state as an external definition, but strictly speaking, we could set an expectation as one of pre-orders generated in a given context. Consequently, an expected state could be chosen a posteriori, which is as a by-product of TODIM. Third, the proposed framework intends to look for robust solution instead of testing the TODIM's method robustness. In this sense, we believe that the same framework could be applied employing other methods commonly used in ranking problems. Finally, we remain close to the Rosenhead et al. (1972) conception of robustness, and then the set of expected states is a choice made by the decision analyst who establishes what is acceptable or good as an outcome.

conclusions
A flexibility framework has been proposed where robustness of a solution in a decision aiding process is measured as the distance from that solution to an expected outcome, chosen by the decision-aiding analyst. This framework has been applied to a replicated TODIM-based multicriteria evaluation model for predicting rental ranges of properties in a Chilean city. The framework is illustrated by applying the process on the assumption that a ranking of alternatives built with TODIM would be robust if it fits to an expected ranking. We have applied the robustness analysis procedure to a feasible solution, taken as an initial ordering of rental properties. This process searches for local robustness conditions, instead of globally robust solutions for a problem. As an illustration, the technique for analysis searches for weights of criteria such that the first pre-order built by TODIM is compatible with different weighted qualified majorities. Next, a distance measure, issuing from the Information Retrieval domain, is introduced in order to calculate the distance between a pre-order built by the method and an expected ranking, stated a priori. However, an expected state could be chosen a posteriori, among solutions produced by TODIM.
In order to undertake a global robustness analysis, we have proposed to consider a series of expected states, as they are defined in different problem situations or contexts. Next, local robustness analysis concerning each expected state might be done. Consequently, a global measure has been introduced, which adapts the minimax-regret rule in the case of our framework. This measure allows us to find a globally robust solution, among different rankings produced in the analysis.
As a future research, we are interested on the application of our framework to the setting of sequential decision-making problems. In such case, the set of expected states could be issued from progressive decision steps, each one concerning a context. When time is considered, the dynamic nature of flexibility is revealed and robustness should measure the system capacity to adapt to the flood of uncertain conditions. acknowledgements This work has been partially supported by the following grants: (1) CNPq/Brazil, Research Project No. 310603/2009-9;and (2) Fondecyt/Chile, Project No. 1130455.