Performance evaluation of private universities based on balanced scorecard: empirical study based on Iran

Private universities are new generation of universities in Iran that their existences are less than 20 years. The research in this area was never completed about evaluating of these universities all these years. The aim of this research is to make a framework for evaluating these universities with Balanced Scorecard (BSC) and MCDM methods. Three MCDM methods are applied in this research. DEMATEL is applied for research on cause and effect relations of perspectives of BSC, ANP is applied to calculate weights of indices in perspectives and finally VIKOR for ranking universities that is selected as a case study. This research totally used 38 experts in two groups for evaluating of universities. Results of this research are: (1) Internal Process is the most effective perspective of BSC. (2) Brand, Academic Excellence, Product Quality, Student Satisfaction and Budget Control are five more important indices that calculated with ANP. (3) Shomal University of Amol is the best university among five universities that were selected for this research and they have been established more than ten years.


Introduction
The concept of Balanced Scorecard (BSC) was proposed by David Norton, the CEO of Nolan Norton Institute, and Robert Kaplan, a professor at Harvard University (Kaplan, Norton 1992). The BSC is a popular tool that is applied by many businesses to assess their performance in diverse aspects of their organization (Frigo et al. 2000). Davis and Albright (2004), the Balanced Scorecard (BSC) is a multi-attribute evaluation model that highlights the value of non-financial attributes. Kaplan and Norton (1996) presented four perspectives for performance measurement: financial, customer, internal business process and learning and development perspectives. By combining the financial, customer, internal process, and learning/growth perspectives, the Balanced Scorecard helps managers to understand many interrelationships and causal effects. This understanding can help managers to break free from traditional notions about functional barriers and ultimately lead to improved decision making and problem solving (Huang et al. 2011). The BSC framework also does not provide the quantitative and qualitative indicators how much each perspective contributes, even on the relative importance weight for each perspective and its corresponding indicators. However, the BSC framework does not provide guidance as to how these weights should be computed. Youngblood and Collins (2003) proposed that although the BSC provides valuable feedback on a variety of performance metrics, but those metrics did not consider the relative importance weigh and the issue of interaction and trade-offs between metrics and for these reasons quantities methods like MCDM methods applied with BSC.
The first private universities were established about 20 years ago in Iran and today there are more than 250 well-known institutes and universities in Iran. In this research BSC applies for evaluating of Iranian private universities that there is no any research about evaluation of these universities in Iran. The aim of this research is only to identify important indices in this area. The literature review revealed that MCDM methods in many researches were using (Fuzzy) AHP, (Fuzzy) ANP for calculating the weights of indices (Dytczak, Ginda 2009;García, Melón et al. 2010;Azimi et al. 2011;Timoshenko 2008) and in some researches they were used DEMATEL method base on cause and effect relation between perspectives and indices. There are numerous researches about applications of MCDM methods to BSC in many areas but there is not any research about evaluating of private universities in Iran. In this research three MCDM methods applied for evaluating of private universities. At first DEMATEL used for evaluating cause and effect relations between perspectives of BSC and in next step ANP applied for identifying important criteria and weights of them and finally VIKOR applied for comparing selected universities as case study and rank them. The process of this research is shown in Figure 1.   Economics and Management, 2013, 14(4): 696-714

Literature review
Owing to its ability to assist organizations or firms in selecting among alternative missions/visions, selecting among alternative strategies, and allocating resources to implement organizational strategies and objectives, AHP has been successfully applied in numerous BSC studies, including Huang (2009), Kim, H. S. and Kim, Y. G. (2009), Varma et al. (2008, Chan (2006), Leung et al. (2006), Fletcher and Smith (2004), Reisinger et al. (2003), Stewart and Mohammed (2001), and Liberatore and Miller (1998). AHP is a method enabling evaluation of both qualitative and quantitative variations in evaluating problems together.
AHP and ANP were used in developing the analytical structure of BSC model, which are multiple-criteria decision-making methods. AHP is a multiple-criteria decisionmaking method developed by Saaty (1996). The AHP method assumes that the factors presented in the hierarchical structure are independent; however, it assumes that it may be inappropriate in light of certain internal and external environment effects. Therefore, it is necessary to employ of analytic network Process (ANP) method (Lee 2007). The traditional financial method cannot fully reflect the performance of enterprises, as a result of which the Balanced Scorecard (BSC) method was developed. However, BSC also has some disadvantages. By giving power weights on indicators, ANP method can make up those disadvantages (Lee 2007). In ANP the hierarchical relation between criteria and alternatives is generalized to networks. Many decision problems cannot be structured hierarchically, because they involve the interaction and dependence of highlevel elements on lower-level elements (Saaty 2003). ANP uses to analyze the relative weights of performance indices.
The Multi-criteria Optimization and Compromise Solution (called VIKOR) is a suitable tool to evaluate each alternative for each criterion function (Opricovic 1998;Opricovic, Tzeng 2004, 2007Tzeng et al. 2005). The concept of VIKOR is based on the compromise programming of MCDM by comparing the measure of "closeness" to the "ideal" alternative. The multi-criteria measure for compromise ranking is developed from the Lp-metric that is used as an aggregating function in compromise programming (Yu 1973;Zeleny 1982). The most popular MCDM methods, VIKOR and TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), both apply the concept of compromise to solve the competing problem among the evaluation criteria and then rank the order of the alternatives (Opricovic, Tzeng 2004, 2007. However, the TOPSIS method is used to provide information on how to improve the gaps among the criteria so as to achieve the desired/aspired level and it cannot be used for ranking purpose due to its blind point proven by Opricovic and Tzeng (2004).
The DEMATEL method is applied to determine causal relationships and mutual influence among perspective (Wu et al. 2011). The process for building a strategy map could be viewed in a general body of a unified group decision making context. If we see the strategy map, as a structural modeling framework for making the cause and effect relationships among the strategic objectives, it is possible to deploy DEMATEL as a framework for structural modeling approach subject to the problem. The DEMATEL method gathers collective knowledge to capture the casual relationships between strategic criteria (Jassbi et al. 2011). DEMATEL was used for cause and effect relationship in each perspective of BSC for identifying the most important indices (Safaei Ghadikolaei et al. 2011).

Experts information
In this paper, 57 criteria were selected for establishing a BSC framework for private universities. We selected 30 experts for this research with target sampling. After a questionnaire, 22 criteria were selected for establishing BSC for universities. Information about experts is shown in Table 2.

Selected criteria for establishing BSC
Targets of plans and purposes (Duqrette, Stowe 1993) stated that performance indices are a kind of tool or indicators which are used for assessing performance of organizations. They could be quantification information and also could be a qualitative written description. Therefore, the selection of criteria is very significant for assessing the operating performance of organizations to achieve effective operational management and raise the efficiency of operation and create advantages and values to organizations. Ability to get access to "needed" courses and ease in getting "good" job Farid et al. (2008) C3. Academic excellence Quality of students admitted and quality of faculty Farid et al. (2008) C4. Service to the university Adequacy of participation in campus-wide activities Farid et al. (2008) C5. Brand Reputation of university Mehregan & Nayeri (2008) Internal process (P)

P1. Customized courses
If there are new courses or services that are created according to the demands of potential students Kaplan & Norton (1996) P2. Opera tional Business process If there are periodic reviews of operational business processes for improvement in order to close to the market and meet students' needs Kaplan & Norton (1996)

DEMATEL method
The DEMATEL, originated from the Geneva Research Centre of the Battelle Memorial Institute (Fontela, Gabus 1976;Gabus, Fontela 1973), aims to convert the relationship between the causes and effects of criteria into an intelligible structural model of the system (Liou et al. 2008). In a totally interdependent system, all criteria of the system are mutually related, directly or indirectly; thus, any interference with one of the criteria End of Table 3 S. Hashemkhani Zolfani, A. Safaei Ghadikolaei. Performance evaluation of private universities ... affects all the others, so it is difficult to find priorities for decision-making . The DEMATEL method is briefly described as follows: Step 1: Compute the average matrix. Each respondent was asked to evaluate the direct influence between any two factors by an integer score ranging from 0, 1, 2, and 3, representing "no influence", "low influence", "medium influence", and "high influence", respectively. The notation of x ij indicates the degree to which the respondent believes factor I affects factor j. For i = j, the diagonal elements are set to zero. For each respondent, an n × n non-negative matrix can be established as X k = k ij x     where k is the number of respondents with 1 ≤ k. H, and n is the number of factors. Thus, X 1 , X 2 , X 3 , … ,X H are the matrices from H respondents. To incorporate all opinions from H respondents, the average matrix A = [a ij ] can be constructed as follows: Step 2: Calculate the normalized initial direct-relation matrix. Normalize initial direct- Step 3: Calculate the total relation matrix. The total relation matrix T is defined as T = D(I D) -1 , where I is the identity matrix. Define r and c be n × 1 and 1 × n vectors representing the sum of rows and sum of columns of the total relation matrix T, respectively. Suppose r i be the sum of i throw in matrix T, then r i summarizes both direct and indirect effects given by factor i to the other factors. If c j denotes the sum of j th column in matrix T, then c j shows both direct and indirect effects by factor j from the other factors. When j = i, the sum (r i + c j ) shows the total effects given and received by factor i. That is, (r i + c j ) indicates the degree of importance that factor i plays in the entire system. On the contrary, the difference (r i -c j ) depicts the net effect that factor i, contributes to the system. Specifically, if (r i -c j ) is positive, factor i is a net cause, while factor i is a net receiver or result if (r i -c j ) is negative.
Step 4: Set up a threshold value to obtain the digraph. Since matrix T provides information on how one factor affects another, it is necessary for a decision maker to set up a threshold value to filter out some negligible effects, in doing so, only the effects greater than the threshold value would be chosen and shown in digraph. In this study, the threshold value is set up by computing the average of the elements in matrix T. The digraph can be acquired by mapping the dataset of (r + c, r -c).

The ANP method
ANP, also introduced by Saaty, is a generalization of the AHP. Saaty (1996) suggested the use of AHP to solve the problem of independence on alternatives or criteria, and the use of ANP to solve the problem of dependence among alternatives or criteria. Many decision-making problems cannot be structured hierarchically because they involve the interaction and dependence of higher level elements on lower level elements. This is a network system.
The process of ANP involves three sub steps and shown as follows (Shyur 2006): Step 1: Without assuming the interdependence among criteria, the decision makers are asked to evaluate all proposed criteria pair wise. The responses were presented numerically and scaled on the basis of Saaty's 1-9 scale. Once the pair wise comparisons are completed, the local weight vector W 1 is computed as the unique solution to AW 1 = l max W 1 . (2) Step 2: Where l max is the largest eigen value of pair wise comparison matrix A, The obtained vector is further normalized by dividing each value by its column total to represent the normalized local weight vector W 2 , all the criteria on each other by using pair wise comparisons as well. These pair wise comparison matrices are needed to identify the relative impacts of criteria interdependent relationships. The normalized principal eigenvectors for these matrices are calculated and shown as column component in interdependence weight matrix of criteria B, where zeros are assigned to the eigenvector weights of the criteria from which a given criterion is given.
Step 3: Now we can obtain the interdependence weights of the criteria by synthesizing the results from previous two steps as follows:

Introduction to VIKOR
The VIKOR method is a compromise MADM method, developed by Opricovic and Tzeng (Opricovic 1998;Opricovic, Tzeng 2002) started from the form of Lp-metric: The VIKOR method can provide a maximum "group utility" for the "majority"and a minimum of an individual regret for the "opponent" (Opricovic 1998;Opricovic, Tzeng 2002.

VIKOR steps
1) Calculate the normalized value: 2) Determine the best and worst values: For all the attribute functions the best value was * j f and the worst value was j f − that is, for attribute J = 1 -n, we get formulas (2) and (3) * j f = max f ij , i = 1, 2 … m, where * j f is the positive ideal solution for the jth criteria, j f − is the negative ideal solution for the jth criteria. If one associates all * j f one will have the optimal combination, which gets the highest scores, the same as j f − .

3) Determine the weights of attributes:
The weights of attribute should be calculated to express their relative importance.

4) Compute the distance of alternatives to ideal solution:
This step is to calculate the distance from each alternative to the positive ideal solution and then get the sum to obtain the final value according to formulas (7) and (8).
where S i represents the distance rate of the ith alternative to the positive ideal solution (best combination), R i represents the distance rate of the ith alternative to the negative ideal solution (worst combination). The excellence ranking will be based on S i values and the worst rankings will be based on R i values. In other words, S i , R i indicate L 1i and L 0i of L p -metric respectively.

5)
Calculate the VIKOR values Q i for i = 1, 2, …, m, which are defined as: where S -= max i S i , S * = min i S i , R -= max i R i , R * = min i R i , and v is the weight of the strategy of "the majority of criteria'' (or "the maximum group utility"). [(S -S * )/ (S --S * )] represents the distance rate from the positive ideal solution of the ith alternative's achievements. In other words, the majority agrees to use the rate of the ith.
[(R -R * )/(S --R * )] represents the distance rate from the negative ideal solution of the ith alternative; this means the majority disagree with the rate of the ith alternative. Thus, when the v is larger (> 0.5), the index of Q i will tend to majority agreement; when v is less (< 0.5), the index Q i will indicate majority negative attitude; in general, v = 0.5, i.e. compromise attitude of evaluation experts.

6) Rank the alternatives by Q i values:
According to the Q i values calculated by step (4), we can rank the alternatives and to make-decision.

Assessing the performance of the private universities of Iran
We employ four perspectives as a framework for assessing the standards of performance (Table 3). Based on this framework, the research uses DEMATEL for cause and effect relations between perspectives, ANP to weight the indexes and VIKOR to assess the performance of the five private universities that established more than ten years and selected as case study.

DEMATEL results
The results of cause and effect relations of perspectives presented in Tables 4 and 5 (Fig. 2). Table 4 show results of financial perspective; Table 5 shows results of customer perspective; Table 6 show results of internal process perspective and Table 7 illustrate results of learning & growth perspective. In this section used ideas of all 30 experts of Table 2.
*Internal Process is the most effective perspective of this research, according to results of Table 5.

ANP results
The results of the ANP demonstrate in Tables 7 and 8 which presented results of indices in perspectives. In this section and VIKOR results used 8 experts' ideas that Information about experts is shown in Table 6.
Final results demonstrate clearly in Table 8 with specific information.     As the results in Table 8 shown four important indices are in customer perspective that describes that customer perspective in the most important perspective in BSC for private universities. There is a meaning relation between customer and internal process perspectives because results of DEMATEL method shown that internal process is the most effective perspective in BSC. In section five (conclusion), we will describe more about ANP results.

VIKOR results
In this section according to results of results of ANP, VIKOR applied for final ranking of universities that are: 1. Imam Reza University (Mashhad), 2. Shomal University (Amol), 3. Shaikh bahaei University (Isfahan), 4. Mazandaran University of Science and Technology, 5. University of Science and Culture (Tehran). In this part according to section 4.2 and Table 6, eight experts participate in decision making. Life of private universities in Iran is less than 25 years old and that means these universities are so weak in infrastructure and facilities and they need more time to become top universities in competitive world. Most of these new private universities are small and are called Institute of higher education and most of them are less than 10 years old and it means this research selected universities that are more than 10 years old and are fairly developed in comparison with best universities of Iran and they are just 5 universities that selected as case study of this research because this kind of university didn't develop fairly in Iran and authors selected the best developed private universities in this research. The information about decision matrix of VIKOR method is shown in Table 9 and it is clear that information of decision matrix is based on group decision making and finally final results and ranking of alternatives based on VIKOR presented in Table 10.

Conclusions and discussions
By summarizing, this research has two different groups of experts that participated in two section of this article. First group include 30 experts that participated for selecting final model of BSC and indices and also for DEMATEL. Second group include 8 experts that help us for ANP and VIKOR section. The final model of BSC for private universities is illustrated in Table 3 that consist of 22 indices in perspectives. Results of DEMA-TEL that has been shown in Table 5, describe that Internal Process is the most effective perspective on other perspectives of BSC in among perspectives that universities should concentrate on that more than always because this perspective has a great influence on other perspectives. Weights and ranking of indices has been shown in Table 8 that results describe that (1. Brand, 2. Academic Excellence, 3. Product Quality, 4. Student Satisfaction, and 5. Budget Control) are five important indices of BSC for universities. One other point of ANP section is customer perspective is the most important perspective in BSC and it means that criteria of this perspective are more important than other perspectives that we can find out there is a clear relation exists between Internal Process and Customer perspectives because from the base internal process perspective prepared to develop customer perspective. Finally VIKOR applied for comparison universities that selected as a case study and ranked them. Results have been shown in Table 9 (1. Shomal University, 2. University of Scinece and Culture, 3. Mazandaran University of Science and Technology, 4. Shaikh Bahaei University, and 5. Imam Reza University).
Authors suggest that (1) each student likes to study in a famous and high quality university because it gives them the sense of confidence and they can be more relaxed about their future. Authors suggest that to these universities develop their plans with other organizations and industries. International participates like held international conferences can helpful for developing brand of universities.
(2) Academic excellence is an aim while establishing each university and some important points exist that universities develop their brands and can hire better academic staff and this plan can be helpful for attraction students with better qualities. Appropriate relation between private universities with high quality universities in country and world is another plan for developing their brands.
(3) Hiring expert personnel in higher education management, educational class for developing human resources are effective ways in educational planning management and have good influence to increase level of Product Quality. (4) Private universities are funded with fees that student pay for their educations then we can see logical relation between student satisfaction and existence of these universities. There are many factors which have been influenced in this research and out of this research that are not related to this research. (5) Budget control is one of easiest and important factors of existence and development of an organization. Budget control should be assigned with strategic aims and according to situation; aims policy of organization should be used.
Authors also suggest that in future researches other new methods applied with BSC like SWARA (Kersuliene et al. 2010) that can be used rather than ANP. Results of this research can be comparisons with Fuzzy ANP and Fuzzy VIKOR. Finally this research can be useful as a framework for private universities in Iran and all around the world.