AN ANALYTIC NETWORK PROCESS BASED RISK ASSESSMENT MODEL FOR PPP HYDROPOWER INVESTMENTS

The number of public-private partnership (PPP) projects has gone up especially in developing countries. The risk assessment of PPP projects is essential in ensuring project success. The objective of this study is to develop an Analytic Network Process (ANP) based risk assessment model for hydropower investments, and a tool to facilitate quantification of risk ratings based on this model. The results show that the three most important risk factors that affect the overall risk rating of a PPP hydropower investment are legal risks, contractor/subcontractor risks, and operator risks. In addition, the three most important risk clusters were identified as stakeholders, government requirements, and resources, whereas market was the least important cluster. The tool that measures the risk rating of a PPP of hydropower project was tested on ten real cases, and satisfactory results were obtained in terms of its predictive capability. The contributions of this research include (1) identification of the risk factors and clusters of factors associated with PPP hydropower investments; (2) determination of the priority of each risk factor and cluster; (3) development a tool that guides the investors through the risk assessment of PPP hydropower investments.


Introduction
A Public Private Partnership (PPP) is a long-term cooperation between a public agency and the private sector to provide public services (Liu et al., 2015). It is a project delivery system where a public agency benefits from the private sector's financing opportunities and operations expertise in fulfilling public needs (Mazher et al., 2018;Ashuri et al., 2012). PPP has been widely used in developing countries where governments do not have enough funds to provide better public services (Nguyen et al., 2018). More than 6,000 PPP projects have been performed in developing countries in the past 25 years (Ahmadabadi & Heravi, 2019). PPP projects include a higher degree of risk compared to traditional projects due to its complex nature (Wu et al., 2018). Therefore, the risk assessment of PPP projects should be performed as realistically as possible to ensure project success.
In this study, an ANP-based risk assessment model for PPP hydropower investments was developed for realistic risk assessment and for the prioritization of risk factors associated with hydropower investments. The objective is to develop an Analytic Network Process (ANP) based risk assessment model for PPP hydropower investments, and a tool to facilitate quantification of risk ratings based on this model. The section following the Introduction provides background research. The subsequent second section describes the methodology of the ANP-based risk assessment model for PPP hydropower investments. In this section, the risk factors and clusters that affect PPP hydropower investments are clarified, general information about ANP is described, the prioritization of risk factors and clusters is performed, the development of a risk assessment tool is introduced. The third section discusses the results of the prioritization process. The fourth section presents the testing of the risk assessment model. Finally, the conclusion and the limitations of this study are reported in the last section.
Several research studies focus on selecting an appropriate risk assessment method for prioritizing the risk factors. For example, Wu et al. (2018) proposed a threedimensional risk assessment model that involves probability, losses, and uncontrollability. Kuru and Artan (2020) proposed a canvas model to perform risk assessment and performance evaluation in PPP projects where they prioritized the PPP risk factors by using Analytic Hierarchy Process (AHP). Tah and Carr (2000) developed a model for qualitative risk assessment by using a hierarchical risk breakdown structure. Xu et al. (2010) prepared a fuzzy synthetic evaluation risk assessment model for PPP projects in China. Wu et al. (2017) developed a multi-criteria fuzzy synthetic evaluation framework for public-private partnership straw-based power generation projects. Ameyaw and Chan (2015) assessed the risk level of PPP water supply projects in developing countries using fuzzy synthetic evaluation approach. Li et al. (2017) proposed a fuzzy-grey comprehensive method to perform the risk assessment of China's overseas investments in oil refineries.
A hierarchical risk breakdown structure is constructed and an appropriate multi-criteria decision-making method is used to perform risk assessment in most factor-based studies. Unfortunately, the interrelations between the risk factors are usually ignored in these studies. However, in practice, interrelations do exist between the risk factors. For instance, the demand risk is normally correlated with the risk of economic stability. Therefore, it is obvious that using a network structure is more realistic than a hierarchical structure. The Analytic Network Process (ANP) which allows users to define the interactions among the elements in a network structure is an appropriate choice in developing a PPP risk model.

Methodology of ANP-based risk assessment model for PPP hydropower investments
The proposed ANP-based risk assessment model for PPP hydropower investments is presented in Figure 1. It includes four steps: (1) Identifying the risk factors; (2) Defining risk clusters and constructing a network structure; (3) Prioritizing the risk factors/clusters; and (4) Developing a risk assessment tool.

Identifying the risk factors
The first step of identifying the risk factors for PPP hydropower investments was performed in two stages. In the first stage, an extensive literature review was performed and an initial checklist of risk factors for PPP hydropower investments was formed. In the second stage, a questionnaire survey was administered to fourteen experts employed by seven different hydropower investment companies. The demographic information of the respondents is presented in Table 1. The initial checklist of risk factors was made available to the experts who were asked to mark those factors that are applicable to PPP hydropower investments in Turkey and to add any risk factors missing in the checklist. The experts evaluated the risk factors in the questionnaire by mentioning valid (V)/not valid (N). Two rounds of a Delphi process were performed online to minimize the spread of the answers. The final checklist of risk factors, their possible adverse consequences, and the corresponding research papers are presented in Table 2.

Defining risk clusters and constructing a network structure
The second step involved a brainstorming session with seven experts (a subset of the original fourteen experts; namely experts 1, 2, 4, 5, 7, 9, and 11) employed by seven different hydropower investment companies to develop a risk matrix that shows the relationships between the risk factors. With the help of these seven experts, the factors were organized in eight clusters, namely resources, stakeholders, construction, external environment, market, operation, financing/economic conditions, and government requirements. As seen in Table 2, each cluster is represented by related factors. Following Saaty's (2003) instructions, the risk clusters were organized into a network structure presented in Figure 2 that shows the interactions between the factors.

Prioritizing the risk factors using ANP
The third step of the study involved prioritizing the risk factors using ANP. Analytic Network Process (ANP) is the generalized form of the analytic hierarchy process (AHP), which was first introduced by Saaty (1996). It is a flexible and useful multi criteria decision making method that enables decision makers to arrange priorities and to select the best alternative by considering both the tangible and intangible aspects of a problem Dikmen et al., 2007). As ANP shows the interactions and interdependencies among the elements, it can allow decision makers to solve more complex problems (Saaty, 1996). ANP has been used in various fields to solve several multi-criteria decision-making problems such as contractor selection (e.g., El-Abbasy et al., 2013;Cheng & Li, 2004;Hasnain et al., 2018), project selection (e.g., Niemira & Saaty, 2004;Shang et al., 2004;Meade & Presley, 2002;Dikmen et al., 2007), construction method selection (Ozcan-Deniz & Zhu, 2015), risk assessment (e.g., Do et al., 2017;Li & Wang, 2019;Valipour et al., 2016Valipour et al., , 2015Bu-Qammaz et al., 2009;Dehdast et al., 2017) and performance assessment (e.g., Zhao et al., 2017;Erdem & Ozorhon, 2015;Zhang et al., 2016;Tohumcu & Karasakal, 2010;Ozorhon et al., 2007). No studies have been reported in the literature about its utilization for risk assessment in PPP hydropower investments. Hydropower investments are large-scale and complex projects (Zhang et al., 2013). They have a long running construction duration, and as a result, they are riskier and more complicated compared to other types of projects (Sharma & Kar, 2018). Based on the research presented in the preceding paragraphs, an ANP-based risk assessment model was used in this study.
ANP requires that pairwise comparisons of the factors in each cluster be made (Saaty, 2003(Saaty, , 2004. For this purpose, another brainstorming session was organized with the same seven experts who participated to the first brainstorming session. A total of 31 pairwise comparisons

Laws/ Regulations
Adverse impact, delay, change * * * * * * * * * * * * * * * * * were made by the experts by using the fundamental scale specified by Saaty (2008) in Table 3. An example of a comparison matrix for the "construction" cluster is presented in Table 4. The consistency of the responses was calculated for each cluster. This ratio has to be smaller than 0.1 for a 2-factor cluster, 0.05 for a 3-factor cluster, and 0.08 for a 4-factor cluster for consistency (Saaty, 1994). The consistency ratios of all clusters satisfied the requirements.  Extreme importance The evidence favoring one activity over another is of the highest possible order of affirmation

Developing a risk assessment tool
The last step of the study involved developing a risk assessment tool for PPP hydropower investors. The total risk rating (TRR) was calculated by considering the priority of each risk factor in Eqn (1): where: TRR is the total risk rating; P i is the priority of the i th risk factor and R i is the risk rating of the i th risk factor. By entering the risk rating of each risk factor using a 1-5 Likert scale where 1 means no risk and 5 extreme risk, a potential PPP hydropower investor can easily see the total risk rating of the project using a scale of 0-100. The tool's user-friendly interface is presented in Figure 3.

Discussion of results
According to Table 5, the most important risk factor in a PPP hydropower investment in Turkey is legal risks. The Turkish government provided a legal framework for PPP renewable energy investors in 2005, but the policies have not yet been finalized. There have been several modifications in energy policies for the design, construction and operation phases of PPP hydropower investments between 2005 and 2020. Because Turkey is a developing country, the existing energy policies are mostly ambiguous. As a result, it is almost impossible to predict legal risks. Consequently, investors seriously hesitate to commit to a PPP  Figure 3. The risk assessment tool ANP was performed by a software named "SuperDecisions" which was developed by Saaty (1996). The pairwise comparisons were input to SuperDecisions which calculated a super matrix in three steps. First, it constructed an unweighted super matrix using the local priorities derived from the pairwise comparisons; second, it calculated a weighted super matrix by multiplying the values of the unweighted super matrix with their related cluster weights; and in the last step, it derived a limiting super matrix by raising the weighted super matrix to higher powers until all the columns have stable values. The priority or importance weight of each risk factor and each cluster were thus obtained by examining the limiting super matrix. The results are presented in Table 5 and Table 6, respectively. investment. It would be safe to say that if this research had been conducted in an industrialized country with fully established energy policies, legal risks would not have had such importance.
The second most important risk factor is contractor/ subcontractor risk. It is one of the most typical risk factors for PPP projects not only in Turkey but also in other countries (Nguyen et al., 2018;Mazher et al., 2018;Chou & Pramudawardhani, 2015;Hwang et al., 2013;Yu et al., 2018;Ahmadabadi & Heravi, 2019). As Choma (2008) stated, a company must take preventative action against contractor/subcontractor risk to prevent incurring significant losses rather than planned profits. Any contractor or subcontractor failure can cause significant delay in the construction phase of the PPP project, which leads to a shorter operation phase, which in turn directly affects the cash flow of the investment. In addition, poor construction quality also increases the potential risks in the operation phase (Yu et al., 2018). Compared to legal risks over which the investor has no control, the contractor/subcontractor risk is much more controllable and transferable. Indeed, the investor can mitigate this risk by paying extra attention to contractor selection (experience of contractor in similar type of projects, company structure, references, etc.), specifying contract conditions clearly (Aladağ & Işik, 2019;Choma, 2008), and by transferring this risk to a third party through bonding and insurance. Furthermore, as Yu et al. (2018) pointed out, the probability of project success can be dramatically increased by mitigating the contractor/subcontractor risk properly.
The third most important risk factor is operator risk. Operating revenue is one of the most important cash flow parameters in PPP investments (Nguyen et al., 2018), and has a direct impact on the Net Present Value (NPV) of the investment. Like picking the right contractor, the selection of the right operator (experience in similar type of projects, competence, references) is vital in successful PPP projects. The experts who participated in the surveys and brainstorming sessions mentioned that as hydropower plants are complex systems that involve advanced technology and equipment. Inexperienced and incompetent operators may not be able to operate in this environment, causing frequent interruptions in the operation of the power plant to perform costly maintenance and significant drops in revenue caused by the deactivation of the power plant.
According to Table 6, the three most important risk clusters are "stakeholders", "government requirements", and "resources", whereas "market" is the least important cluster.
-That "stakeholders" is the most important cluster is an expected result because two elements of the "stakeholder" cluster (contractor/subcontractor risk and operator risk) are three of the most important risk factors in PPP hydropower investments. -"Government requirements" is the second most important risk cluster. This is not a surprise as one of the elements of this cluster (legal risk) is the most important risk factor of PPP hydropower investments. According to Mazher et al. (2018), the legal risks involved in PPP projects cause more problems in developing countries compared to more industrialized countries. -The third most important risk cluster, "resources" that involves labor, material and equipment related risks also makes sense because according to Zayed et al. (2008), the quality and availability of the right material, labor and equipment are the key elements in achieving project success. -The elements of the "market" cluster include demand risk, competition risk, and tariff rate risk. "Market" is the least important risk cluster because governments typically provide guarantees to buy the energy produced at a rate agreed upon in the contract, hence by and large minimizing the risk of fluctuating demand, the risk of aggressive competition from other investors, and the risk of changes in the rates. The results were discussed with the fourteen experts who participated in the surveys and the brainstorming sessions. The experts were unanimous in their opinion that the ANP-based risk assessment model that identifies the priority of each risk factor/cluster is realistic as the method considers the interdependencies between the risk factors.

Testing the performance of the risk assessment model
Ten real PPP hydropower investments made in different regions of Turkey were chosen as case studies to test the performance of the proposed model in terms of its predictive capability. These projects were undertaken by the seven companies that supported this research by providing the participants of the survey and of the brainstorming sessions. The capacity of each of the hydropower stations is larger than 10 MW, considered to be large size projects. An expert team was formed of 10 project managers who were directly involved in all stages of the investments. In the first step, the experts were asked to rate the risk of each factor by using a Likert scale (1-5 scale), taking into consideration the characteristics of the projects. The overall risk rating of each project was calculated out of 100 by using the risk assessment tool. In the second step, the experts were asked to assign an overall risk rating to each of the 10 case study projects out of 100 by using their subjective judgments. The reliability of the model is determined by calculating the percentage error between the calculated risk rating and the risk rating assigned by an expert. As seen in Table 7, the percentage error of each case study ranges between 0.29% and 9.11% with an average of 3.56% for the 10 cases, which indicates that the proposed model generates results that are consistent with expert judgements. Therefore, the proposed model is of value to inexperienced decision-makers who can use this tool for risk assessment without expert involvement.

Conclusions
This paper presents an ANP-based risk assessment model that can be used to quantify the total risk rating of a PPP hydropower investment by considering the risk factors associated with PPP hydropower investments. Using the proposed model, one should be able to obtain the total risk rating and the relative importance of each risk factor. In this study, first, the risk factors associated with PPP hydropower investments were identified by means of an extensive literature review, and a checklist survey of risk factors was administered to 14 experts. Second, the risk factors were grouped into 8 clusters named resources, stakeholders, construction, external environment, market, operation, financing/economic conditions, and government requirements. Third, a brainstorming session was arranged to identify the interactions between these risk factors and a network structure of risk factors was constructed. A second brainstorming session was organized to make pairwise comparisons of the factors. By using the "SuperDecisions" software, the weighted, unweighted, and limited super matrices were calculated and the priorities of each risk factor and cluster were identified with the help of the limited super matrix. The results presented in Table 5 and Table 6 indicate that the most important risk factor is "legal risk", and the most important risk cluster is "stakeholders". Fourth, a risk assessment tool was created to calculate the total risk rating of a PPP hydropower investment.
To test the performance of the model, ten real PPP hydropower investments were chosen as test cases. The average percentage error between the calculated (by the model) and estimated (by an expert) total risk ratings for the ten test cases was found to be 3.56%. The tool can be used by decision-makers who have only limited experience with PPP hydropower projects to estimate the risk level and to formulate response strategies. The contributions of this research include: (1) identifying the risk factors associated with PPP hydropower investments in Turkey, which may be similar to the risk factors in most developing countries; (2) determining the relative impor-tance of each risk factor and cluster; (3) creating a tool that can inform and guide investors with limited experience in PPP hydropower investments. Although this model was proposed for PPP hydropower investments in Turkey, it can also be used in different countries by duplicating the four steps in the proposed model. The proposed model can also be used to perform the risk assessment of other PPP investments such as renewable energy investments, using a different set of risk factors, and a different risk network relevant for the investment.
It should be noted that the limitation of the proposed model involves the subjectivity of risk ratings. Indeed, it is difficult to reach consensus when the pairwise comparisons are made by multiple experts. Also, as the number of relationships between the risk factors increases, the number and complexity of the pairwise comparisons of risk factors increases, endangering the consistency of the ratings. Further research could focus on ways to minimize the subjectivity and enhance the consistency of the ratings calculated by ANP models.

Disclosure statement
I don't have any competing financial, professional, or personal interests from other parties.