RISK PERFORMANCE INDEXES AND MEASUREMENT SYSTEMS FOR MEGA CONSTRUCTION PROJECTS

. In recent years, interest in and studies on urban regeneration projects have increased. New urban regeneration projects, i.e., redevelopment projects, are being used to help solve the social and economic problems caused by antiquated buildings and degraded infrastructure. However, such projects can offer no performance guarantees because they have various and complex stakeholders and their huge scale exposes them to many risks. We propose a risk performance index to improve the efficiency of general performance measurement for mega projects by extending the existing cost/schedule-based performance measurement system to include the risks of such mega projects. The risk performance index method proposed in this study is similar to the conventional EVMS and makes it possible to perform a three-dimensional integrated performance measurement of cost/schedule/risk through 18 indexes and variables.


Background
Some urban areas have experienced significant degradation, such as population decline, building deterioration, and limited development activities, despite overall increases in population, industries, and development activities in recent years. A type of construction business that embraces the concept of "urban regeneration" has emerged to address many of these problems (Kang et al. 2008). However, urban regeneration projects are very complex because they have many project sponsors and stakeholders as well as long project life cycles. In particular, many of these mega projects involve threedimensional mixed-use space development.
A mega project can generally be defined as a project that costs more than 1 billion dollars US (1 trillion won) and includes many risk factors that can cause delays or failures during the project life cycle (Flyvbjerg et al. 2003). Thus, it is important to establish a method and system to manage these risk factors effectively in advance. Moreover, it is necessary to reduce the probability of such risk factors causing failures in the project by measuring the performance of projects from the point of view of risk management. We define a risk performance index (RPI) that measures the performance of projects by integrating the cost/schedule/risk factors and by adding risk management activities to the EVMS, which is the existing integrated cost/schedule-based performance measurement system for construction projects. We also propose a method to produce and analyze the RPIs to improve the accuracy and efficiency of the general per-formance measurement for mega projects by extending the conventional cost/schedule-based performance measurement system to include risk management

Methodology
The subject and scope of this study are focused on mega construction projects. The process and method of this study can be summarized as follows.
First, we survey the existing performance measurement methods used in general construction projects and discuss the advantages and disadvantages of each method.
Second, we assess the need for performance measurement of mega projects and its expected effects on risk management.
Third, we propose an RPI to measure the performance by integrating cost/schedule/risk by augmenting the EVMS, which is the conventional cost/schedule based performance measurement method.
Fourth, we propose calculation and analysis methods for the proposed RPIs and the expected effect of the integrated cost/schedule/risk performance measurement method. and function of an organization. A performance goal is subordinate to the strategy goal and shows major projects planned in a particular year or a specific goal covering multiple aspects of a business group.
A performance index is a scale to measure the level of achievement of the performance goal. It is important to identify quantitative measures of the goals pursued in the project. The development of a performance index enables the efficiency of the project to be measured by comparing and evaluating quantitatively the achievement and level of the performance goal.
We studied three methodologies of performance measurement systems used in existing construction businesses: EVMS, BSC, and KPI.

EVMS
The Earned Value Management System (EVMS) is the most widely used performance measurement system in construction businesses. The United States Department of Defense (2008) has described it as "a performance-based management system for measuring actual progress against the criteria configuration for the cost, schedule, and performance goals in projects". Fleming and Koppelman (1996) defined the EVMS as "a continuous measurement for practical works under precisely managed work schedules and a management method that estimates the final cost and schedule in a project through this measurement".
The performance measurement applied by using the EVMS integrates cost and schedule. It helps identify how any difference between the planned budget and the actual cost influences the project, by comparing and managing the performance vs plan and estimating the reduction or delay in the schedule from the earned value to the completion of the project and the excess of the budget. As shown in Table 1, the elements of the EVMS can be classified as plan, performance measurement, measurement for management analysis, and analysis elements.

BCS
The Balanced Score Card (BSC) is another representative performance measurement system. The BSC method, proposed by Kaplan and Norton (1993), is a strategic management method that uses traditional financial or accounting measurements to overcome the limits and problems of performance measurement in the short term, and provides a way to establish performance measurement for a general project in the long term. It is widely used to establish performance indexes in construction businesses throughout the world. The BSC is a financial index that represents the results of the project execution and customer satisfaction and that shows operational activities, internal management, and operational indexes for learning and growing.
Although it has the advantage that it performs its management processes strategically by measuring nonfinancial aspects, the BSC"s financial and nonfinancial approaches comprehensively differ from traditional measurement methods. During the strategy establishment ACWP+(BAC-BCWP)/ CPI process, it can be difficult to reach an agreement on should be measured because organizations differ in their strategies, visions, and goals.
The balanced performance table is a scale that evaluates the business management. It has certain limitations in the evaluation of the satisfaction level of a project, even though it is useful for evaluating the business management, because it only evaluates the management strategy focused on operational effectiveness.

KPIs
Key performance indicators (KPIs) are a representative performance measurement system established in Britain based on the construction renovation movement called "Rethinking Construction". The system, which was first promoted in 1998, was intended to improve productivity in construction businesses. It can be used to measure not only construction performance, such as construction cost and duration reduction, but also the performance of a business in terms of profits and productivity (The KPI Working Group 2000).
The construction renovation movement can be classified into seven major performance indexes: duration, cost, quality, customer satisfaction, design change, project performance, health and safety for the construction culture, recognition, production method, and production system. Performance can be measured based on these classifications and the results are applied to plan the effi-ciency and productivity of the construction business. It also establishes a partnership between the government, owner, and construction businesses and promotes best practices. It has been shown to improve project performance and cost effectiveness by removing inefficiencies and unproductive factors.

Need for Risk Performance Measurement System Related to Construction Processes
In recent years, the main trend in urban regeneration projects and large-scale development projects throughout the world has been the development of three-dimensional mixed-use spaces that include such functions as residential, commercial, business, public, cultural, and leisure and that arrange these in horizontal and vertical spaces.
Although this type of development has the advantage of providing all the facilities required in a specific area, thus simultaneously maximizing the usability of the space, it involves many risks throughout the project, such as complicated interests in the major subjects, mixed development areas dominated by the civil and public spheres, operation and maintenance, and property management. In addition, there have been few studies on performance management for construction businesses because conventional performance management only measures the visible performance in businesses, such as financial and management performance. In particular, few studies have examined the risk factors that affect the performance management of mega projects.
Therefore, it is necessary to create a performance management method related to such risk factors to help estimate these factors" influence on a project in a timely and effective manner by developing a technology that continuously manages performance in relation to the risk factors in the early stages of mega projects and that suggests strategies for responses. Thus, we define an RPI for measuring the performance related to risks in construction businesses and derive calculation techniques and measurement methods. We then propose a new performance measurement method that considers the internal risk factors that affect the success or failure of the project in the context of the conventional cost/schedule-based approach.

Definition of Risk Performance Indexes
An RPI can assess the risk management in threedimensional mixed-use development projects and can be combined with similar measurement systems such as that for the EVMS. The combined performance measurement index can then be used to measure the performance in the three aspects of cost/schedule/risk.

Component of the Risk Performance Index
The RPI used in this study recognizes the internal risks in a project from the point of view of risk management and quantizes those risks as schedule and cost risk values based on the estimation of each residual risk. The RPI consists of 18 detailed indexes and variables. Table 2 shows the English terms for these indexes and variables with their descriptions and abbreviations.

Measuring the Risk Performance Indexes
(1) Cost Risk Performance Index (CRPI) As noted in Eq.
(1), the cost risk performance index (CRPI) can be calculated by subtracting the residual cost risk variance (RCRV) from the forecast cost risk variance (FCRV) and dividing by the FCRV at a specific point during the business period.

FCRV RCRV CRPI FCRV
where CRPI: Cost Risk Performance Index, FCRV: Forecasted Cost Risk Value, RCRV: Residual Cost Risk Value. The analysis of the CRPI can be performed as follows. First, if the CRPI is 1, then the RCRV is 0, showing the perfect elimination of the cost risk. It can also be seen that the residual risk in the project is 0, which is the best condition of the cost risk. Second, if the CRPI is greater than 0 and less than 1, it shows that the RCRV is lower than the FCRV. This means that although there are still some risks in the project, they are at a low level compared with the forecasts and so the cost risk shows a good status. Third, if the CRPI is 0, the FCRV is the same as the RCRV. Because this shows that there has been no reduction in the FCRV, it also shows no reduction in the cost risk. Fourth, if the CRPI is less than 0, it shows that the RCRV exceeds the FCRV, indicating an increase in the cost risk in the project. Table 3 shows the cost risk and its analysis method. The schedule risk performance index (SRPI) can be computed by subtracting the residual schedule risk variance (RSRV) from the forecast schedule risk variance (FSRV) and dividing by the FSRV at a specific point during the business period. The calculation formula can be expressed as Eq. (2).
The SRPI can be analyzed as follows. First, if the SRPI is 1, it shows that the RSRV is 0, indicating the perfect elimination of the schedule risk. The remaining risk in the project is 0, which shows the best condition of the schedule risk. Second, if the SRPI is greater than 0 and less than 1, it shows that the RSRV is lower than the FSRV. This means that although there are still some risks in the project, they are at a low level compared with the forecasts, indicating that the schedule risk is in an excellent state. Third, if the SRPI is 0, the FSRV is the same as the RSRV. Because this shows there is no reduction in the FSRV, it also shows no reduction in the schedule risk. Fourth, if the SRPI is less than 0, it shows that the RSRV exceeds the FSRV, indicating an increase in the schedule risk in the project. Table 4 shows the schedule risk and its analysis method. It is obviously possible to verify the change in the cost/schedule/risk according to the measurement points of the performance index using a method in which the cost/schedule/risk performance can be presented by integrating the CRPI and SRPI in a quadrant, as illustrated in Fig. 1.

Fig. 1. Integrated Cost/Schedule Risk Performance Measurement
The analysis of the integrated chart of the cost/schedule RPIs is as follows. First, if the CRPI and SRPI are both 1, it shows that the cost/schedule risks have been totally removed. Second, if the CRPI and SRPI are greater than 0, it shows an excellent condition in which the cost and schedule risks have all been reduced. Third, if the CRPI is greater than 0, but the SRPI is less than 0, the cost risk has decreased, but the schedule risk has increased. Fourth, if the CRPI is less than 0, but the SRPI is greater than 0, the cost risk has increased, but the schedule risk has decreased. Fifth, if the CRPI and SRPI are both less than 0, the cost risk and schedule risk have significantly increased and the project"s state has deteriorated.
(4) Cost Impact Variance (CIV), Schedule Impact Variance (SIV) The cost impact variance (CIV) and schedule impact variance (SIV) verify the effective execution of the response to risks by comparing the cost/schedule impact forecast by the cost and schedule risks at a particular point with the cost/schedule impact that has actually occurred. These can be calculated by using Eqs (3) and (4), respectively.
The analysis of the CIV and SIV can be performed as explained in Table 5.

Table 5. CIV, SIV Analysis
Index Description CIV > 0 ACI is less than FCI, risk response has been efficient or cost risk has been decreased CIV < 0 ACI is greater than FCI, risk response has been inefficient or cost risk has been increased SIV > 0 ASI is less than FSI, risk response has been efficient or schedule risk has been decreased SIV < 0 ASI is greater than FSI, risk response has been inefficient or schedule risk has been increased (5) Cost Risk Response Variance (CRRV), Schedule Risk Response Variance (SRRV) The cost risk response variance (CRRV) shows the difference between the actual cost impact and the actual response cost impact investigated at a particular point, and the schedule risk response variance (SRRV) represents the difference between the actual schedule impact and the actual response schedule impact investigated at a particular point. The calculation of these values can be carried out using Eqs (5) and (6), respectively. CRRV ACI ARC

SRRV ASI ARD
where CRRV: Cost Risk Response Variance, ACI: Actual Cost Impact, ARC: Actual Response Cost, SRRV: Schedule Risk Response Variance, ASI: Actual Schedule Impact, ARD: Actual Response Days.
The analysis of the CRRV and SRRV can be performed as explained in Table 6. It is possible to examine the total change in the efficiency of the response strategy for the cost/schedule/risk by integrating the CRRV and SRRV in a quadrant as shown in Fig. 2.

Fig. 2. Integrated Cost/Sch. Risk Response Variance Measurement
The integrated chart of the cost/schedule risk response variances can be analyzed as follows. First, if the CRRV and SRRV are both greater than 0, it shows that the efficiency of the response strategy is excellent in both cases. Second, if the CRRV is greater than 0, but the SRRV is less than 0, it shows that the efficiency of the strategy of the SRRV is poor. Third, if the CRRV is less than 0, but the SRRV is greater than 0, the efficiency of the CRRV is poor, but the efficiency of the SRRV is good. Fourth, if the CRRV and SRRV are both less than 0, the efficiencies of the CRRV and SRRV are both poor.

(7) Cost Risk Response Efficiency (CRRE)
The cost risk response efficiency (CRRE) measures the efficiency of the actual cost impact (ACI) vs the forecast cost impact (FCI) at a particular point during the project period. However, the FCI, ACI, and actual response cost (ARC) show different tendencies in their changes. In general, the three curves begin at 0, approach their peaks three-quarters of the way through construction, and return to 0 at the completion of the project. The scale of the changes in the curves is largest for FCI, but the changes in the ACI and ARC are about equal. Fig. 3 illustrates the tendency in the change of the forecast vs actual cost impact and response cost.
The difference between the FCI and the ACI becomes the CIV, and the difference between the ACI and the ARC becomes the CRRV.
As shown in Fig. 3, the CRRE at a particular point during the project period can be obtained by dividing the CIV by the ARC. It can be expressed as Eq. (7).
where CRRE: Cost Risk Response Effective, CIV: Cost Impact Variance, ARC: Actual Response Cost.

Fig. 3. Relationship between Forecasted/Actual Cost Impact and Response Cost
The analysis of the CRRE can be performed as follows. First, if the CRRE is greater than 1, it shows an excellent condition for the CRRE because the ARC is greater than the CIV. Second, if the CRRE is 1, there is no CRRE because the CIV is the same as the ARC. Third, if the CRRE is less than 1, the CRRE shows a bad condition because the CIV at that point is greater than the ARC. The analysis of the CRRE is explained in Table 7. The schedule risk response efficiency (SRRE) measures the efficiency of the actual schedule impact (ASI) vs the forecast schedule impact (FSI) at a particular point during the project period. The difference between the FSI and the ASI becomes the SIV, and the difference between the ASI and the ARD becomes the SRRV.
The SRRE at a particular point during the project can be obtained by dividing the CIV by the ARD. It can be expressed as Eq. (8): where SRRE: Schedule Risk Response Effective, SIV: Schedule Impact Variance, ARD: Actual Response Days.
The analysis of the SRRE can be performed as follows. First, if the SRRE is greater than 1, it shows an excellent condition in the SRRE because the ARD is greater than the SIV. Second, if the SRRE is 1, there is no SRRE because the SIV is the same as the ARD. Third, if the SRRE is less than 1, the SRRE shows a bad condition because the SIV at that point is greater than the ARD. The analysis of the SRRE is explained in Table 8. The relationship between the contingency reserve (CR) and the actual risk cost (ARC) can be generally defined as follows.
As the project proceeds, the contingency reserve at the project start (CR 0 ) will decrease and the contingency reserve at the project completion (CR 100 ) becomes 0. On the other hand, the actual response cost at the project start (ARC 0 ) is 0, but as the project proceeds, the actual response cost will increase and the cumulative sum of actual response cost at the project completion (ARC 100 ) matches the contingency reserve at the project start (CR 0 ). Fig. 4 shows the relationship between CR and ARC.

Fig. 4. Relationship between Contingency Rezerve (CR) and
Actual Risk Cost (ARC) From Fig. 4, the interpretation method of CR n and ARC n at a specified project time n is as follows. First, if CR 0 = CR n + ARC n , that is, if the sum of the contingency reserve and actual response cost is equal to the contingency reserve at the project start (CR 0 ), we can determine that the contingency reserve at the specified project time is appropriate. Second, if CR 0 > CR n + ARC n , that is, if the sum of the contingency reserve and actual response cost is less than the contingency reserve at the project start (CR 0 ), we can determine that project risks are decreasing and the contingency reserve at the specified project time should be reduced because it is too high. Third, if CR 0 < CR n + ARC n , that is, if the sum of the contingency reserve and actual response cost is greater than the contingency reserve at the project start (CR 0 ), we can determine that project risks are increasing and that the contingency reserve at the specified project time should be increased because it is too low. The analysis of CR n and ARC n at the specified project time n is explained in Table 9. Table 9. CR n and ARC n Analysis Index Description CR 0 = CR n + ARC n Contingency Reserve at the specified project time is proper CR 0 > CR n + ARC n Project risks are decreasing or Contingency Reserve at the specified project time should be reduced because it is too much CR 0 < CR n + ARC n Project risks are increasing or Contingency Reserve at the specified project time should be increased because it is too low

Risk Performance Measurement Tables
It is necessary to produce a format that verifies the risk factors existing in a mega project and their influences by analyzing the RPIs and calculation results proposed in this study. Thus, we classified the performance indexes into qualitative aspects that measure the risk performance as indexes, and quantitative aspects that measure risks in monetary amounts. We therefore propose the Qualitative Risk Performance Measurement Table and Quantitative  Risk Performance Measurement Table, which can verify each risk factor and the results of the measurement as shown in Figs. 5 and 6, respectively. The Qualitative Risk Performance Measurement Table, in Fig. 5, configures the forecast risk value (FRV) and residual risk value (RRV), which can be used as criteria for presenting the RPIs as columns that are calculated on a reference day, and shows the results of the calculation of the CRPI and SRPI based on this table. The Quantitative Risk Performance Measurement Table, in Fig. 6, configures the FCI/FSI, ACI/ASI, and ARC/ARD, which can be used as criteria for presenting the risk performance as columns that are calculated on a reference day, and demonstrates the results of the calculation of the CIV/SIV, CRRV/SRRV, and CRRE/SRRE based on this table. It is evident that these risk performance measurement tables help the project manager to judge the scale, influence, and response efficiency of the various risk factors included in the mega project.

Risk Performance Measurement Example
Figs. 5 and 6 show the calculation of risk performance using existing housing redevelopment data. These examples nicely illustrate the theoretical and practical value, as well as the validity, of the risk performance measurement model proposed in this paper. The risk performance measurements in Figs. 5 and 6 are evaluated every three months.
A qualitative risk performance measurement for the "Low rate of apartment sales" on two risk factors is shown in Fig. 5. Ratings on the probability scale and cost impact scale for April 1, 2010 were 4 and 5, respectively. Therefore, the forecasted cost risk value (FCRV) was calculated to be 20. Also, the rating on the schedule impact scale was 2, yielding a forecasted schedule risk value (FSRV) of 8. The residual risk values of the "Low rate of Qualitative Risk Performance Measurement On the other hand, because the schedule impact scale value increased to 4, the residual schedule risk value (RSRV) is 12. Using the FCRV, FSRV, RCRV, and RSRV numbers in Eq. (1), the cost risk performance index (CRPI) is 0.7. Using Eq. (2), the schedule risk performance index (SRPI) is -0.5. A CRPI between 0 and 1 indicates that the cost risk has been effectively controlled, or the residual cost risks are smaller than the forecasted cost risks, as illustrated in Table 4. However, when the SRPI is less than 0, as it is in this case, the schedule risk has not been effectively controlled, or the residual schedule risks are higher than the forecasted schedule risks (see Table 4). This analysis of the CRPI and SRPI numbers tells the project team that they should focus on controlling the schedule risk of the "Low rate of apartment sales." Fig. 6 shows the results of a quantitative risk performance measurement for the same risk item, the "Low rate of apartment sales." With respect to cost risk, the forecasted cost risk impact (FCI) based on a previous forecast date was quantitatively determined to be 200,000,000 won, whereas the actual cost impact (ACI) as determined from the base date was 150,000,000 won. Thus, using Eq. (3), we can see that the cost impact variance (CIV) is 50,000,000 won. A CIV of 50,000,000 won indicates that the cost risk response was effective, or cost risk has decreased, as shown in Table 5. Also, because the actual response cost (ARC) on the base date was 30,000,000 won, Eq. (5) tells us that the cost risk response variance (CRRV) is 120,000,000 won, which means that the cost risk response strategies are good, as is shown in Table 6. Furthermore, using Eq. (7), the cost risk response efficiency (CRRE) is calculated to be 1.67, and anything above 1 indicates good CRRE, as shown in Table 7. For schedule risk, the forecasted schedule risk impact (FSI) based on a previous forecast date was quantitatively determined to be 65 days, whereas the actual schedule impact (ASI) based on a base date was 80 days. Thus, using Eq. (4), the schedule impact variance (SIV) is -15 days. An SIV less than 0 indicates that the schedule risk response was not effective, or the schedule risk has increased (see Table 5). Also, because the actual response days (ARD) value on the base date was 86 days, Eq. (6) yields a schedule risk response variance (SRRV) of -6 days. An SRRV less than 0 means that the schedule risk response strategies are bad, as shown in Table 6. Furthermore, using Eq. (8), we can see that the schedule risk response efficiency (SRRE) is -0.17, and anything less than 0 indicates poor SRRE (see Table 8).

The Value and Validity of the Risk Performance Indexes and Measurement System
Generally, project risk management includes risk identification, analysis, and response at a project-specific time. The traditional EVMS cannot conduct the project performance measurement considering the project uncertainties and risks integrated with the cost and schedule. However, the risk performance indexes and measurement system proposed in this paper account for changing project risks, the evaluation of residual risk values, and the efficiency of risk response strategies by periodically comparing previous forecasted risk performance variables with those at a base date-risk performance indexes are calculated every three months rather than at one project-specific point in time. Furthermore, the measurement system integrates the traditional EVMS and risk management concepts by considering project risks during the project performance measurement.

Conclusions
Interest in urban regeneration projects has recently increased greatly. Such projects use redevelopment projects to solve the social and economic problems caused by antiquated buildings and degraded infrastructure. However, some characteristics of these projects-numerous stakeholders and a huge scale that exposes them to many risks-compromise guarantees of the performance of the projects. We have proposed risk performance indexes to improve the efficiency of the general performance measurement for mega projects by extending the existing cost/schedule-based performance measurement system. The expected effects of the risk performance index method proposed in this study can be summarized as follows.
First, we constructed our system to be similar to the EVMS, which is the existing cost/schedule integrated performance measurement method. It is therefore possible to conduct three-dimensional integrated performance management using the 18 detailed indexes and variables employed in the risk performance index.
Second, we can perform integrated qualitative performance measurement for cost/schedule/risk by measuring the risk-related cost performance index and schedule performance index.
Third, we can perform integrated quantitative performance measurement for cost/schedule/risk by measuring the cost impact variance, the schedule impact variance, the cost risk response variance, and the schedule risk response variance.
Fourth, we can measure the risk response efficiency by comparing the cost impact variance with the actual response cost, and we have proposed a method to analyze the extra project expenses and actual response cost at a particular point during the project.
Furthermore, using the risk performance measurement of "Low rate of apartment sales" as an example, the theoretical and practical value and validity of our risk performance indexes and measurement system can be summarized as follows: first, because risk is a dynamic phenomenon, the forecasting and reevaluation of risk factors should be performed periodically; second, our risk performance indexes provide the theoretical foundation for an integrated evaluation of cost and scheduling risks inherent in housing redevelopment projects; and third, by using our risk performance indexes and measurement model, a project team is required to forecast and evaluate project uncertainties and risks continually, thereby generating more proactive and diverse analyses than the traditional EVMS model.