USING RISK SIMULATION TO REDUCE THE CAPITAL COST REQUIREMENT FOR A PROGRAMME OF CAPITAL PROJECTS

This paper combines various concepts related to (i) project risk management, (ii) Monte Carlo simulation, (iii) project contingency cost estimation, and (iv) the relationship between project and programme risks, to illustrate that the contingency requirements are lower when simulating all the risks in the programme when comparing it with the individual project contingency requirement. A case study organisation provided 86 quantified risk registers related to port and rail capital projects. For each of these risk registers, the project contingency was estimated using a prescribed risk register template and Monte Carlo simulation software. The same 86 quantified risk registers were then used to simulate the programme contingency. The simulation results indicated that the programme contingency requirement was approximately 8% points lower than that of the sum of the individual projects. The first implication of this research result is that, should borrowed capital be used to fund the projects, the interest bill would be higher when calculating project contingency on a project-by-project basis. The second is that regularly appearing low probability, high impact risks, should be identified and these risks should be quantified not in the projects themselves, but in a centrally managed, programme cost contingency fund.


Introduction
There are various terms which are used to describe funds which finance realised project risks. ISO31000:2009 states that Risk Financing is as form of risk treatment, and it involves "contingent arrangements for the provision of funds to meet or modify the financial consequences" of realised risks (American Society of Safety Engineers 2011a). PMBoK™ uses the term Reserve and defines it as a provision made in the project management plan to mitigate cost and/or schedule risk (Project Management Institute 2013). Chapman & Ward (2011) uses a different definition by stating that contingency is the difference between an acceptable commitment level and the expected value of future performance.
Contingency is generally included in most cost estimates, and it is expected to be expended (Association for the Advancement… 2013). Further to this, Godfrey (1996) stated that there are three basic types of project contingencies: (i) tolerance in the specification, (ii) float in the programme and (iii) money in the budget. For this paper, the term contingency refers to "money in the budget". The importance and accuracy of contingency is covered by Oberlender & Trost (2011), Baccarini (2004), Mak et al. (1998) and Loizou & French (2012).
Papers by Baccarini (2006) and Bakhshi & Touran (2014), Hamad et al. (2016), Hollmann (2008) and Lam & Siwingw (2016) listed various methods (and methodology categories) for estimating project budget contingency. These estimating techniques include methods such as (i) traditional percentage, where contingency is calculated using an overall percentage which is added to the base estimate and (ii) Monte Carlo simulation. Monte Carlo simulation is a computerized mathematical technique that allows for the calculation of risk in quantitative analysis and decision making (Hillson 2009). It is used to aggregate variation in a system resulting from variations in the system, for several inputs, where each input has a defined distribution and the inputs are related to the output via defined relationships (American Society of Safety Engineers 2011b). It is popular because it uses basic statistics, can used with existing project data as baseline and there are many good software tools available (Hillson 2009).

Research context and motivation
Up to 2010, the Case Study Organisation (CSO) in this research used a joint venture, as an Engineering, Procurement, Construction Management Consultant. This consultant's employees conducted quantitative risk analysis on a small number of large projects since there were no similar internal resources at the CSO. The quantitative risk analysis was used to (i) estimate project contingency and to (ii) evaluate the level of risk for each project.
When the relationship with Consultant ended, internal project risk management capacity was created at the CSO. It was the first time in South Africa's history that a quantitative risk management approach was rolled out on part of a port and rail capital investment programme, and not only on individual, unrelated projects. This also means that the amount, quality and extent of data which was collected during this process is unique in a South African context. There were two primary reasons for the roll-out of quantitative risk analysis at the CSO as part of this research. The first was that (i) the implemented project contingency guidelines were inaccurate and that the (ii) Enterprise Risk Management (ERM) approach could not handle the complex projects which were being executed by the CSO.
The contingency guidelines provided by the CSO Project Lifecycle Process (PLP), used traditional percentage values to calculate project contingency. This meant that for a project in execution phase, a standard 10% was applied. The problems associated with this approach have been widely described by Baccarini (2005) and Bakhshi & Touran (2014) and criticism uses terms such as "un-scientific", "not project context specific" and "arbitrary".
In addition to this, CSO ERM methodology followed a risk qualification approach. This process ranked risks for further analysis or action by assessing and combining their probability of occurrence and consequence but did not provide for quantitative analysis of the effect of identified risks on overall project objectives. A quantitative approach fulfils this requirement and had to be able to model risks of the following complexity: "Risk XYZ is a multiple occurrence risk, which can happen 4 times over the duration of the project, has a time delay of up to 4 weeks, with a weekly cost of R250 000 and additional capital cost of between R3 and R6 million".
A more accurate approach was required at the new risk register (document used to identify, analyse and evaluate risks (International Organization for Standardization 2009)) template and had to provide for the following: -The calculation, testing and tracking of project contingency/reserves. These provisions in the project estimate to be used in the mitigation of cost and/or schedule risk (Project Management Institute 2013) estimate project management plan. -The integration of a risk register with the Project Estimate and Project Schedule, to provide a risk loaded Final Project Cost and Completion Date.
A risk quantification methodology using a MS Excel based risk register template and simulation software (@Risk) was rolled out on 106 projects as part of this research. Of these 106, 86 were quantified. The first phase of the research in this case study was to simulate the contingency requirements for each of the projects. Do to this, @Risk was used to generate an output distribution for each of the project risk registers. The 80 th percentile (P80 value) of these output distributions was then added to the project base costs as cost contingency. The next phase was the introduction of cost uncertainty into the estimate itself and the integration of the risk register into the estimate. This allowed the concurrent simulation of the estimate and the risk register to produce a "risk loaded estimate". The final step in the roll-out was the integration of the risk register into the project schedule to produce a "risk loaded schedule".
The last two steps were not rolled out to all projects due to a lack of appropriate skills, the complexity of integrated risk register / cost estimate / project schedule simulation models and inappropriate resource and task allocation. It is assumed that this had no effect on the results presented in this paper. The latter refers to the risk managers doing work on risk identification and quantification, but not on assisting the project manager in decision making, managing the risks and their treatment plans.

Research objective
After the implementation of this quantitative approach at the CSO, the following research question was therefore asked as manifestation of the research objective: "How does the sum of the P80 values from a risk register for each of the 86 project compare to the P80 value of the entire programme of projects?" This question needed to be answered in terms of two types of projects -Large and Small. The large projects were identified, by using criteria such as (i) being complex, high capital cost, multi-disciplinary engineering projects, having (ii) complex hydrogeological conditions (for example lots of water and rock) and (iii) projects with unfamiliar scope which are only executed once every 20 years etc. A total of eight projects like this were identified.
Tasks: the following tasks as part of the research process were followed to address the research objective and answer the question: -Verifying the validity of simulation as research methodology, -Using Monte Carlo simulation to model likelihood (chance of occurring) for both once-off and multiple occurrence risks as well as various types of consequence. -Giving a high-level overview of the simulation process.

Simulation as research methodology
Several sources indicate the validity of simulation in management studies and research. Harrison et al. (2007) stated that although some simulation studies were published in major management journals in the 1980s, simulation-based work did not begin to appear in management and social science journals with any regularity until the 1990s. It was particularly found in disciplines related to Management, Sociology, Psychology, Economics and Political science. This view was confirmed by Berends & Romme (1999). Harrison et al.'s (2007) conclusion started with "Computer simulation can be a powerful way to do science" and concluded with "…computer simulation promises to play a major role in the future…". They also stated that, together with theoretical analysis (deduction) and empirical analysis (induction), computer simulation is now a recognised technique in scientific research. Additionally, Jahangirian et al. (2010) stated that a wide scope of simulation techniques were being used and that the employed techniques focussed on real-world applications.

Monte Carlo simulation and the modelling of single and multiple occurrence risks and consequence
Monte Carlo simulation can be used to perform risk analysis by developing models of possible results by substituting a range of values, in the form of a probability distribution, for any factor in a project that has inherent uncertainty (Palisade Corporation 2014; Cooper et al. 2005). In the context of project management, these factors include variables such as project delays, time variable cost and additional capital requirements. The simulation then calculates results repeatedly, each time using a different set of random values produced by probability distribution functions. After executing up to tens of thousands of iterations, a Monte Carlo simulation produces distributions of possible outcome values (Palisade Corporation 2014; American Society of Safety Engineers 2011b). In practical terms, it produces a number in conjunction with a likelihood, for example: "There is an 80% likelihood that the project will be completed on 4 April 2019" instead of "on 4 April 2019". The 80% likelihood may also be presented in terms of a P-value, i.e. P80.
A MS Excel risk register was used, which made provision for the modelling of both single and multiple occurrence risks. For single occurrence risks, Table 1 below was used as probability values. These values were selected from the likelihood ranges prescribed by CSO's ERM policy since @Risk requires a discrete value to simulate likelihood. These use of likelihood ratings like this forms the probability part of probability-impact grids (PIG) as described by Cooper et al. (2005) and Hillson (2009). Cox (2008), Hillson (2009), with Chapman and Ward (2011) present some criticism on the use of such matrices, mainly related to their simplicity (not being able to support complex decision making), as well as their focus on risk and the exclusion of opportunities. Another shortcoming is that they generally do not assess risk urgency, and do not make provision for multiple occurrence risks. It was however, the approach used in the CSO's Risk Register Template (RRT).
A binomial distribution was used to model single occurrence risks. The binomial distribution is a discrete distribution returning only integer values greater than or equal to zero (Palisade Corporation 2014). In the context of a construction project, risk such as Safety incidents, Procurement delays; and Material Deliveries can realise multiple times and were modelled as such. A Poisson distribution was used to model the frequency of these type of risks. The Poisson distribution is a discrete distribution returning only integer values greater than or equal to zero (Palisade Corporation 2014).
The RTT made provision for the modelling of various types and combinations of consequence (outcomes of the risk event). These include time variable cost as well as direct capital cost. Risk consequence was modelled in terms of the monetary impact on the project, using the following method (1) and (2) The latter is best described here by using the following example: A project has two contractors: Contractor A with a weekly average rate of R150 000, and Contractor B with a weekly average rate of R10 000. During the risk workshop conducted as part of the case study research, it was established that should a specific risk realise, Contractor A will have a 10% loss and Contractor B a 25% loss. The Weekly Weighted Average Cost would therefore be as follows (3): Weekly Weighted Average Cost = R150 000*10% R10 000*25% + = R17 500 To ensure that sampling took place at the tail end of more uncertain risks, two different distributions from @Risk were used to estimate time delay and direct capital cost (4), (5) and (6) Taking the above, the overall logic used in creating simulation results, appears in Figure 1. Occurrence requires exceptional circumstances, exceptionally unlikely; even in the long-term future; only occur as a "100-year event". 1.0%

B Unlikely
May occur but not anticipated, or could occur in "years to decades".
20.0% C Moderate May occur shortly but a distinct probability it won't, or could occur within "months to years".

45.0% D Likely
Balance of probability will occur, or could occur within "weeks to months".

E Almost Certain
Consequence is occurring now, or could occur within "days to weeks".

High level overview of simulation process
The first step in the simulation process was to combine the existing risk registers into one MS Excel workbook, on a single sheet, using the logic displayed in Figure 1. This process is displayed in Figure 2. The cleaned-up Complete Risk Register (CRR) contained 1063 different risks, with a total of 165 individual risk names. Steps 2 and 3 of the simulation process is best described in terms of MS Excel and @Risk, as in Figure 3. In Cell E4, the simulation results for the project "Export Trippers, Port of Richards Bay" was collected, using a RiskMakeInput() statement. Cell D4 displays the P80 value from the risk register after the simulation was run. Cell C4 displays this P80 value as a % of the project estimate.
To ensure that the simulation results were reliable, simulation was executed and error checking took place. During the developing of the simulation model, two types of errors related to the following occurred: -The cells (or some cells) returned no results due to data description or data type mismatches. This was corrected by simply removing these errors and re-running the simulation. -If #Value errors occured, it meant that there are problems in the CRR_Simula-tion_Result named range, where certain of the line items in the data range returned errors. Filters were used to sort and correct these types of error.

Findings
After error checking was completed, 5 000 iterations were execution for each of the three project groups considered in the CSO. The first included all 86 projects, the second the 78 smaller projects and the last, the eight large projects. The results for the three groups of projects appear in the next table (Table 2):  When studying at the result for the 86 projects, 7.9% reflects a substantial difference of approximately R445.5 million. Should the CSO have elected to use the P90 value, the "over requirement" would have been 25.1%, which in turn reflects a difference of just more than R1 920 million. The disparity between the P80 values of the two project groups indicates that the eight large projects primarily influence the risk profile of the programme.

Discussion
The discussion covers (i) the financial impact of the contingency calculation methodology as well as (ii) recommendations related to the estimation of risks on a programme basis.

Financial impact of applied contingency calculation methodology
The implication of this simulation result is that when a company uses the P80 values from a risk register and simply adds them to the project estimate, the contingency requirements are overstated.
This also has an impact related to the interest bill on the lent capital, as in the following example: Taking a programme of R100 billion, which contains a contingency of R15 billion, the contingency would be over-estimated by R1.2 billion. The annual interest bill on R1.2 billion, using a interest rate of 8%, would be R96 million.
Should the CSO have selected to use the P90 value, the contingency requirements could have been over-stated by approximately R3.75 billion, leading to an annual interest bill of nearly R300 million.

Some recommendations
The impact of the current methodology to estimate contingency as the P80 value of the output from a risk register, on a project by project basis, is leading to higher capital requirements and the accompanying finance charges. In an environment where skills are in short supply, a possible way to reduce the contingency requirement in the absence of a concurrently run Monte Carlo simulation on the entire programme of projects, is to review the risk registers, and to classify the risk and the related treatment plans in terms of what Aritua et al. proposed (2011). Their recommendation is to distinguish between three types of risks: (i) Programme, (ii) Amplified and (iii) Generic Project risks, as described in Table 3. Table 3. Common to programme, amplified and generic project risks (Aritua et al. 2011) Category Qualitative Description Typical risks Common to Programme risks These risks related to the function of managing multiple projects and aligning them to the organisation's strategy and policies. It included the following risks Linking strategy and projects. Challenges in procurement. Competition for contractors. Stakeholder expectation management.
Amplified risks These risks are simple to deal with but exacerbated because of the multi-project environment.

Fraud.
Cash flow and funding problems. Changes in government policy.

Generic project risks
The next step would be to decide which of the various Programme and Amplified risks should be quantified in the projects themselves, or be removed from the projects, and quantified and managed on a programme basis. An added advantage of having treatment plans and owners, on a programme basis, would be that the effect of successful treatment plans would affect the entire programme.
Another recommendation would be to implement a policy regarding which type of risk identification and contingency estimating process would be applied on which type of projects. A decision regarding this might be that the current P80 rule is applied to the smaller projects and that the large projects undergo the risk loaded estimate and schedule route.
The last recommendation relates to low probability / high consequence risks which appear on many or all the project risk registers.
Although the CSO has a deeply embedded safety culture and accompanying safety statistics, the risk and consequences of having an on-site fatality is reflected in nearly all the risk registers. It is suggested that the data which was collected during this research be used to identify these risks and to include, but not quantify these risks, on a projectby-project basis. As described in previous paragraph, contingency provision for these risks should be made on a programme level.

Conclusions
This paper combined various concepts related to (i) project risk management, (ii) Monte Carlo simulation, (iii) project contingency cost estimation, and (iv) the relationship between project, and programme risks to illustrate that the contingency requirements are lower when simulating all the risks in the programme, when comparing it with the individual project contingency requirement.
There are various implications and recommendations. The first is that, should borrowed capital be used to fund the projects, the interest bill would be higher when calculating project contingency on a project-by-project basis. The second is that the CSO needs to identify low probability, high impact risks which regularly appear in the risk registers and to make provision for them not in the projects themselves, but in a central contingency fund. This should reduce some of the duplication in the project programme. In cases of resource constraints, organisations could introduce policies regarding which types of project would undergo which type of contingency estimation methodology.
The simulation results also showed that a small set of projects caused the most uncertainty in the programme. This is significant for project and risk managers since it brings the question "Which projects cause the most risk?" to mind, which in turn, should assist in ensuring that project management resources are proportionally assigned.