FRAMEWORK FOR THE VALIDATION OF SIMULATION-BASED PRODUCTIVITY ANALYSIS: FOCUSED ON CURTAIN WALL CONSTRUCTION PROCESS

As construction projects have become more complicated in design and construction, it is necessary to establish the construction operational plans in advance. However, there were some limitations in analyzing construction productivity due to the difficulty of collecting accurate data. To address this challenge, this study aimed to develop the framework for the validation of simulation-based productivity analysis, which consisted of three measures: (i) validation of the measured productivity data as target variable; (ii) validation of the measured duration data as input variable; and (iii) validation of the simulation model compared to the actual construction process. To verify the feasibility of the proposed framework, this study focused on the curtain wall construction project of “S” office building as a case study. The T-test was applied to investigate the statistical difference between the measure and simulated productivity. It was determined that the significance level α in the T-test for the unloading process was 0.136 with 95% confidence interval; the lifting process, 0.106; and the installing process, 0.311. As a result, there was no significant difference between the measured and simulated productivity. The proposed framework could enable executives and managers in charge of project planning and scheduling to accurately predict the simulation-based productivity.


Introduction
Construction projects have become larger in scale, higher in height, and more complicated in design and construction. According to these changes in construction projects, there has increased the demand for reliable operational plans before the start of construction project. In accordance with a desire for a scientific approach to operational plans, several methods have been used to establish the construction operational plans effectively in advance (Thomas, Yiakoumis 1987;Herbsam, Ellis 1990;Chao, Skibniewski 1994;Portas, AbouRizk 1997;Sonmez, Rowings 1998;Shi 1999;Smith 1999;Goodrum, Haas 2002;Tam et al. 2002;Lu et al. 2006;Han et al. , 2011. Some studies used multiple linear regression methods for predicting construction productivity (Thomas, Yiakoumis 1987;Herbsam, Ellis 1990;Smith 1999;Goodrum, Haas 2002). Others examined an artificial neural network approach, one of the decisionmaking tools based on artificial intelligence, in estimating construction productivity (Chao, Skibniewski 1994;Portas, AbouRizk 1997;Sonmez, Rowings 1998;Shi 1999;Tam et al. 2002;Lu et al. 2006). However, these methods have limitations in analyzing construction productivity due to difficulty of collecting accurate data. This is the reason why these methods have worked excellently in other fields such as manufacturing industries where it is simple to collect data in repetitive process, but failed to present reliable results for construction projects.
To address these challenges, the simulation techniques have been used to replace the measured data with the simulated data which would be used as input variable in decision-making tools. In particular, CYCLONE (CYCLic Operation NEtwork), one of the discrete simulation techniques, has been used extensively in predicting construction productivity. Various developments and improvements in system stabilization or user-friendly interface have been continually achieved through techniques such as RESQUE (RESource based QUEuing network simulation), SIREN (SImulation of REpetitive Networks), GPSS (General Purpose Simulation System), and STROBSCOPE (STate and ResOurce Based Simulation of COnstruction ProcEsses). Recently, WebCYCLONE has been mostly used to conduct operation analysis and to predict construction productivity. The WebCYCLONE has the advantages of an easy application in a web-based environment and the strong adoption of the CYCLONE method, which has been ranked as one of the best discrete operation simulators in manufacturing industries (Han et al. 2006Cho et al. 2011Cho et al. , 2013Hong et al. 2011). This simulation technique has been validated with actual construction operations (Zayed, Halpin 2004, 2005Zayed 2009). However, the previous studies mostly focused on the generation of simulated data and the prediction of construction productivity using regression analysis or artificial neural network. A few of studies listed above presented simulation results which were validated with actual test data, however, the scientific methods based on statistic knowledge are still required. Also, there were some limitations in effectively analyzing construction productivity due to the difficulty of collecting accurate data. Therefore, it is necessary to conduct further studies for evaluating the significant difference (i.e., significant similarity) between the measured and simulated data through scientific approaches such as statistical methods. This study aimed to develop the framework for the validation of simulation-based productivity analysis, which consisted of three measures: (i) 1 st measure: validation of the measured productivity data as target variable; (ii) 2 nd measure: validation of the measured duration data as input variable; and (iii) 3 rd measure: validation of the simulation model compared to the actual construction process. To verify the feasibility of the proposed framework, this study focused on the curtain wall construction project of "S" office building as a case study. This study was conducted in four steps: (i) step 1: establishment of construction process; (ii) step 2: data collection; (iii) step 3: simulation modeling; and (iv) step 4: validation of simulation-based productivity analysis (refer to Fig. 1).

Step 1: establishment of construction process
Through interviews with the site engineers in the field of curtain wall construction project and the site observation in "S" office building, it is concluded that curtain wall construction project consists of largely three processes: (i) unloading process; (ii) lifting process; and (iii) installing process. The unloading process refers to the entire process of transporting the units by trucks and unloading the units by a forklift. The lifting process refers to the entire process of lifting the units to the installation area, which can be done by cranes and hoists. The installing process refers to the entire process of lifting the units to the exact location by a winch and fixing the units by fasteners. This study established the detailed activities of the three main processes in curtain wall construction project. The established construction processes can be used for collecting the duration data of the specific activities establishing the simulation model.

Step 2: data collection
The duration data of activities consisting of the main process in curtain wall construction project was collected from "S" office building. The research team conducted interviews with the site engineers in the field of curtain wall construction project and the site observation in "S" office building during two consecutive months from September to October 2012. During this period, this study collected four datasets, five datasets, and ten datasets for the unloading process, the lifting process, and the installing process. Table 1 shows the detailed descriptions on the project characteristics of the "S" office building. Tables 2 to 4 show the first three samples of the collected data, i.e., the duration data of activities consisting of the main process in curtain wall construction project.   Note: Unit (minute); and the activity (node) can be found in Figure 2. Note: Unit (minute); and the activity (node) can be founded in Figure 3. Note: Unit (minute); and the activity (node) can be found in Figure 4.

Step 3: simulation modeling
This study used the simulation techniques to replace the measured data with the simulated data which would be used as input variable. Based on the established construction process explained in Section 1.1 and the collected data explained in Section 1.2, the simulation models by the main process in curtain wall construction project were established using WebCYCLONE (WebCY-CLONE 2014). For more detailed information regarding CYCLONE and its basic modeling elements, please refer to Halpin and Riggs (1992).
The precedence relationship among activities should be defined using COMBI and QUE, basic modeling elements of CYCLONE. The activities defined as COMBI (e.g., nodes 3 and 7 in Fig. 2) can start only after the precedent activities (e.g., nodes 1, 2, 5, and 6 in Fig. 2) are prepared. Namely, in order to implement the activity 3. "Truck arriving" (i.e., node 3 in Fig. 2), the precedent activities (i.e. nodes 1 and 2) should be prepared. In addition, in order to implement the activity 7. "Forklift approaching" (i.e., node 7 in Fig. 2), the precedent activities (i.e., nodes 2 and 6) should be prepared. NOR-MAL element indicates commencement of the work task without considerations of any ingredience constraints like COMBI. The activity 8. "Forklift controlling" (i.e., node 8 in Fig. 2) was capable of being implemented after arriving one resource from the activity 7. "Forklift approaching" (i.e., node 7 in Fig. 2). COUNTER element as shown in node 13 in Figure 2, was for performing a function such as flow unit consolidation. It was to count the number of times of passing by a unit. It ultimately indicated the number of production which is one of crucial parameters for estimating productivity. GENERATE and CONSOLIDATE elements which were shown in nodes 5 and 12 denoted as GEN and CON in Figure 2, respectively, were used for cases that one unit of entity was broken into multiple entities and merged into the one entity again. A simulation model in Figure 2 illustrated that a curtain wall material delivered by one truck after the activity 4. "Material checking" (i.e., node 4 in Fig. 2) was separated into two pallets which were waiting for unloading by a folklift. When two pallets were unloaded and placed in the position, one curtain wall material was finally considered as completion of one cycle of unloading process. This site observation described above made GEN 2 and CON 2 at nodes 5 and 12, respectively, in Figure 2 (Halpin, Riggs 1992).
As shown in Figures 2 to 4, based on the precedence relationship among activities, this study established the simulation models by the main process in curtain wall construction project.
For achieving statistic task duration information in simulation models, searching the best fitted distribution on the collected datasets was conducted. Also, parameters of the probability distributions as task duration input variables were acquired based on basic rules of simulation techniques. Parameters as duration input data in simulation are totally various depending on a specific type of the probability distributions. For instance, parameters for a beta distribution, which has been widely known as well-fitted in construction, are divided into four values (the lowest value, the highest value, a, b). The values of a and b represented a basic shape of beta distribution. On the other hand, those of normal distribution in simulation are composed with just two values, mean and variance.
One truck and one forklift were assigned in unloading process in Figure 2. In lifting process shown in Figure 3, three types of resources such as one crane, one team for labor A and another team for labor B were considered as resource input data. Also, it designed that one winch equipment and one team for labor C and another team for labor D were provided in installing process in Figure 4.
All simulations for three processes were designed to be quit after running 1,000 times of cycles which were supposed to reach to steady state conditions.

Step 4: validation of simulation-based productivity analysis
In this study, WebCYCLONE, one of the representative discrete simulation techniques, was used to establish simulation models. To verify the feasibility of the simulation-based productivity analysis, the T-test was used to investigate the statistical difference (or similarity) between two types of values (i.e., the measure and simulated productivity). The statistic theory of significant probability (p value) was used in this study. With 95% confidence interval, if the significance level α is higher than 0.05, it can be said that there is no significant difference (i.e., significant similarity) between the measured and simulated productivity. On the contrary, with 95% confidence interval, if the significance level α is lower than 0.05, it can be said that there is the significant difference between the measured and simulated productivity (Lee, Lim 2011).
Meanwhile, this study conducted the validation of simulation-based productivity analysis from the perspectives of three measures to better reflect the actual conditions of construction sites, as follows: -1 st measure: validation of the measured productivity data as target variable; -2 nd measure: validation of the measured duration data as input variable; and, -3 rd measure: validation of the simulation model compared to the actual construction process.

T-test results between the measured and simulated productivity in the initial simulation
This study conducted the T-test to investigate the statistical difference between the measured and simulated productivity. Table 5 shows the T-test results between the measured and simulated productivity by the main process in curtain wall construction project: -Unloading process: the simulated productivity of the unloading process was determined at 5.104 (cycle/ hr). Compared to the four measured productivity of the unloading process, the significance level α (0.312) in the T-test was determined to be higher than 0.05. Namely, it could be concluded that there was no significant difference between the measured and simulated productivity in the unloading process. -Lifting process: the simulated productivity of the lifting process was determined at 6.664 (cycle/hr). Compared to the five measured productivity of the lifting process, the significance level α (0.000) in the T-test was determined to be lower than 0.05. Namely, it could be concluded that there was the significant difference between the measured and simulated productivity in the lifting process. -Installing process: the simulated productivity of the installing process was determined at 3.096 (cycle/hr). Compared to the ten measured productiv-ity of the installing process, the significance level α (0.014) in the T-test was determined to be lower than 0.05. Namely, it could be concluded that there was the significant difference between the measured and simulated productivity in the installing process.

1 st measure: validation of the measured productivity data as target variable
Through the T-test results between the measured and simulated productivity in the initial simulation model, it was concluded that there was the significant difference between the measured and simulated productivity in the lifting process and the installing process. To solve this problem, this study firstly conducted the validation of the measured productivity data as target variable. The actual conditions of construction sites had great effects on the measured productivity data. The risk of collecting biased data in construction sites is much higher than other manufacturing industries where simulation techniques have been effectively applied. Thus, it is necessary that additional measure should be applied to remove the biased data (i.e., outlier). In statistics, an outlier refers to an observed point that is distant from other observations. This study detected an outlier in the measured productivity data of the installing process. As shown in Figure 5, the boxplot (one of the functions in statistic package program, IBM SPSS Statistics v21.0) can detect outliers of the measured productivity data in the installing process. Among a total of ten cycles in the installing process, the productivity data collected in the fifth cycle (i.e., 1.388) was determined to be statistically distant from the other data. This outlier was considered as measurement error and removed from the dataset.
However, compared to the nine measured productivity of the installing process, the significance level α (0.000) in the T-test was determined to be lower than 0.05. Namely, it could be concluded that there was still the significant difference between the measured and simulated productivity in the installing process.

2 nd measure: validation of the measured duration data as input variable
Through the T-test results between the measured and simulated productivity after the 1 st measure, it was concluded that there was still the significant difference between the measured and simulated productivity in the lifting process and the installing process. To solve this problem, this study secondly conducted the validation of the measured duration data as input variable. The durations of the activities in simulation techniques were randomly generated under the specific distributions that could be established based on the measure duration data from construction sites. The actual conditions of construction sites had great effects on the measured duration data that would be used for simulation techniques. Thus, it is necessary that the additional measure should be applied to remove the biased data, i.e., an outlier in statistics. In this study, outliers were detected in the measured duration data of both the unloading process and the installing process: -The unloading process: among a total of four cycles for the activity 4. "Material checking" (i.e., node 4 in Fig. 2) in the unloading process, the measured duration data in the first cycle (i.e., 9.300) was determined to be statistically distant from the other data (refer to Table 6). This outlier was considered as measurement error and removed from the dataset. As a result, the simulated productivity was improved from 5.014 (cycle/hr) to 7.050 (cycle/hr). In addition, compared to the four measured productivity of the unloading process, the significance level α (0.136) in the T-test was determined to be higher than 0.05. Namely, it could be concluded that there was still no significant difference between the measured and simulated productivity in the unloading process (refer to Table 7). The graph also indicates that the simulated productivity is close to the measured productivity. -The installing process. As shown in Figure 6, the boxplot can detect outliers of the measured duration data for the activity 8. "Preparing" (i.e., node 8 in Fig. 4) and the activity 22. "Dropping winch hook" (i.e., node 22 in Fig. 4) in the installing process. Among a total of ten cycles in the installing process, the duration data collected in the fifth cycle (i.e., 15.467 of the activity 8. "Preparing" and 15.517 of the activity 22. "Dropping winch hook") were determined to be statistically distant from the other data (refer to Table 8). These outliers were considered as measurement error and removed from the dataset. Figures 7 and 8 show the statistical distributions of the measured duration data for the activity 8. "Preparing" and the activity 22. "Dropping winch hook" in the installing process before removing the outliers. Figures 9 and 10 show the improved statistical distributions of the measured duration data for the activity 8. "Preparing" and the activity 22. Journal of Civil Engineering and Management, 2017, 23(2): 163-172 Table 5. T-test results between the measured and simulated productivity by the main process in curtain wall construction project Class Initial simulation model .311 ---Note: 1 2, 3, 4, 5, 6, 7, 8, 9, and 10 stand for the measured productivity by the main process from the first to tenth cycle.
"Dropping winch hook" in the installing process after removing the outliers (i.e., the duration data collected in the fifth cycle). As a result, the simulated productivity was improved from 3.096 (cycle/hr) to 4.494 (cycle/hr). In addition, compared to the nine measured productivity of the installing process, the significance level α (0.311) in the T-test was determined to be higher than 0.05. Namely, it could be concluded that there was no significant difference (i.e., significant similarity) between the measured and simulated productivity in the installing process (refer to Table 9). The graph also indicates that the simulated productivity is close to the measured productivity. As shown in Figures 7 to 10, removing outliers from duration datasets in the activities of "Preparing" and "Dropping winch hook" (i.e., nodes 8 and 21 in Fig. 4.) made duration information changed. Depending on this change, duration input information in the simulation model shown in Figure 4 was changed to beta distribution with four parameters of 0.367, 1.333, 0.308, 0.334 from beta distribution with four parameters of 0.367, 15.467, 0.184, 0.372 in node 8 and changed to exponential distribution with one parameter of 0.549 from beta distribution with four parameters of 0.450, 15.517, 0.146, 0.337 in node 21 in Figure 4. Note: The activity 4. "Material checking" can be found in node 4 of Figure 2. Note: 1 2, 3, and 4 stand for the measured productivity of the unloading process from the first to the fourth cycle; and 2 nd measure refers to the validation of the measured duration data as input variable, where an outlier in the unloading process was removed.    , 3, 4, 5, 6, 7, 8, 9, and 10 stand for the measured productivity of the installing process from the first to the tenth cycle; and 2 nd measure refers to the validation of the measured duration data as input variable, where an outlier in the installing process was removed.

3 rd measure: validation of the simulation model compared to the actual construction process
Through the T-test results between the measured and simulated productivity in the initial simulation model, it was concluded that there was the significant difference between the measured and simulated productivity in the lifting process and the installing process. By taking the first and second measures as explained in Sections 2.2 and 2.3, it was concluded that there was no significant difference (i.e., significant similarity) between the measured and simulated productivity in the installing process, but there was no change in the lifting process (i.e., significant difference between the measured and simulated productivity in the lifting process). Thus, it is necessary that the additional measure should be applied to better reflect the actual conditions of construction sites. That is, the initial simulation model for the lifting process should be carefully reviewed and modified.
As shown in Figure 11, the initial simulation model for the lifting process was modified as follow. Through the reviews of the initial simulation model shown in Figure 3, it was concluded that the activity 7. "Preparing", which is the precedent activity of the activity 5. "Linking crane", cannot start until the crane is arrived at the ground floor after the activity 11. "Unlinking crane" is ended in the target floor. Thus, the activity 14. "Available" as QUEUE node was newly added to establish the relationship between the activity 11. "Unlinking crane" as NORMAL node and the activity 7. "Preparing" as COMBI node.
As a result, the simulated productivity was changed from 6.664 (cycle/hr) to 4.306 (cycle/hr). In addition, compared to the five measured productivity of the lifting process, the significance level α (0.106) in the T-test was  determined to be higher than 0.05. Namely, it could be concluded that there was no significant difference (i.e., significant similarity) between the measured and simulated productivity in the lifting process (refer to Table 10). Note: 1 2, 3, 4, and 5 stand for the measured productivity of the lifting process from the first to the fifth cycle; and 3 rd measure refers to the validation of the simulation model compared to the actual construction process, where the initial simulation of the lifting process was modified.

Conclusions
This study aimed to develop the framework for the validation of simulation-based productivity analysis, which consisted of three measures: (i) 1 st measure: validation of the measured productivity data as target variable; (ii) 2 nd measure: validation of the measured duration data as input variable; and (iii) 3 rd measure: validation of the simulation model compared to the actual construction process.
To verify the feasibility of the proposed framework, this study focused on the curtain wall construction project of "S" office building as a case study. The results of this study are summarized as follows: -The series of three measures enabled the simulated productivity to be statistically close to the measured productivity. Namely, the proposed framework can make the relationship between the measured and simulated productivity more statistically similar. Boxplot was used to identify the outliers from the collected dataset. Through the careful reviews of the initial simulation models, they were also modified to better reflect the actual conditions of construction sites. Finally, the T-test was applied to investigate the statistical difference (or similarity) between the measure and simulated productivity. The significance level α in the T-test for the unloading process, the lifting process, and the installing process in the curtain wall construction project were determined at 0.136, 0.106, and 0.311, respectively, which was higher than 0.05 with 95% confidence interval. As a result, it was concluded that there was no significant difference (i.e., significant similarity) between the measured and simulated productivity. -In addition to the statistical similarity, the simulated productivities in all three main processes in curtain wall construction were generally higher than the measure productivities. This was because there were a lot of unpredictable factors in construction sites that can result in delays (Halpin, Riggs 1992). Thus, it was concluded that the final simulation models for all three main processes in curtain wall construction were appropriately established in accordance with the actual conditions of construction sites. -The proposed framework could enable executives and managers in charge of project planning and scheduling to statistically evaluate the significant difference between the measured and simulated productivity and to accurately predict the simulationbased productivity in the complicated and unpredictable construction project. It could be also applied to any other country or any other type of construction project. Meanwhile, this study has manually assessed the applicability of the proposed framework through a case study. If the proposed framework can be developed as the decision-support system using Microsoft-Excel-based VBA or the network-based real time system in the future study, it can be more useful for practical purposes and offer a systematic management foundation as a platform.