CONCEPTUAL FRAMEWORK FOR INTEGRATING BIM AND AUGMENTED REALITY IN CONSTRUCTION MANAGEMENT

. The need for systematic data collection and processing to generate real-time building site progress information is critical. Building Information Modelling (BIM) provides the benefit of aggregating information about the building site on a single platform. Augmented reality (AR) emerges to enhance BIM concerning visualization of the building site, through processing and automatic absorption of information. This work aims to analyse the potential of AR association to BIM, by adopting an approach based on literature review. Trends in contemporary research are checked by categorizing applied research methods, areas of expertise, and AR technologies. Publications produced between 2008 and 2018 from journals of architecture, engineering, and construction areas in databases Web of Science, SciSearch, SCOPUS, INSPEC, Google Scholar, Academic OneFile, EBSCO, OCLC, VINITI, SCImago, and ProQuest were investigated. As main results, it was found that the case study approach was adopted in 41% of the publications analysed. The building site inspection was the research object in 48% of papers. Fiducial markers, GIS/GPS, laser scanners, and photogrammetry emerged as main options for automatic data capture on the progress of the building site. Integration between AR and BIM has the potential to solve information processing problems and improving construction management


Introduction
Current construction industry's projects are increasingly complex and challenging to control. Solutions have been developed based on the use of information and communication technologies. BIM was created by aiming to produce technology to absorb the information within a 3D visualization environment (Grazina, 2013). Through BIM, project's stakeholders can work collaboratively and simultaneously, minimizing the risk of incompatibilities. At any stage of the project, BIM systems insert, extract, update, or modify the information contained in models (Martins, 2014).
BIM has proven to support various life-cycle activities and assist managers in decision making. Despite the increasing use of BIM in recent years, most existing buildings do not have complete information within the model, due to changes in the construction phase. Incomplete or even incorrect information in the records are the main reasons for the low level of efficiency in construction management. A high precision model should be created, inserting all information being carried out in the construction stage, to generate a representation of the reality about what was executed in the construction site (Hamledari, Azar, & Mccabe, 2018). Opposing this need, BIM has been restricted to a representation and simulation tool, presenting obstacles to interact with the vast amount of parameterized data and the gap between the plan and the executed project (Wang, Kim, Love, & Kang, 2013a;Hamledari et al., 2018).
Integration between BIM and augmented reality (AR) can work as a reliable tool for coordination and communication. Visualization of the construction, associated with the planned model, can improve the identification, processing, and communication of progress discrepancies in the construction site (Golparvar-Fard, Peña-Mora, & Savarese, 2009b;Shin & Duston, 2010;Park, Lee, Kwon, & Wang, 2013;Kwon, Park, & Lim, 2014). Although this visualization may theoretically be the primary means of communicating BIM content in-place in 3D, there is cur-rently no complete understanding of how this can impact the building and performance of industry professionals (Chalhoub & Ayer, 2018).
Fast development of information technology provided an opportunity to adopt AR in various industrial sectors. AR concept was well described by Azuma (1997), who defined it as the linkage between real and virtual elements, in any degree of complexity, increasing both individual and collective visual perception of graphics information. According to Grazina (2013), AR is a technology that allows overlapping information and computer-generated graphics to real-world images. AR makes possible to combine a real environment with computer-generated information, developing a space in which generated computational elements are superimposed on user's real field of vision (C. Kirner, Zorzal, & T. G. Kirner, 2006;Wang et al., 2013a).
According to Fernandes (2012), a virtual window such as a computer screen, mobile phone or a head-mounted display (HMD) can produce an AR experiment. It is necessary to select the type of tracking that will allow associating images with digital information to integrate the models. The process comprises identifying the user's point of view, position, and orientation, about the elements in the space that have associated digital information. Position and orientation sensors such as laser scanners, GPS, gyroscopes or accelerometers (Golparvar-Fard, Bohn, Teizer, Savarese, & Peña-Mora, 2011a;Behzad & Kamat, 2007;Irizarry, Gheisari, Williams, & Roper, 2014;Williams, Gheisari, Chen, & Irizarry, 2015) and visual markers, also known as fiducial markers (Kiziltas, Akinci, Ergen, & Tang, 2008), which are identified by a combination of digital cameras and computer vision algorithms can achieve the tracking process. Each technique has advantages, regarding costs, practicality, and maintainability. On the other hand, the limitations are related to low resolution, precision, and computational cost (Fernandes, Cunha, Lopes, & Mota, 2011). Peres, Scheer, Faria, and Vian (2015) emphasize the need for some devices to use AR as a tracker, virtual information (text, image, and video), and video camera. Moreover, they suggest the use of a Global Positioning System (GPS) as an alternative to the visual markers. AR system was initially used in scientific visualization applications, as described by Caudell and Mizell (1992). Furthermore, it was applied in aircraft parts assembling, power plants inspection (Klinker, Stricker, & Reiners, 2001), medicine (Vogt, Khamene, & Frank, 2006), entertainment games (Fan & Liu, 2011;Juan, Llop, Abad, & Lluch, 2010), and construction of building pipes (Hou, Wang, & Truijens, 2015).
Keen interest has emerged to introduce AR in architectural, engineering, and construction sectors (AEC) by improving conventional methods of project visualization, monitoring, and control of activities (Wang & Love, 2012;Chi, Kang, & Wang, 2013;Wang et al., 2013a). To reinforce this idea, Rankohi and Waugh (2013) argue that the complex nature of AEC provides a significant demand for information to evaluate, communicate, and monitor the progress of construction site operations. Chi, Chen, Kang, and Hsieh (2012) point out main shortcomings identified in AEC can be detected in the lack of information regarding work on construction site, gaps between planning and execution, and interaction among project's actors. The authors argue that AR can contribute to solving these difficulties, becoming a vital ally to the development of the construction industry.
AR was initially studied by Sutherland (1968), who proposed a display using a device connected to a computer put on observer's head, denominated HMD. After, a joint effort conducted by the United States Air Force, the Massachusetts Institute of Technology (MIT), and NASA Research Centre has taken over development of AR. Subsequently, Boeing Corporations' scientists Caudell and Mizell (1992) developed an experimental AR system to aid assembly work. Loomis, Golledge, and Klatzky (1993) created a GPS-based AR system with navigation assistance to visually impaired people with spatial audio overlay. Mobile devices and the advancement of computing technologies have leveraged development and popularity of AR (Starner et al., 1997;Kopsida & Brilakis, 2016). In this sense, Feiner, Macintyre, Höllerer, and Webster (1997), Azuma (1997), and Azuma et al. (2001) developed prototypes for the mobile AR system, reporting advances and constraints. Klinker et al. (2001) developed an AR system for power plants inspection. Friedrich and Wohlgemuth (2002) presented another application of AR to assist in solving electrical problems in Ford vehicles assembling process. Wang and Dunston (2007) have used AR in a training platform for equipment's operators of the heavy construction industry. Golparvar-Fard et al. (2009b) applied the model using a localization system with overlapping photographic images of the environment, generating information from the context of the construction site, helping to monitor its progress. Behzadan, Timm, and Kamat (2008) and Hakkarainen, Woodward, and Rainio (2010) applied the GPS to automatically capture the data regarding the operations performed on the construction site. As a result, they presented a framework that could be reused by any AR application. Golparvar-Fard, Peña-Mora, and Savarese (2011b) produced a conceptual platform to promote integration between BIM 4D and AR for monitoring and to plan the activities of a building. Yeh, Tsai, and Kang (2012) applied a location-based information technology called iHelmet for services performed at the construction site in order to improve the efficiency of the transfer of information on the progress of the work through real-time visualization. Wang et al. (2013a) used AR to control the progress of a gas installation project. Jiao, Zhang, Li, Wang, and Yang (2013) developed an application that integrated AR and BIM, showing the use of construction 3D objects information modelling embedded in an augmented reality online environment. The contribution was to use an open platform, called web3D, where on-premises images are processed and registered with virtual objects. Grazina (2013) described a theoretical concept of an integration platform for the BIM and AR technologies using GPS as the tracking method. They found as results a database generated by the information extracted from the integration of BIM and AR, feeding back the planning. Irizarry et al. (2014) studied the application of AR as a tool to support the activities of building pipe installations, through the position and orientation of the observer. The authors developed a system called In-foSpot (Information Surveyed Point for Observation and Tracking), which uses tracking sensors such as gyroscope, accelerometer, and GPS embedded in a mobile data collection device. They concluded the solution is affordable and helps facility managers in their routine tasks, since the display space may be accompanied by information on projects in a single interface. H. S. Kim, S. K. Kim, Borrmann, and Kang (2017) proposed an AR-based 4D CAD system developed to reflect real-time construction site information and provide a practical simulation of 4D and 5D models. The system manages the schedule of activities through the images extracted in real-time, using web cameras installed in the construction site. Li, Yi, Chi, Wang, and Chan (2018) investigated the state-of-the-art application of virtual reality and augmented reality technologies considering safety in the productive processes carried out in the construction industry.
Considering the previous scenario, answers to the following research question must be sought: how can BIM be integrated with AR technology? The ideal solution to this problem would involve an experimental investigation considering the development of a system integrating BIM and AR technologies. However, before this critical step of the study, appropriating the results of the research carried out should be done, in order to verify what already a conceptual basis on the subject has, which, in the view of the authors of this work, represents a complex research problem. As formerly discussed, various proposals integrating AR technologies to BIM have been developed, but there is not a comprehensive study associating technological options to specific situations. It is in this research opportunity that this work has been inserted. Therefore, to fill the knowledge gap in the area, this work aims to carry out a systematic literature review to identify the potential of integrating AR to BIM, especially concerning the adopted AR technology and the purpose of its application. The work contributes to state of the art by getting involved with the improvement of the visualization of the activities carried out in the construction site in real time.
The paper is organized as follows. Section 1 presents the methodology used to collect and analyze publications for the review. Section 2 offers an analysis of the potentialities of integration between augmented reality and BIM, through the discussion of the research carried out in the area. In Section 3, trends related to traceability technologies are discussed. Finally, a conclusion about the study is presented.

Research methodology
In this work, a systematic literature review produced between 2008 and 2018 concerning AR applications in the construction industry related to construction management was carried out. The research approach adopted was based on the proposal presented by Rankohi and Waugh (2013), as shown in Figure 1.
Initially, indexed journals were selected in the major databases of research in the AEC area, such as Web of Science, SciSearch, SCOPUS, INSPEC, Google Scholar, Academic OneFile, EBSCO, OCLC, VINITI, SCImago, and ProQuest. Articles were selected considering the existence in their titles and summaries of the keywords combinations "Building Information Modelling" and "Augmented Reality" and "BIM" and "Augmented Reality".
Thirteen journals were selected in the initial stage of the research, as presented in Table 1.
The definition of categories was extracted from the 64 articles found in the search within the selected journals in the databases. Publications were considered once when they appeared in more than one database. Each article was classified according to the periodical it was published, the publisher, the first author of the work and his / her country of residence.
Considering the aspects related to integration between BIM and AR, four criteria were selected to review the literature: the research approaches, areas of activities, technologies used to integrate BIM to AR, and trends in the application of AR technologies. The reviewed articles were analyzed according to these characteristics in order to build a framework to provide future directions in the development of integration between BIM and AR.
A quantitative and qualitative analysis of the articles was carried out, classifying them in relation to the quantity of annual publications, publications by journals, publications by country of residence of the first author, publications by the first author, publications by research methodology employed, publications by area of activity, and publications by technologies used in AR tracking. The annual publication history was analysed considering each one of the 13 journals selected in the study. Next, the trend of production of articles considering the theme associating BIM with AR in construction management was verified. The following analysis involved the identification of countries where there was significant interest in the subject studied in this article, considering the first author's country of residence. After, an analysis was made in search of the author who had appeared most in publications associating BIM and AR. In the analysis of the work concerning the research methodology, based on Rankohi and Waugh (2013), four approaches were considered: case study, experimental/empirical study, proof of concept (proof of principle study), and literature review. A case study is a research method in which the researcher passively analyses (without making interventions) a contemporary phenomenon over a period. Experimentation is an empirical scientific method in which the researcher arbitrates between competing models or hypotheses. A proof of concept has a research approach in which an assumption method or model is put to demonstrate its feasibility or to check whether a concept, theory or prototype has the potential to be used. The literature review is based on research using a method that considers the critical points in the knowledge chain, including substantive results, as well as methodological contributions to a given topic (Cohen, Manion, & Morrison, 2007;Rankohi & Waugh, 2013).
Another analysis involved the investigation of the technologies, purposes, and areas related to the association between BIM and AR in the construction industry. AEC industry has many areas of research. In this research, inspection of activities in the construction site, verification of the execution of the operations in the construction site in comparison to project (Chi et al., 2013), and building maintenance were considered (Jiao et al., 2013;Olbrich et al., 2013;Nagy, 2013). Also, it was investigated technological development in the process of data capture in virtual environments of the model integrating BIM and AR. For this analysis, the main existing and developing technologies in the area were analysed.

Preliminary analyses
The first analysis carried out the history of publications in the 13 journals previously presented in Table 1. Results obtained are shown in Table 2.
Among the journals selected in this research, we highlight Automation in Construction, with 25 publications, corresponding to 34.4% of the articles found. The second journal with the highest production was the Journal of Computing in Civil Engineering, with nine publications, corresponding to 14.7% of the total. The Journal of Infor- mation Technology in Construction -ITcon appeared in third place, with seven publications, accounting for 11.4% of the total. The first three journals that appear in Table 2, present 64% of the articles. These journals have a policy of publication of topics that aggregate research applying information technology within the AEC sector, becoming a reference to the researchers. When analyzing data in Table 2, it is interesting to note that, considering the publications of all journals together, there is an exceptional volume of scientific productions (16 articles) in 2013, the year in which journal Automation in Construction focused on the study of AR in architecture, engineering, and construction, through a special edition in the issue. In 2014, the nine publications appeared to have been influenced by the 2013 call. In the remaining years, excluding 2017 and 2018 (which will be discussed below), there were not a substantial number of publications, ranging from a minimum of 2 to a maximum of 5 (in 2011), indicating little repercussion of actions in the construction sites related to AR. In 2017 (with eight publications) and 2018 (with nine publications), however, there seems to be a clear trend towards improvement efforts within the construction industry for the use of AR.
The subsequent preliminary analysis involved the identification of countries where there was an interest in the subject AR. This study was based on the consideration of the first author's country of residence. USA researchers ranked first in 34% of journals, followed by authors from Australia with 25% and South Korea with 10%. China and Canada have lead authors with the same number of publications, representing 5% of the total.
The following analysis sought to determine the author who had most published on the topic BIM associated with AR. Xiangyu Wang appeared in first place, with 11% of articles published. It was found that Wang's publications were distributed from 2008 to 2015. Among his various works, X. Wang prospectively investigated the applica-tion of AR in the architecture and design sectors in 2009 (Wang, 2009). In this research, after analyzing the various existing technological options, the author argued that an adequate system of AR should work on any industrial environment without the need to study it before. Also, the AR system should function properly in open or closed environments. When analyzing the evolution of the AR, it is verified that the challenges launched a decade ago by Wang were overcome and became a reality. In 2013, X. Wang published a review on AR in the built environment, classifying and demonstrating implications for future research (Wang et al., 2013a). In the same year, X. Wang and other researchers presented a proposal for a conceptual framework to integrate BIM with AR (J. Wang, X. Wang, Shou, & Xu, 2013b). Wang and his research group have suggested that AR should operate in conjunction with tracking and detection technologies such as RFID, sensors, and motion tracking. In 2014, X. Wang and his team created an integration of BIM with AR, presenting a model that solved real problems of the oil and gas industry, such as low productivity in information retrieval, the tendency to make mistakes in building system and low communication efficiency (Wang et al., 2014a). In the same year, X. Wang and his research team developed a collaborative project, integrating AR technologies and telepresence (X.

Researches approaches
Among the 64 articles analysed, 42% adopted a proof-ofconcept approach, 32% carried out bibliographic reviews, 14% performed case studies, and 13% conducted experimental work, as shown in Figure 2. A proof of concept approach is used to indicate a practical model that can prove the concept established by a research. It may also be considered a generally summarized or incomplete implementation of a method or an idea carried out to verify that the concept or theory in question is capable of being exploited in a useful way. As highlights of the use of the proof of concept approach, Wang et al. (2014b) and Park et al. (2013) obtained a visualization of parts of the project using fiducial markers. Grazina (2013) and Martins (2014) produced a conceptual platform getting updates on progress information. In research using case studies, Clemente and Cachadinha (2012) addressed the visualization of AR in updates of information regarding the progress of the building site. Kim et al. (2017) examined the construction of a bridge over the AR performing simulations. In the research using the experimental approach, Chu, Matthews, and Love (2018), Shin, Park, Woo, and Jang (2013) and Shin and Dunston (2010) demonstrated detection of defects using AR was significant, improving the perception about occurrences of this nature. In studies employing literature review, Rankohi and Waugh (2013) found in 133 articles a strong tendency of application of AR in the construction industry. Irizarry et al. (2014), Wang et al. (2013a), and Leite et al. (2016) found visualization for the construction helps planning, operation, and maintenance of work. Proof-of-concept method contributes to the consolidation of innovative aspects of AR that still need to be evolved, explaining the high percentage of publications in the area using this scientific approach. The reduced pres-ence of case studies in the analyzed publications can be explained by the reduced amount of study objects in the maturity stage that allows the application of this research approach. Although it represents small participation in the sample of articles analyzed in this work, the experimental approach seems to be the most adequate at a time when the creation of an integrating model between BIM and AR is still in development. Figure 2 supports some considerations regarding investigations on research theme adopted in this work. It was verified that 42% of the researches took proof of concept approach. These investigations involved studies with prototypes, allowing to test the potential for integration between BIM and AR technologies in the construction industry. Experimental approach represented only 12% of the investigations conducted, indicating there is a need for further practical researches, which coexists with intense technological innovation requiring field discoveries and validations.

Areas of activities
Another analysis was directed to the identification of the areas of activities in which AR applications were applied in construction management. Table 3 presents the result of this study. Behzadan and Kamat (2007), Golparvar-Fard et al.  Fazel and Izadi (2018), and Chalhoub and Ayer (2018) conducted a large number of applications of AR to assist in the execution and detection of information on the progress of activities in the building site. Nagy (2013) researched the implementation of AR for a building maintenance system. Hou et al. (2015) applied AR to the assembly of a piping system, achieving an improvement in productivity and professional performance and reducing the cognitive workload. For the infrastructure area, Behzadan, Dong, and Kamat (2015) worked in trench excavations for infrastructure facilities, providing, through the application of AR, higher reliability of the location for the operation. Figure 3 shows the percentage of shares of the identified research areas.
As presented in Figure 3, the inspection area is the most significant occurrence (with 64% of the research done). Maintenance appears with 11% of the researches, installation of building systems appears with 11%, and infrastructure area represents 12%.

Technologies used to integrate BIM with AR
In addition to previous studies, a particularly important aspect in the study of AR interaction to BIM involves understanding the technological development in the area for automatic data capture in a virtual environment. These technologies provide user's interaction, producing experiences in immersive environments, desktops, mobile devices, stationary environmental scanning equipment, HMD devices, and glasses (Rankohi & Waugh, 2013). Technologies studied in the analysed articles are presented in Table 4. In general, when assessing a tracking technology for the tasks that users can perform, consideration should be given to the workload, the desired range of motion, accuracy, and precision required, and the likelihood of crawler occlusion (Dunston & Wang, 2011). For Meža et al. (2014), the primary technical challenge of AR systems not using landmarks as visual markers is to determine the user's position in space accurately.
Photogrammetry is one of the most popular tools for acquiring three-dimensional data (3D) and providing a digital surface model (Liarokapis, 2007). This technology can provide simultaneous real-time positioning information over multiple entities, as well as self-calibrate and minimize positioning errors when multiple cameras are installed (Zhou, Duh, & Billinghurst, 2008). Development of this technology has become popular due to the practicality of its acquisition, low cost, and easy manipulation (El-Omari & Moselhi, 2008;Golparvar-Fard et al., 2009b;Bae et al., 2013;Barazzetti et al., 2015). However, for Golparvar-Fard et al. (2011a), automating the detection of the progress of a productive operation from the images of a site is a challenge, as there are limitations on its use at the construction site. These limitations occur due to climatic changes, affecting lighting and making it difficult for the camera to capture the image and automatically overlay the projected with the actual.
Lasers scanners are promising to automate data collection. However, they are still expensive, and implementation of this technology is a challenge at the construction sites. Limitations such as movements in the field of view of the scanner impair the continuity of the capture of spatial information. Also, the level of detail within the captured components is reduced, requiring regular calibrations as well as a warm-up time to start capturing the cloud of points. Other factors are associated with the difficulty of transporting the equipment within the construction site, the impossibility of using it in closed environments, delay in data processing and difficulty of generating semantic information that can associate scanned points with the respective structural components. Therefore, the laser scanner is still an error-prone system (Golparvar-Fard et al., 2009b).
Fiducial markers can be easily implemented in construction sites (Le et al., 2015;Zhou et al., 2017). However, the use of markers within the construction site becomes difficult because the recognition rate is reduced under sunlight. In this sense, improving the recognition of markers through image pre-processing should be considered (Kwon et al., 2014). Also, the calibration of the camera must be precise, to obtain the alignment in the images (Fazel & Izadi, 2018). Figure 4 presents a comparative analysis of the most used tracking technologies: Table 4 and Figure 4 present the technologies applied in the investigations conducted between 2008 and 2018. GPS / GIS mapping represents the highest percentage of use, with 46% of applications, followed by fiducial markers, with 40% photogrammetry, with 10% and laser scanner, with 5%. Significant amounts in the utilization of GIS / GPS and fiducial markers derive from their low costs, as well as the functionality and practical process of implementation compared to the other technologies.

Considerations of some trends related to AR technologies
With the improvement of positioning systems, AR devices are not required to rely on markers to know where to locate the virtual element. Through the real environment elements analysis, the new systems can identify relationships between the camera and the real-world.
Advancement of tracking and detection technology relies on research and development efforts from industrial and academic areas. In addition to accurate and longrange tracking, it is essential for AR systems to have high quality and real-time rendering (Wang et al., 2014b).
Google's online services and software company introduced its Google Glass project in 2013 in the form of a head-mounted display device (HMD), which allows the user to interact with the real world. Continuing this evolution, in 2016, DAQRI and Autodesk developed an HMD associated with virtual reality, with the goal of transforming the productive processes in the construction industry. In 2017, the Microsoft organization launched the Hololens (updated in 2019 to Hololens 2), which is an AR glasses, expanding its operations to the construction industry, by having an integrated system and sensors more potent than the existing mobile devices. HMD based technology moves beyond simple viewing displays, including more sophisticated environmental inputs, capturing spatial sounds, and location elements. Improvement of human and environmental understanding, functionality, ergonomics, and connectivity with information processing technologies means new HMD technologies surpass alternatives discussed in this paper emerged in the last decade.
There is a trend directing the technological development aimed at modelling of AR based on the advance of sensors embedded in mobile devices and software development packages, called SDKs, that allow the structuring of 3D models and position them in environments without the need for markers. This technology identifies critical points in the environment and tracks their movements by combining this information with additional information from the equipment's sensors to determine the position and orientation of the device as it moves through the environment.
AR technology is suffering a fast evolution, and there is not a consensual understanding of what device will be the most prominent. Challenge for future research lies in the development of accessible and practical devices in the implantation process within the construction site, generating a higher precision and occlusion activity inspection at the construction site.

Conclusions
This research found a high potential for applicability of integrated AR to BIM modelling to assist in operations inspection, building maintenance, infrastructure, and instal- 4% % lation assemblies. This integration allows interacting with the user intuitively, generating immersion within the BIM models, to reduce the response time concerning possible solutions for re-fitting activities at the construction site. The most intuitive visualization platform seeks to quickly update the planning of the construction through information generated in the building site (as-built). The potential use of the integration between BIM and AR increases, due to the evolution in the performance of portable computers, mobile devices, and other solutions of visualization devices in a virtual environment. In this sense, it is verified that academic investigations are focused on the development of structures to perform the inspection in the constructions for data capture.
Techniques of tracking images such as visual (fiducial) markers and use of positioning sensors for information and guidance, such as GPS has stood out. Much of this dissemination is due to low cost and ease of use and deployment of these technologies. However, there are still difficulties in obtaining more significant results using AR, associated with limitations of precision, calibration, and occlusion. Investments to popularize AR have been improving the performance of smartphones, mobile computers, and devices using external sensors and embedded accelerometers and gyroscopes.
As a recommendation for future work, it is essential to investigate how AR impacts the performance of work, in terms of its quality, speed of execution, reduction of losses and increase of productivity of the workforce. Also, an integrative model between BIM and augmented reality must be developed in order to validate the promising implementation of the integration.