PROJECT PORTFOLIO SELECTION PROBLEMS: A REVIEW OF MODELS, UNCERTAINTY APPROACHES, SOLUTION TECHNIQUES, AND CASE STUDIES

Project portfolio selection has been the focus of many scholars in the last two decades. The number of studies on the strategic process has significantly increased over the past decade. Despite this increasing trend, previous studies have not been yet critically evaluated. This paper, therefore, aims to presents a comprehensive review of project portfolio selection and optimization studies focusing on the evaluation criteria, selection approach, solution approach, uncertainty modeling, and applications. This study reviews more than 140 papers on project portfolio selection research topic to identify the gaps and to present future trends. The findings show that not only the financial criteria but also social and environmental aspects of project portfolios have been focused by researchers in project portfolio selection in recent years. In addition, meta-heuristics and heuristics approach to finding the solution of mathematical models have been the critical research by scholars. Expert systems, artificial intelligence, and big data science have not been considered in project portfolio selection in the previous studies. In future, researchers can investigate the role of sustainability, resiliency, foreign investment, and exchange rates in project portfolio selection studies, and they can focus on artificial intelligence environments using big data and fuzzy stochastic optimization techniques.


Introduction
A portfolio consists of various components such as projects, programs, portfolios, and other tasks like maintenance and ongoing operations. All the components are grouped in order to ease the management of the work so that the strategic business objectives could be reached effectively. The projects or programs of a portfolio are not necessarily interdependent or directly related. In other words, it can be stated that they are normally unrelated. On the other hand, the components could share a common resources pool or even compete for funding (Project Management Institute [PMI], 2008). To put differently, a set of projects that share and compete for limited resources forms a portfolio of projects. A portfolio is directed under the sponsorship of a particular organization .
Firm managers should select portfolios of projects to invest to achieve objectives. Project portfolio selection (PPS) is known as a periodic and continuous effort that involves selecting and funding portfolios of projects that are supporting organizations stated goals and objectives. An important aspect of this decision-making process is considering resources and other constraints (Schniederjans & Santhanam, 1993;Killen & Hunt, 2013). In other words, one of the most important reasons for PPS is the fact that the accumulated funding that all the candidate projects need highly exceeds the available investment resources (Mohagheghi, Mousavi, & Vahdani, 2016;Mohagheghi, Mousavi, Aghamohagheghi, & Vahdani, 2017a;Mohagheghi, Mousavi, Vahdani, & Shahriari, 2017b).
PPS has been an interesting point of many scholars in the last 4 decades. It is very practical in areas such as new product development (NPD) and research and development (R&D). Moreover, PPS is applicable in technology selection problems and similar topics (Iamratanakul and Patanakul, 2008;Mohagheghi, Mousavi, & Vahdani, 2015b).
Project portfolio selection background: PPS has been studied by many scholars in the last two decades. Due to the importance of research and development (R&D) projects, a large portion of studies was mainly concerned with R&D projects. One of the first studies of PPS was carried out by Chu, Hsu, and Fehling (1996). They developed a Decision Support System (DSS) for R&D project portfolio selection. However, most of the scholars refer to the initial work of Ghasemzadeh (1998, 1999) as the turning point of PPS studies. Their pioneer study (Archer & Ghasemzadeh, 1998) introduced a DSS that used a novel framework for PPS with six operational stages. Later Archer and Ghasemzadeh (1999) developed their framework. Figure 1 shows a project portfolio selection framework introduced by Archer and .

Project portfolio selection research objectives:
The main goal of PPS is to form an optimal portfolio of projects that simultaneously achieves the company's strategic objectives and considers the limitations that are imposed on the process. Moreover, controlling the risk and the performance objectives are some of the other goals that should be considered (Better & Glover, 2006;Ebrahimnejad, Mousavi, Tavakkoli-Moghaddam, Hashemi, & Vahdani 2012).
PPS studies have developed over the years. In the initial studies, financial criteria of projects formed the main focus. Later, frameworks were developed to attend to PPS with an emphasis on strategic criteria. Recently, there has been scattered focus on other criteria such as sustainable development, strategic alliance, the risk of investment and organizational readiness (Khalili-Damghani & Sadi-Nezhad, 2013a). The following presents some of the main objectives of PPS: -Maximizing financial conditions with indexes like net present value (NPV), return on investment (ROR), etc.; -Maximizing non-financial benefits; -Cost reduction; -Risk control; -Optimizing scheduling of activities; -Optimizing allocation of resources; -Handling uncertainty and vagueness. The significance of project portfolio selection research: Researchers have not treated PPS in much detail before 2000s that the number of studies on PPS had been very low. In fact, unlike project selection studies or portfolio selection studies, PPS studies became increasingly popular since the early 2000s. Studies of Ghasemzadeh (1998, 1999) paved the way and created a new path for scholars to conduct research on PPS.
In the next following decade (the 2000s), the number of studies on PPS increased slowly but stayed at a low level. However, in recent years (the 2010s), the number of studies on PPS has dramatically increased. To further illustrate the trend of PPS studies, the term "project portfolio selection" was searched in SCOPUS on 31st May 2019. The results are presented as follows: The number of documents by year is shown in Figure 2. This figure perfectly presents the increasing trend of PPS studies. Table 1 presents the results of searching on SCOPUS. Some of the main results are as follows: -In total, 259 papers were discovered.
-Growing trend of this topic in recent years can be observed.
-91.8% of the sources belong to the group of conference papers and journal articles. This implies that most researchers are attracted to project portfolio studies.     The best journal sources of PPS are presented as follows: -European Journal of Operation Research (10); -Annals of Operation Research (5); -Journal of Operational Research Society (5); -Expert Systems with Applications (4); -Information Sciences (3); -Sustainability (3).
When it comes to analyzing subject area, it can be observed that the topic is investigated mostly in areas of engineering, computer science, decision science, business, management and accounting, mathematics, social sciences, environmental sciences, and energy, respectively.
The rest of this paper is organized as follows: the method and base for the review in this paper are presented in the next section. In section 2, various factors and criteria involved in PPS decision-making are reviewed. Due to the uncertain project environment, section 3 reviews different approaches to uncertainty modeling tools. Modeling and selection approaches are reviewed in section 4. Section 5 presents a review of solutions approaches of PPS. Due to the importance of this decision-making process in a real-world application, a review of applications and real-life case studies is presented in section 6. Finally, further research directions for both academic and practitioners and concluding remarks are addressed in last Section.

Describing the method and base for the literature review
This paper presents a wider literature review at the intersection of project management and project portfolio optimization. A structured keyword search was applied to databases and major publisher websites to identify related papers for this review. Keywords such as "optimization", "selection", "evaluation", "mathematical modeling" were combined with project-related words such as "project portfolio", "project management", "construction project", "research and development" and "new product development". The papers were applied in a research method in which different categories like selection and evaluation criteria, uncertainty, modeling and scoring approaches, solution approaches and applications, and case studies were identified. In addition, all the papers which met the above criteria between 1993 and 2018 were extracted from Scopus or Web of Science. The technique for data collection and for the review is similar to approaches applied by Seuring and Muller (2008) and Seuring (2013). Figure 3 presents the study flowchart for the identification, screening, eligibility, and included articles. The taxonomy of the applied literature review method is depicted in Figure 4.
In this study, the literature search is used for material collection. In the review section, after setting the criteria and the categories for review, the literature was reviewed on the basis of the identified topics. This has led to a presentation of the survey for each section.
The terminologies applied in this paper such as "project portfolio", "project portfolio selection", "sustainable", "uncertainty", etc. are described in the context of this paper.
Criteria applied in content analysis: Two main categories of deductive and inductive approaches are often employed to form criteria for content analysis. Given the fact that to the best of our knowledge, comprehensive reviews on PPS did not exist, this paper has employed a deductive approach based on the existing studies on similar subjects to obtain review criteria. The following aspects will be discussed: -The selection and evaluation criteria such as financial, risk, strategic, green, social, sustainable, etc. are evaluated. The assessment contains in Tables 2 to 9 in which the papers applying various approaches to the studied criteria are categorized. Then, a brief critical analysis of the previous research based on the selected criteria is presented to evaluate the existing research and to find the potential gaps for further investigation. -Given the uncertain environment of projects, uncertainty is discussed in a separate section (Section 4). Different approaches to modeling and expressing uncertainty which have been applied in the literature are reviewed. This review contains an analysis of papers based on stochastic, fuzzy, grey, and uncertain theory tools.  -The modeling approach applied to PPS is reviewed in Section 5. Given the fact that there was no clear starting point, the categories were established based on an inductive approach. The main identified categories are frameworks and DSSs, optimizing and scoring methods. Frameworks and DSSs form a more general approach that could include optimizing and scoring methods. However, given their importance in the application, they are first reviewed and then optimizing and scoring methods are discussed. -Solution approaches are categorized by following the main groups mentioned in optimization literature reviews in Section 6. Therefore, two main categories of exact and heuristics and meta-heuristics solutions approaches are formed, and the papers are reviewed accordingly. -Given the fact that PPS is highly applicable in real-life problems, a separate section (Section 7) is presented to mention the applied cases of PPS. The aim of this section is to offer the areas in which PPS studies have been carried out.

Selection and evaluation criteria
In this section, the criteria which have been utilized in PPS problems by scholars in previous research between 1993 and 2018 are identified and categorized. In single-objective approaches often cost was referred to as the selection and evaluation criteria. However, since this problem affects various parts of an organization, often multi-criteria decision-making approaches are employed, and multiple conflicting criteria are addressed. The objective of this section is to give a review on various criteria applied in PPS problems.

Financial
Financial aspect was one of the first criterion that was investigated in PPS problems. Various ways have been tried to assess financial impacts of projects and project portfolios. Some of the applied criteria of financial assessments are net present value (NPV), financial return, and return on investment (ROI). Table 2 gives a summary of some of the research using financial assessment methods.
It can be observed that using NPV is one of the most common approaches in considering financial criteria in PPS. One of the trends in using NPV is simultaneously considering NPV and some measures of financial risk. Using variance and semi-variance of NPV in addition to NPV is one of the trends in financial assessment of PP. However, using NPV has its drawbacks. The main issue with applying NPV is that uncertainty profoundly influences this index and given the fact that at the initial phases of projects high degrees of uncertainty exist, the results would lose their reliability. One solution is to increase the level of knowledge by using experienced experts or using historical data from similar projects. Another solution could be using proper tools to model uncertainty.  (2014)

Risk
Risk is a vital topic in project portfolio management. Risk refers to a vague event or situation which, if it happens, makes major positive or negative impacts on at least one of the objectives of a project portfolio (PMI, 2008). Risks are manageable at the portfolio and project level. However, attending to risks at the portfolio level can enhance the effectiveness of the process (Aritua, Smith, & Bower, 2009). PMI's Standard for Portfolio Management (2008) groups risks of portfolio in three main categories of structural, component, and overall risks. Structural risks mean the risks that are related to the formation of the group of projects in addition to the potential problems among the elements. The second group is component risks that are the risks which the project manager has to escalate to the portfolio level for information or action. Finally, the last group of risks is overall risk that attends to the interdependencies between projects. This is more than only the sum of risks associated with single projects (Olsson, 2008). The management of risks a very substantial element of project portfolio management. Risk management gives the organization the power to handle opportunities and threats (Teller & Kock, 2013 Finding the best portfolio of projects requires addressing projects' risks. In the literature, risk has been addressed from various perspectives. Risk of investment, risk of project implementation and risk of uncertainty are some of the aspects. Table 3 presents various aspects of risks addressed in project portfolio selection literature. One of the aspects of addressing risks at the portfolio level is considering downside risk (e.g., Zhang, et al., 2011;Mohagheghi et al., 2017a;Li, et al., 2017). This approach divides the impacts of risks on the portfolio in two main groups of positive and negative impacts. Then, the approach tries to minimize only the negative consequences. In other words, risks have both positive and negative impacts and minimizing both impacts would reduce the efficiency of the method (Ebrahimnejad, Mousavi, Tavakkoli  Li, Cao, S. Li, Guo, and Zhao (2012), Rabbani, Najjarbashi, and Joudi (2013), Gurgur (2009)

Strategic criteria
PPS is a strategic level problem. To put differently in this process, the aim is to achieve strategic goals through project implementation. As a result, it is necessary to attend to strategy and strategic criteria in PPS. This factor has been used from the initial studies to the recent ones.
To present various forms of considering strategy in PPS studies, Table 4 is provided. Obviously, better offering the strategic criteria makes the studies closer to real-world conditions.

Green and environmental
Today's environmental condition has improved the necessity of considering green and environmental issues in different decision-making problems. When it comes to project evaluation, these issues become even more important. This is caused by the fact that projects have different environmental impacts that have to be regarded while making project-related decisions. Table 5 presents a review of various ways green and environmental criteria were applied in PPS.

Social
Social impacts of projects are taken in various PPS studies. For instance, a group of PPS studies mainly focuses on social PPS. These projects have different characteristics and therefore cannot be addressed by using financial approaches. Some of the studies concentrate on staff assignment and issue like learning in PPS (e.g. Gutjahr, Katzensteiner, Reiter, Stummer, & Denk, 2010;. Table 6 gives a briefing of approaches applied in addressing social criteria in PPS.  Chu et al. (1996) Strategic selection algorithm Ghasemzadeh (1998, 1999) Strategy Development (determination of strategic focus, using techniques such as strategic Mapping, and setting resource constraints) Olundh and Ritzen (2004) Strategic level decision making Carlsson et al. (2007) Strategic fit Gutjahr et al. (2008) The strategic benefits accrued from the increments of the efficiency values in objective function Stummer, Kiesling, and Gutjahr (2009) Strategic weights of competencies in objective function  Strategic gains in mathematical modeling Koppinen and Rosqvist (2010) Asset strategy Wen (2010) Strategy-oriented process model Zhang et al. (2011) Optimal investment strategy Zhu and Wang (2012) Strategic balance Abbasianjahromi and Rajaie (2012) Strategic planning Khalili-Damghani and Sadi-Nezhad (2013a), Khalili-Damghani and Tavana (2014) Strategic framework Kaiser, El Arbi, and Ahlemann (2015), Lifshits and Avdoshin (2016) Strategic response and goals Jeng and Huang (2015) Strategy for differentiating products and services Jadda and Idrissi (2015) (2015) Reinvestment strategy Table 5. Applying green and environmental criteria in PPS

Researcher
Social approach ,  Employee competencies and Staff assignment Koppinen and Rosqvist (2010) Staff issue and social changes in Infrastructure Sector Shou and Huang (2010) Maximizing the overall social efficiency of the market Wang and Shou (2011) Social objectives like maximizing social benefits and customer satisfaction Khalili-Damghani et al. (2013) Social Effect (Direct social effect of a portfolio of the project in a long-term period) Fernandez, Lopez, Mazcorro, Olmedo, and Coello (2013) Public project portfolio selection with highest social returns Zaras, Marin, and Boudreau-Trude (2012) Social, welfare and health Cruz-Reyes, Medina, and López (2013) A DSS for social PPS Rivera et al. (2013) Ant

Sustainability
The concept of sustainability is based on the interrelationship among social, environmental and financial development. Sustainable development cannot be reached without adequate understanding of financial decisions impact on the society and the environment (Hutchins & Sutherland, 2008). One example of application of sustainability is a sustainable market valuation of buildings (Zavadskas et al., 2017a). Sustainable project portfolio selection is a step towards organizational sustainable development. In recent years, a number of studies have employed the concept of sustainability in PPS. Table 7 presents these studies.

Other criteria
Since PPS is utilized in many areas, different criteria have been applied to get the optimal portfolio of projects. In other words, to find the best portfolio, it is necessary to identify criteria according to features of the application environment. Therefore, given the high applicability of this problem, it can be concluded that it is not possible to fully categorize all the criteria or groups of criteria used in PPS. However, to present other criteria that were applied in this problem, Table 8 provides a brief description of other criteria applied in PPS. One of the new trends in these studies that has made them closer to real-world conditions is addressing project interdependency and synergies. Projects, while selected together, can affect the level of required resources and efforts. On the other hand, they can affect the expected outcome while addressed together.

Uncertainty
In fact, in any real-world project selection process, two concepts increase the complexity of the process. One is the constraints and limitations imposed on the process, and the other one is the uncertainty that exists in the project evaluation (Mavrotas & Pechak, 2013a. In investment-related problems, experts often are handling insufficient data. Uncertainty has a vital impact on project management problems. Given the role of vagueness in project  (2017) Sustainable strategic decision making in an electricity company

Stochastic uncertainty
Using stochastic approaches is one of the methods applied in PPS. The stochastic theory is based on using historical data. Stochastic optimization covers a collection of tools applied to either minimize or maximize an objective function while dealing with randomness (Mousavi, Jolai, & Tavakkoli-Moghaddam, 2013). In recent decades, such methods have proved themselves as vital tools for science, engineering, business, computer science, and statistics. Often there are two main ways for randomness to enter the problem: one is the cost function, and the other one is the constraint set (Hannah, 2015). Despite the high applicability of this approach in various areas, using it in a project environment is not very common. The main reason could be the fact that projects are unique and having historical data in projects in some cases is not even possible. However, in some projects, data from similar past projects could be used to overcome this shortcoming. In Table 9 a review of studies that have used stochastic tools to find the best portfolio of projects is presented.

Fuzzy sets theory
In a project environment, vagueness in addition to the imprecision of information and lack of proper data make using experts' ideas inevitable. A proper tool in considering uncertainty is fuzzy sets theory. Many studies have utilized fuzzy sets theory to handle uncertainty in PPS (Ebrahimnejad et al., 2012;Mohagheghi, Mousavi, Vahdani, & Siadat, 2017c;Mousavi, Vahdani, Hashemi, & Ebrahimnejad, 2015). Through the years, the necessity for enhancing fuzzy sets theory arose as it was more utilized in real-world problems. One of classical fuzzy sets theory's inadequacies happens when an expert is expected to provide an exact opinion in a number in the interval For example, intuitionistic fuzzy sets address degrees of membership, non-membership, and hesitancy. This provides the ability to address agreement, disagreement and lack of knowledge in the process (Atanassov, 1994). The same thing is done with different levels of flexibility and constraints in Pythagorean (Yager, 2013) and Neutrosophic fuzzy sets (Smarandache, 2015). Type 2 fuzzy sets utilize fuzzy membership function. The complexity of such sets has led to the development of interval type 2 fuzzy sets (Mendel, John, & Liu, 2006). Table 10 shows a review of fuzzy set applications in PPS literature.  Fuzzy rule based approach Fernandez et al. (2013) Fuzzy outranking relations Zhu and Wang (2012) Fuzzy compound real option evaluation model of R&D project Abbasianjahromi and Rajaie (2012), Tavana et al. (2015) fuzzy multi criteria decision-making approach Perez and Gomez (2016), Perez, Gómez, Caballero, and Liern (2018) Fuzzy constraints in the model Mohagheghi et al. (2015a) Intuitionistic fuzzy sets Mohagheghi et al. (2015b) Interval valued fuzzy sets in mathematical modeling Mohagheghi et al. (2016) Interval valued fuzzy sets in MADM approach Alexey et al. (2016) Fuzzy multi-objective model Y. Liu and Y. K. Liu (2017) Robust fuzzy optimization Mohagheghi et al. (2017), Wu, Xu, Ke, Tao, and Li (2019) Interval type 2 fuzzy sets in mathematical modeling Wu et al. (2018) Triangular intuitionistic fuzzy numbers Lukovac et al. (2017) Neuro-fuzzy modeling Mohagheghi and Mousavi (2019) Pythagorean fuzzy sets Dong and Wan (2019) Fuzzy multi-objective linear programming

Grey theory
Another approach in addressing uncertainty in a project environment is using grey theory. Grey systems are effective tool for modeling incomplete information (Julong, 1989). These sets develop a way of presenting vagueness in systems. Grey sets use the basic concepts of grey numbers in grey systems and consider the characteristic function values of a set as grey numbers (Yang & John, 2012). This trend is new in PPS, and only a few studies have applied these sets. Bhattacharyya (2015) developed a grey approach for R&D project portfolio selection. Balderas, Fernandez, Gomez, and Cruz-Reyes (2017) presented a TOPSIS-Grey approach to handle project portfolio problem. Balderas et al. (2018) also applied the grey mathematical approach to address project portfolio optimization. Zhao, Wu, and Wen (2018) applied grey entropy to discuss the evaluation of green construction projects.

Uncertainty theory
Using uncertainty theory introduced by Liu (2007) is a new approach in addressing project uncertainty. Liu (2007) developed a new uncertain tool based on normality, duality, subadditivity, and product axioms. His presented approach has been used by Huang and Zhao (2014), Huang, Zhao, and Kudratova (2016),  and Yan and Ji (2017). Besides, Rough set theory (Tavana et al. 2019) is another approach in addressing uncertainty. Rough set theory applies upper and lower approximations to address uncertainty. In this section, uncertainty modeling tools in PPS were reviewed. Although different approaches were applied to model uncertainty, given the nature of PPS and the fact that this problem is applicable in various fields, there is no approach that would perfectly suit all the problems. For instance, Mohagheghi et al. (2017) (2019) utilized Pythagorean fuzzy sets to address project portfolio selection. However, hybrid methods have not been examined in PPS. In other words, in conditions where different tools work well for different situations, using hybrid tools could improve the approach. Therefore, using hybrid approaches such as fuzzy stochastic methods could be an interesting direction in uncertain PPS. Archer and Ghasemzadeh (1999) classified the approaches into five main groups of ad hoc methods, comparative methods, scoring approaches, portfolio matrices, and optimization approaches. In another categorization, Iamratanakul et al. (2008) grouped the project selection models into categories of scoring methods, economic methods, mathematical programming, real options analysis, simulation modeling, and heuristics methods. In this paper, in order to present different approaches used to address this problem, first studies introducing frameworks and decision support systems (DSSs) are presented. Then, optimization approaches are addressed. Finally, scoring and ranking methods are reviewed.

Frameworks and DSSs
One approach is PPS studies is introducing frameworks. Frameworks are employed to ease the portfolio selection process and provide flexibility. A framework is also a basis for decision support systems (DSSs). Using framework with computer support can provide several advantages such as recording and retrieving data needed in the analysis, providing computerized algorithms to do the necessary computations, display information, and enabling interaction with available data and aid decision making. In a DSS the software assists in integrating user tasks in each of the decision-making stages smoothly while providing a high level of usability (Archer & Ghasemzadeh, 1998). Table 11 provides a review of frameworks and DSSs used in PPS studies.

Optimization methods
One of the most common approaches in PPS is using optimization methods. Single, bi, and multi-objective models have been widely used to address this problem. To address uncertainty, stochastic, fuzzy and robust optimization techniques have been used in PPS. In mathematical programming, integer programming (IP), mixed integer programming (MIP), linear programming (LP), non-linear programming (NLP), quadratic programming (QP), etc. are used. However, given the vast area of applications of PPS and its varying features, it cannot be stated that which approach is the best and each problem requires its approach. In Table  12 a review of optimization approaches is provided.

Scoring methods
A wide variety of scoring and ranking methods have been applied in project selection and PPS. Another approach used in this process is using hybrid methods. In these methods, a combination of ranking methods and optimization approaches is used to find the best portfolio of projects. Table 13 presents some of the scoring based methods used in PPS.  Chu et al. (1996) DSS and Dynamic Programming Archer and Ghasemzadeh (1998) DSS and Framework Archer and Ghasemzadeh (1999) Decision making Framework Dong, Lai, and Wang (2005) Ghasemzadeh, Archer, and Iyogun (1999), Urli and Terrien (2010), Shou and Huang (2010), Zhu and Wang (2012), Yu, Wang, Wen, and Lai (2012), Nikkhahnasab and Najafi (2013) In this section, modeling approaches were reviewed. Various models have been developed to attend to PPS (e.g., Sefair et al., 2017;Schaeffer & Cruz-Reyes, 2016;Li, Wang, Yan, & Zhao, 2018). Such approaches limit the ability of top managers in the process of PPS. Moreover, there are some complex limitations that cannot be properly presented in the mathematical model or would form a model that is almost impossible to optimize. Various scoring studies have been also given in project management (e.g., Brauers & Zavadskas, 2010;Zavadskas, Turskis, Tamošaitiené, & Marina, 2008;Zavadskas, Vilutienė, Turskis, & Šaparauskas, 2014;Zavadaskas, Turskis, Vilutienė, & Lepkova, 2017b) and PPS (e.g., Debnath et al., 2017;Balderas et al., 2017). Such methods are based on judgments and are easily affected by opinions of experts. PPS is concerned with qualitative and quantitative data; therefore, it is often better to form frameworks that use both the scoring methods and the mathematical models (e.g. Tavana et al., 2015;Mohagheghi et al., 2016). A comparison of PPS studies with similar managerial problems suggests that in PPS expert systems have not been comprehensively applied. To put differently, forming proper expert systems could result in utilizing the expertise of managers and benefiting from various modeling and scoring methods.  Ghaeli et al. (2003), Koppinen and Rosqvist (2010), Conka et al. (2008) AHP Conka et al. (2008), Khalili-Damghani et al. (2013), Tavana et al. (2015) Data Envelopment Analysis (

Solution approaches
In this section, a review of solution approaches applied in mathematical modeling approaches of PPS is presented. Given the variety of PPS modeling approaches and applications, several approaches have been used to find the best project portfolio. The applied approaches are categorized into main groups of exact, inexact, and heuristics approaches.

Exact
Given the characteristics of PPS mathematical models, exact approaches were used in some of the studies to address small size problems. CPLEX has been used as an appropriate solver in some studies to obtain the exact solution. Bender's decomposition has been used to address the solution approach of some of the studies. Another common approach is obtaining linear and solvable equivalents through linearization methods. To better illustrate the exact approaches, Table 14 is presented. Robust augmented weighted Tchebycheff programs Hall et al. (2015), Sefair et al. (2017) Benders decomposition Roland et al. (2016) Cutting-plane approach Y. Liu and Y. K. Liu (2017) The equivalent analytical expressions of credibility constraints

Heuristic and meta-heuristic
PPS can be developed in the form of a multi-objective combinatorial optimization (MOCO) problem. In MOCO, obtaining the non-dominated or Pareto-optimal portfolio candidates forms an NP-hard problem. As a result, (meta) heuristic methods are needed to perform tradeoffs among solution quality and the effort required to obtain an acceptable approximation of the solution space (Doerner et al., 2006). Doerner et al. (2006) developed a mathematical model to handle PPS. They worked on a generalization of the classical bin packing problem that made the model NP-hard. Tofighian et al. (2018) developed a model that handled risks, stochastic incomes, and the possibility of investing extra budget in each time period. Their model was NP-hard and required a meta-heuristic solution approach. Panadero, Doering, Kizys, Juan, and Fito (2018) suggested that by increasing the pool of project proposals and consideration of realistic constraints, PPS becomes NP-hard. Therefore, they presented a variable neighborhood search semi-heuristic for PPS. Wang and Song (2016) presented a NP-hard model with consideration of reinvestment strategy for PPS and scheduling with time-dependent budget. Çağlar and Gürel (2017) addressed public R&D PPS problems with cancellations by introducing a NP-hard model. Shariatmadari, Nahavandi, Zegordi, and Sobhiyah (2017) proposed a PPS and scheduling model that was NP-hard. To conclude, in order to address NP-Hard PPS models, several heuristics and meta-heuristics approaches were applied and developed. Table 15 provides a review of different approaches applied in PPS methods.  Naderi (2013) The imperialist competitive algorithm Rabbani et al. (2013) Multi-objective differential evolution (MODE) Esfahani and Yousefi (2016) Harmony search algorithm Jingmei and Peng (2015) Improved quantum genetic algorithm Lifshits and Avdoshin (2016) SPEA II method Ghodoosi et al. (2016) Multi-objective shuffle frog leaping algorithm (SFLA)

Applications and real-life case studies
Given the characteristics of PPS, an important aspect of PPS studies is using them in real-life case studies. Therefore, in this section, a review of areas where PPS studies have been applied is presented. The application of PPS studies covers a wide range of fields. A review of studies shows that case studies in different areas such as nuclear energy, oil and gas industry, construction industry, research institutes, etc. were carried out. Table 16 shows case studies in PPS papers.  Ghaeli, Vavrik, and Nasvadi (2003) Intelligent Transportation Systems Olundh and Ritzen (2004) Structured development process of Scania Riddell and Wallace (2007) Determining funding levels for R&D projects for the particular example of the Nuclear Emergency Safety Team (NEST) Gutjahr et al. (2008), Stummer et al. (2009, , Gutjahr and Froeschl (2013) The Electronic Commerce Competence Center (EC3) Austria Energy and power generation Jeng and Huang (2015), Gang et al. (2015) Research institutes Jadda and Idrissi (2015) Case study of a Moroccan public organization Hummel et al. (2017) Robotic innovations for minimal invasive surgical interventions Martins et al. (2017) Sustainable strategic decision making in an electricity company End of Table 15 Conclusions and directions for further research As the review of PPS studies suggests, the number of studies on this subject has increased in recent years This is mainly due to recognition of the vitality of possessing well-established methods for practical project portfolio management. In this paper, the papers on PPS were reviewed based on the evaluation criteria, uncertainty modeling, selection approach, solution approach and area of application. To shed light on new insights for further research, this section presents a review of recent trends, literature gaps in addition to future research directions. Some of the recent trends in PPS studies are as follows: -The financial criteria are no more the only or the most important factors and other aspects of project portfolios are recently addressed in PPS studies. Considering the social and environmental aspects of project portfolios is one of the recent trends applied in PPS. -Some of the issues related to strategic management were recently added to PPS literature. Reinvestment strategy, flexible time horizon, and strategic alignment are some of the issues associated with strategic management. -PPS is a part of project portfolio management. To make this step more thorough, project implementation and management should be addressed in this process. One of the recent trends in the literature is the simultaneous consideration of project portfolio selection and project portfolio scheduling. -One of the recent trends is using downside risk measure to address project risk. In this approach, only the negative impacts and outcomes are considered. For instance, in common financial assessments, all the variations from the expected net present value are considered as a risk while in reality, only the negative deviations harm the portfolio. Therefore, in financial assessment using lower semivariance is more efficient than finding all variations as a risk. -To address uncertainty, fuzzy set theory, stochastic and grey uncertainty were used.
One new trend in recent years is using new fuzzy extensions in order to improve modeling uncertainty. Type 2 fuzzy sets, intuitionistic fuzzy sets, and interval-valued fuzzy sets are some of the methods applied. -Given the features of PPS studies, a new trend is presenting frameworks that consist of both MADM and MODM techniques. For instance, one approach was using DEA, TOPSIS, and LP in PPS. Such approaches improve the flexibility of the process and provide more power in addressing various sorts of data. -Since in recent years more optimization approaches have been introduced, one new trend is using Meta-heuristics and heuristics to find the solution of mathematical models. -Using uncertainty theory presented by Liu (2007) is a new approach in addressing uncertainty in the project environment that will improve the existing methods. Some of the main gaps that were recognized in this study are as follows: -Despite addressing sustainability in PPS studies, sustainability criteria are not still comprehensively addressed. In other words, the criteria are not yet tailored for the project management environment.
-Projects suffer from unexpected events and unpleasant surprises. In other words, project success and achieving the goals of projects depend on the ability of projects to withstand harmful events. Therefore, one criterion that would improve the evaluation of projects and project portfolios is project resilience that is not yet addressed in the literature. -Using hybrid methods in addressing uncertainty would improve the existing methods.
In other words, applying methods such as fuzzy stochastic approaches would improve the existing methods in addressing uncertainty. -Several DSSs were introduced in the literature to help decision-making in the process of PPS. However, yet the literature is very weak when it comes to developing expert system. To put it differently, since projects are unique and experts' opinions in some cases are priceless, using expert systems would highly improve the entire process in PPS. -Combining the existing uncertain optimization methods with modeling approaches in PPS is not yet well addressed in the literature. In other words, interval optimization, stochastic optimization, and fuzzy optimization techniques are still new to the literature. -The literature is weak when it comes to using exact solution approaches. Methods such as Lagrangian relaxation, benders decomposition, and branching techniques are still new to the literature. -In MADM based approaches, issues like decision-makers' weights, criteria's subjective and objective weights, aggregation steps, and decision indexes are not yet fully addressed in the literature. However, to overcome the gaps of PSP, in the following some of the possible future trends are presented: -Addressing sustainability through project portfolio selection by using multi-objective optimization methods is a future trend that helps in addressing sustainability through optimization techniques. -Project portfolio evaluation, project portfolio implementation, and management should all be addressed together to achieve more efficiency in the methods. In this approach, methods such as reinvestment strategy and project scheduling will be addressed in the PPS method. -Given the importance of globalization in today's decision-making problems, an important and practical future research direction is addressing an international issue such as foreign investment and exchange rates in project portfolio evaluations. -Using the advantages of earned value analysis in PPS could improve the process.
Therefore, using project progress evaluation techniques in PPS is a future trend that could enhance the process. -Using fuzzy optimization techniques to address uncertain multi-objective optimization approaches in PPS is a future trend that has the merits of MODM and fuzzy optimization. -Using fuzzy stochastic approaches to address project uncertainty is a possible future trend that would provide the methods with the merits of fuzzy and stochastic methods.
-Another interesting future trend is developing proper expert systems to employ the expertise of managers and benefit from various modeling and scoring methods. There are various expert systems that can be applied in PPS studies. A very interesting possible future research direction is using different sorts of expert systems (i.e. rule-based expert system, frame-based expert system, fuzzy expert system, neural expert system, and neuro-fuzzy expert system) and explores the pros and cons of different expert systems under different PPS conditions.