MOONA SOFTWARE FOR SURVEY CLASSIFICATION AND EVALUATION OF CRITERIA TO SUPPORT DECISION-MAKING FOR PROPERTIES PORTFOLIO

. The MOORA for Neural Networks Analysis (MONNA) software was created to classify variables and evaluate the degree of correlation between them, helping to choose a property portfolio and facilitating decision making involving multiple criteria. The MONNA software presents the classification of the alternatives calculated automatically by the MOORA (Multi-Objective Optimization on the Basis of Ratio Analysis) and provides a Global Average Rate (GAR). Artificial Neural Networks (ANNs) analysis provides the degree of correlation between variables and uses GAR as the output parameter. The degree of correlation between the variables allows us to assess whether these variables are dependent on each other and can capture customer preferences. For the application we used a survey that sought to know the preferences of customers, which will serve to make the decision of which properties should be part of the company’s portfolio. The contribution and originality of the MONNA software is that through the integration of the MOORA and ANN methods, the classification and criterion evaluation calculations are faster and standardized. The use of software by decision makers helps to have more accurately find and classify available options, preventing simulations from being done by iterative processes and providing validated numerical data for management evaluation.


Introduction
Most day-to-day problems include alternatives and various criteria and it is up to the manager to decide on the most appropriate alternative (Safarzadeh et al., 2018). The availability of information today is one of those responsible for improving organizational performance (Pamučar et al., 2018). Finding the optimal solution for a given situation is the goal of a survey that should represent the decision problem as closely as possible, transforming information into actions and solutions (Lindblom & Tikkanen, 2010). The availability of data for research is what makes it possible to directly investigate any subject or problem, but it is important to understand the expectations of researchers or interviewers (Frydmann & Stillwagon, 2018). According to the type of approach, questioning, and in-terpretation of the research, the results may be considered unreliable because the surveys are not representative to be useful, or because the respondents misinterpret the issues or do not take into account all of the factors involved (Cochrane, 2011). Thus, it is necessary to search for different and innovative solutions (Dimitrova, 2018). Greenwood and Shleifer (2014) stated that no data from a survey can be discarded, and for this they presented a stock market survey that related the data obtained on investors' expectations to the decisions of market participants in determining where to invest capital. In the property market, in general, there is a gap between customer service and customer preferences. This orients decisions to the consumer, aiming at differentiated service strategies that can solve this gap (Chiang & Perng, 2018). The problem is that the researcher does not always have data so that he can make a comparison to assess whether the survey will help him to make a decision. Prior to a formal survey, Wang et al. (2018) conducted a pretest by submitting a preliminary questionnaire to 10 workers to identify possible confusing items. Based on the worker feedback, the researchers simplified and clarified some items, and then they performed a definitive search. However, this approach made it slow to change and react to the different types of events and circumstances that may occur and which may be important to the survival of a company, for example (Elmaraghy & Wiendahl, 2009). When carrying out a pretest with a small group of people, there is still a risk that the final survey will not be fully understood by another group of workers, for example, because sometimes the sector is disorganized (Hazarika & Goswami, 2018) and has internal barriers to the adoption of new management practices (Gonçalves et al., 2019).
For organizational management, MCDM (Multi-Criteria Decision Making) is a popular methodology for decision-making that is capable of ranking alternatives (scenarios) for a decision concerning several criteria (objectives) by taking preferences into account (Mehdy Hashemy Shahdany & Roozbahani, 2015). Each survey generates a matrix composed of alternatives and criteria (objectives) and, for the analysis of these data and decision-making, the literature suggests many methods. Challenges of contemporary development of the world force stakeholders to look, design and implement well-balanced products for both international and local perspectives (Hashemkhani Zolfani et al., 2013). Moreover, each implemented project changes the environment of projects in which other projects will be implemented (Zavadskas et al., 2009). In this paper we select the MOORA method (Brauers & Zavadskas, 2006) because it is one of the most current MCDM methods and deals with the weaknesses of other methods. Its results are naturally stable with low processing times (Akkaya et al., 2015). Although the method is simple, depending on the number of alternatives and criteria, the number of calculations to be performed can be large and manually laborious (Siahaan, 2018;Alinezhad & Khalili, 2019).
The selection or ranking of alternatives obtained by a survey and submitted to a multicriteria evaluation is not an easy problem to solve because the evaluation indicates a direction but does not indicate if there is an ideal solution, because better quality implies higher price, making the criteria conflict with each other (Zavadskas & Turskis, 2011). To find an ideal solution, researchers have proposed many methods beyond classical logic, such as fuzzy set theory, approximate set theory, word computation and granular computation, and computational theory for dynamic linguistic systems. However, in all uncertainty is a typical feature of preferences when it is necessary to define calculations to deal with these situations (Zavadskas & Turskis, 2011). Tamošaitienė et al. (2013) proposed the evaluation and choice of alternatives through the combination of several methods, but do not present software that would facilitate the application by anyone.
The human being is able to evaluate criteria but is unable to conduct numerous iterations between the criteria in a fast, constant, and standardized way. This is possible through neural networks (ANNs) because they are computational structures designed to imitate the accumulation of knowledge in the biological central nervous system, and the application of ANNs solves the problems of prediction, approximation, functions, classification, and pattern recognition (Osmanbegović & Suljić, 2012). A survey may face uncertainties as several scenarios are identified with various states of nature, thus generating uncertainty about their results. Because they are supervised machinelearning algorithms, ANNs create a set of connections between units called neurons that are grouped in layers (Peral et al., 2017). In a survey, these units are the criteria for each alternative, and the layers are subcriteria.
The literature brings several methods of analysis recent criteria, such as ARAS , Co-CoSo (Yazdani et al., 2018(Yazdani et al., , 2019, multiple criteria decisionmaking hybrid model to assess the shape of a staircase. for homes (Turskis & Juodagalvienė, 2016), TOPSIS method for solving complicated decision-making problems (Zavadskas et al., 2016) and MAUT and Neural Networks for key performance indicators prioritization . We use the MOORA method also because it is already a consolidated method and it gives us a reliable and safe return, which is extremely necessary in the early phase of MONNA, as it will provide the output data for the ANN.
In practice, we have no software that promotes the MOORA method calculations and that simulate the results by ANNs. For this reason, it is useful to have a computational tool to perform the calculations of the MOORA method, which automatically ranks and presents the alternatives and also tests the results of the ANN search. This tool provides data that helps you evaluate whether research is coherently structured and can be used primarily by anyone to test any type of research.
Recently, authors are looking for the integration of methods already consecrated in the literature to arrive at more practical solutions for the day to day. We are looking for some examples such as the integration of AHP (Analytic Hierarchy Process) with MABAC (Multi-Attributive Border Approximation Area Comparison) method for evaluation of university web sites (Pamučar et al., 2018), a new model of FMEA (Failure Mode and Effect Analysis) based on multi-criteria decision-making for risk assessment (Lo & Liou, 2018), extension of the FC-MOPSO (Fuzzy Clustering Multi-Objective Particle Swarm Optimizer) algorithm employed to evaluate optimization problems (Mokarram & Banan, 2018) and a model for reducing attributes in incomplete information systems (Qian & Shu, 2018). What these models have in common is that they need quality input data, which is usually collected by a survey. For this there are statistical methods, and in recent decades several attempts have been made to provide new structures to unite techniques in an integrated methodology (Kazancoglu & Ozturkoglu, 2018).
In this way, this paper aims to present the MONNA software, that was created to assist the classification of the alternatives calculated automatically by the MOORA (Multi-Objective Optimization on the Basis of Ratio Analysis) and provides a Global Average Rate (GAR) that will be used as a output parameter by Artificial Neural Network (ANN) analysis to provide the degree of correlation between the variables. The use of the MONNA method is examined through the simulation of a survey applied by property sellers to potential buyers.
In the next section, we present a description of all methods that comprise the software. In section 2, we present a detailed description of the software operation, and in section 3 we show how the software analyzes the survey. In the last section, we discuss the results and propose some future work.

MOORA
To classify the alternatives, we used the MOORA method first proposed by Brauers and Zavadskas (2006). Comparisons with other MCDM methods have revealed that MOORA is more powerful than other traditional methods in terms of computational time, simplicity, mathematical calculations, and stability and type of information (Kazancoglu & Ozturkoglu, 2018). Arabsheybani et al. (2018) used Fuzzy-MOORA integrated with the Failure Modes, Effects Analysis (FMEA) method to determine a sustainable selection of suppliers. Kazancoglu and Ozturkoglu (2018) also used an MCDM mix to devise an integrated disassembly line balancing structure with green targets. This was also used in areas such as stay projects (Brauers et al., 2008), evaluation of the best living conditions (Kalibatas & Turskis, 2008), project management decisions in an economy in transition , classification of heating losses in buildings (Kracka et al., 2010), ranking of the efficiency of different agricultural operations (Baležentis, 2011), selection of the best intelligent manufacturing system (Mandal & Sarkar, 2012), selection of strategies in the selection of wireless networks (Archana & Sujatha, 2012), and machine selection (Vatansever & Kazançoğlu, 2014).
To calculate the GAR using the MOORA method, we first need to assemble a decision matrix, which must contain all data obtained from the survey and must follow the model of equation 1.
In the matrix, x ij is the measure of performance of the i th alternative of the i th criterion, m is the alternative number, and n is the number of criteria. Then, a relationship system is developed wherein each presentation of an alternative in a criterion is compared to a denominator that represents all alternatives relating to that criterion.
The results of calculations highly depend on the normalization technique (Zavadskas et al., 2007).  proposed a normalization procedure, and it is used when a decision matrix has negative numbers or very large values for a criterion. Thus, the denominator is given by the square root of the sum of the squares of each criteria. This relation can be expressed by equation 2.
where: x ij is a dimensionless number that belongs to the interval [0,1] that represents the normalized performance of the i th alternative in the j th criterion. For multiobjective optimization, these normalized performances are added in case of maximization (beneficial to attributes) and subtracted in case of minimization (for non-beneficial attributes). Then, the optimization problem is expressed in equation 3.
where: g is the number of attributes to be maximized; (n -g) is the number of criteria to be minimized, and y i is the valuation value of the alternative with respect to all of the criteria. This version of MONNA software work only with maximization criteria, but in future the use of MOORA equations allows the software to be upgraded. The weighting will be given according to the situation being analyzed by the decision maker. To define these weights, the decision maker may use methods such as the Hybrid Fuzzy Group Multi-Criteria Assessment (Turskis et al., 2019b), Fuzzy Group Decision-making (Turskis et al., 2019a), Delphi (Delbecq & Van de Ven, 1971), LINMAP (Srinivasan & Shocker, 1973), SWARA Method (Keršulienė et al., 2010) and in exceptional cases, they use values determined by experts In this article, we use the same weight for the criteria because the goal is to show the functionality of MONNA. The MONNA uses the normalized performances of each alternative provided by MOORA method. This normalized performance of each alternative will be used to provide input and some output data, to simulate more quickly and so given more confinable results by ANN. Without providing input and output data, we would need to simulate the ANN by iterative processes. This ANN iterative process of simulations can be considered difficult to do because of the complexity and lack of time in the day to day life of managers and decision makers.

Artificial neural network
The objective of using the Multilayer Perceptron (MLP) Artificial Neural Network (ANN) is to test the structure of the alternatives and criteria used in the survey. ANNs can solve problems through continuous data processing, which is impossible in complex structures. Recently, the use of ANNs has grown in business environments (Pagell & Shevchenko, 2014;Dutta et al., 2017). Dutta et al. (2017) used ANNs as supervised machine learning algorithms, which create a set of connections between neurons grouped in layers (Osmanbegović & Suljić, 2012).
ANNs have advantages over other decision-making models, particularly in cases with complex nonlinear data (Li et al., 1997). This advantage exists because the ANNs have learning capacity (Da Costa et al., 2019). MLP ANNs work with more than one occult layer. Thus, they are the basis for other ANNs, as demonstrated in other studies (Egmont-Petersen et al., 1998;Park & Jo, 2016;Tang et al., 2015). This is a machine learning environment that provides practical knowledge (Roiger, 2017).
The input data for the ANN (X N ) are the answers for each criterion of the survey, and the network output attribute is the GAR initially calculated by the MOORA method. When using ANNs, it is necessary to use one set of training data and another set of data for testing. MONNA uses a cross-validation mode, which simulates the predictions of new objects by repeatedly dividing the original training data set into training and validation objects (Xu et al., 2018). This is used when a validation set is not available, or when the data set is too small to be divided into a training and a validation set.

Pearson's correlation
MONNA presents the "Criterion Correlation. " This corresponds to the product-moment of Pearson's correlation coefficient, which is the linear measure of the dependence between two different criteria, where 0.9 indicates a very strong correlation (Ly et al., 2018). Wu et al. (2018) said that above 0.7, Pearson's correlation can already be considered satisfactory. In the MONNA method, we can simulate several types of survey results until we obtain a correlation coefficient above 0.7 between the variables. Equation 4 corresponds to the calculation of Pearson's correlation between two criteria X and Y that correspond, respectively, to the results of a survey.
The variables need not be dependent on each other, but the correlation coefficient close to 1 indicates that while one variable increases, another increases as well. Correlation coefficient much less than 1 or negative indicates that while one variable improves another may be getting worse. MONNA seeks to show that the closer to 1, there is evidence that indicator modeling is good and does not need to replace indicators or criteria. In the next section, we present a computational implementation using data from a fictional survey to clarify this.

About MONNA development
The MONNA software was developed by the authors using the Java programming language. In developing MONNA, we used the free and open-source software called Netbeans version 8.2 (NetBeans, 2018) which is an integrated development environment (IDE) for programming in Java and other languages. MONNA uses some open-source libraries such as the Weka 3.8 library (WEKA, 2018) which is a collection of machine learning algorithms for data mining tasks. The Weka software is free and open source under the General Public License version 3 (GNUv3, 2018). It was developed by a team of teachers and academics from the machine learning group at the University of Waikato in New Zealand.
Another library used by the MONNA software is OpenCSV (OPENCSV, 2018), which is a free open-source library under Apache license version 2 (Apache, 2018) and is used to manipulate Comma Separated Value (CSV) files in Java programs. The MONNA software is available for download at https://www.unisc.br/pt/cursos/todos-oscursos/mestrado-doutorado/mestrado/mestrado-em-systems-and-industrialprocesses/MONNA. In order to present MONNA, Figure 1 shows the 15 steps of the software. Next, we will illustrate its use by using a fictional survey.
In the following sections, we explain each step in detail.

Stages of obtaining information
Steps 1, 2, and 3 concern the collection of information on alternatives and criteria. One of them is chosen as the way the data will be obtained to be informed in the MONNA software. Form 1 simply lists a set of alternatives, criteria, and their values that will be entered manually in the decision matrix within the MONNA. For form 2, the MONNA allows data to be pasted from an external source such as an Excel or Google Sheets worksheet to be pasted directly into the decision matrix. In form 3, the data can be imported from a CSV file or from a data file previously saved by the MONNA software itself.

Starting with the MONNA software, and other steps
Once you have chosen the data source, you can then launch the MONNA software. The GUI of the software when it is opened presents a screen as shown in Figure 2. The red letters marked in the figure are used to reference each screen item that will be cited in the descriptions of the software execution steps.
Step 4 of Figure 1 occurs only when the user wants to fill the data manually (form 1 of Figure 1). In this step, fields A and B, shown in Figure 2, should be reported as the number of alternatives and the number of criteria, respectively. After completing the fields, press the "New Model" button (C) to create the following tables: criteria weight matrix (H); decision matrix, where the alternatives and their respective values will be informed for each criterion (I); and the table where the results of evaluating the alternatives and the individual performance rates (L) are shown. After pressing the C button, the tables will be generated according to Figure 3. In Figure 3, we create a table composed of six alternatives and four criteria, which will be exemplified in the following sections.
Step 5 refers to filling the decision matrix (I), where if form 1 is used to obtain the data, the values of the alternatives with respect to the criteria can be entered manually in Table I of Figure 3. The names of the alternatives may be included in the appropriate column marked with the letter S in Figure 3.
If you choose form 2 to fill in the decision matrix, by copying the data from a worksheet, you can simply paste it into the decision matrix. To paste, you first need to click on Table I in Figure 3 and then use the <Ctrl + V> keys, or choose the "Edit" menu and then the "Paste" item.
If the user using data collection from step 3, they can import a CSV file. To do this, simply press the "Import from CSV File" button identified by the letter D in Fig you can then proceed to the next steps. MONNA allows the association of names and weights with the criteria in table H, as foreseen in step 6. To edit the name of a given criterion, proceed with a double click of the mouse on the heading of the column of the criteria, represented by the letter R in Figure 3, and then edit the name of the criteria. When table H is generated by the C button, the weight values of each criteria are automatically filled with 1 by default. The maximize setting for all criteria is also performed by default. The values of the weights can be edited, but the settings of maximization are fixed in MONNA for all of the criteria, and it is not possible to change them in this version of the software.
Step 7 refers to the definition of a maximum value to present the individual rate of each alternative within a scale. The individual rate of each alternative (r i ) is obtained by multiplying this value defined by the user by the normalized ordering value of each alternative according to equation 7. To define the maximum value of the scale, fill in the field identified by letter J in Figure 3.
In step 8, calculations based on the MOORA method are applied to find the order of classification of the alternatives and the individual rates of performance of each within the scale defined in step 7. To perform the calculations, press the "Evaluate the Alternatives" button represented by the letter K in Figure 3. The results are then shown in the evaluation table of the alternatives, identified by the letter L in the same figure.
After calculations and results are presented in this table, the values are automatically sorted by the individual rate scaled in the column under the heading "Individual Rates". The user can choose the order by clicking under the column headings in this table. The "Id" column contains the numbers that identify the original ordering of the alternatives in the decision matrix. Another result obtained by calculating the evaluation of the alternatives is the Global Average Rate shown in the field identified by the letter M.
In step 8, several calculations are performed. The first calculation is the normalization of the decision matrix following equation 2. For the MONNA method, this type of normalization is applied to the decision matrix.
The second calculation is performed to find the classification values of the alternatives (y i ) based on the MOORA method, following equation 3 described in item 2 of this article. The third calculation finds the individual rate (r i ), where normalization is first applied to the classification values of the alternatives according to equation 5, and then the normalized values are scaled by multiplying them by the maximum value defined in the scale (E max ). In addition, the overall average rate (GAR) is the simple average of the individual rates. The calculation of r i is demonstrated in equation 6, and the calculation of R occurs in equation 7.
The next major step of the MONNA method is to perform an evaluation of the survey by evaluating the criterion correlation. This value is obtained by submitting the decision matrix data and their respective individual rates as a classification attribute to the ANN Multilayer Perceptron (MLP). The implementation of the MLP algorithm is done through the data-mining algorithm library of the Weka software. ARFF (Attribute-Relation File Format) is the standard file format used by the Weka software.
To perform the ANN model evaluation, it is necessary to go through step 9, which consists of defining the number of data sets for cross-validation during the ANN learning process. This value must be reported in the field identified by the letter N in Figure 3. Generally, 10 sets are used, but if the number of alternatives is less than 10, this number must be less than or equal to the number of alternatives up to a minimum value of 2.
It is also necessary to proceed with step 10, where the hidden layers and the number of nodes in each are defined to be formed in the ANN. This is defined in the field identified by the letter O in Figure 3. The MONNA software has tips on each GUI item that appear when you hover the mouse over an item. In the O field to inform the hidden layers of the ANNs, that can be according the numbers of criteria and sub-criteria or according with the automatic MONNA default.
To define the hidden layers, in the O field, you must enter the number of nodes per layer in the format h 1 , h 2 , h 3 , ..., h n , where n is the number of hidden layers of the neural network, and h is the number of nodes in each layer. For example, if field 0 is filled with "2, 5, 4, " this means that three layers have been defined with the first layer consisting of two nodes, the second layer by five, and the third layer by four nodes. Wildcard characters can be used: "a" = (number of criteria + 1) / 2, "I" = number of criteria, and "T" = number of criteria + 1. The MLP hidden-layer parameter is used by the MLP algorithm to form the ANN.
With the parameters reported in steps 9 and 10, the correlation coefficient is then calculated in step 11. To do this, simply press the "Calculate Criteria Correlation" button identified by the letter P in Figure 3. When executed, the result is shown in the specific field identified by the letter Q in the same figure.
It is important to note that in this step, an Attribute-Relation File Format (ARFF) data file is generated in a temporary system folder.
Next, in step 12, the user performs a criterion correlation analysis. If the result is not satisfactory in the eyes of the user, i.e., significantly away from 1, it can then make adjustments to the ANN (or data) parameters and iteratively return to steps 9, 10, 11, and 12 until it reaches an acceptable value.
In step 13, you can save the model for future use. This can be done by pressing the "Save Data/Model" button identified by the letter E in Figure 3. The user can also restart the work from step 1. This it is necessary to perform step 14, which is a reset of the screen, returning to the initial state of the software with the default values in the fields and empty tables. To perform this screen reset, press the "Reset All Data/Model" button identified by the letter F in Figure 3. From the screen reset, the user can choose to start a new model or exit the program (step 15) by pressing the "Exit" button represented by the letter G in Figure 3.

Initial analysis
In this work, to examine the software functionalities, we will simulate a survey that initially has six alternatives and four criteria. So, the customers gave the level of importance on a Likert scale (Likert, 1932) from 1 to 5 (1 = not important, 2 = not important, 3 = important, 4 = very important, 5 = extremely important) to each criterion researched. If the client is of importance 5 for Near Job, that means he has a preference for real estate as close as possible to his job. After capturing the importance given by clients to each criterion, we simulate the data obtained by ANN to obtain the correlation coefficient. The correlation coefficient aims to find inconsistencies and indications that criteria need to be included or excluded, so that the simulation with real data portrays reality as best as possible. The level of importance survey serves to capture customer preferences, while simulation with real data (price in dollars and distance in miles, for example) can also be simulated by the company itself to select the ideal property for the customer.
The four criteria are Price, Maintenance Cost, Comfort, and Location. There are four criteria that a real estate seller wants to know about his or her client's importance, ranking the alternatives (clients) from the most demanding to the least demanding. Through the GAR, we know the average requirement of customers that will serve to make the decision of which properties should be part of the portfolio of the company to be presented to these customers.
The initial data that make up the matrix are shown in Table 1 below. Recall that this survey is a simulation, and the degree of importance for each criterion was randomly estimated so that we could simulate the functionalities of the MONNA software.
This data was copied to MONNA, giving rise to a new file, as shown in Figure 4.
After entering the survey data, we clicked on the "Evaluate the Alternatives" and "Calculate Criteria Correlation" buttons, and MONNA presented the following results: 1 -Ranked the alternatives of the most demanding to the least demanding customer according to their answers in the survey. The most demanding customer was customer F, and the least demanding was customer C.
2 -Provided the researcher with a GAR of 4.82. This is an average rate according to the survey responses and serves as a parameter for the researcher to know how many respondents are above this rate and how many are below. GAR also serves as a basis for decision-making.  3 -Last, provided a criterion correlation of 0.53. This coefficient is used to evaluate if there is a connection between the criteria of the survey and whether it was elaborated correctly.

Final analysis
A result of 0.53 for the criterion correlation indicated that the survey can be improved. The researcher asked only one question to the clients for each criterion analyzed. This may have caused the correlation coefficient to be low since the criteria do not have many connections with each other. According to Brauers (2002) and Brauers (2006), the assignment of subgoals or subcriteria represents a solution because within the criterion cost we can have subcriteria such as purchase value and devaluation. Following this reasoning, for each criterion presented in Table 1, we added two subcriteria, which gave rise to Table 2. Logically, since subcriteria were added, the survey had to be restructured and answered again, but since the survey is fictitious, the responses were also simulated.
This data was copied again to MONNA, giving rise to a new file, as shown in Figure 5.
Using the subcriteria, MONNA created an array of six alternatives and eight criteria, and presented the following results: 1 -Ranked again the alternatives of the most demanding customer to the least demanding according to their answers in the new survey. The most demanding customer was customer A, and the least demanding customer was customer C.
2 -Provided the researcher with a new Global Average Rate, now 4.87. This rate increased with the new survey.
3 -Finally, provided a criterion correlation of 0.74. This shows us that even when dealing with survey responses, we can analyze if the results are satisfactory or not.

General conclusions
A property seller needs to make decisions consistently in order to buy and negotiate real estate that is in line with what prospective clients require. However, capturing this information is not always easy and usually relies only on the empirical knowledge of the seller. To make good decisions about which properties the client is looking for, a search  Figure 5. Representation of data divided into subcriteria with potential buyers proves to be very useful. However, it is necessary to structure the search in a way that helps the seller. For this reason, we created MONNA. Two simulations of a survey were conducted to exemplify MONNA. In the first simulation, we simulated a survey through a 6 × 4 matrix of responses. In the second simulation, we included subcriteria and simulated a survey using a 6 × 8 matrix.
In the first simulation, the most demanding customer was customer F, and in the second it was client A. The least demanding customer was C in the two simulations. This change occurred because in the new survey, new criteria were included. With these criteria, the researcher can explore the preferences of the clients.
GAR increased from 4.82 to 4.87 in the second simulation. That is, the level of customer demand increased. As more criteria were explored, the rate varied for both. In this case, since the rate increased, the researcher needs to worry even more about the level of customer demand.
Finally, the most interesting and relevant data point from MONNA was the correlation between the criteria. As we say, to be considered a good correlation, this rate should be between 0.7 and 0.9, and is optimal above 0.9. In the first simulation it was 0.52, indicating a low correlation between the criteria. Actually, Price and Cost with Maintenance do not seem to have much of a relationship. Thus, we restructured the survey and created two subcriteria for each criterion so that it would be easier to capture customer requirements and preferences. We reapplied the survey, and the criterion correlation jumped to 0.74, already within the acceptable level (between 0.7 and 0.9). We could create more subcriteria to further increase the correlation and make the results even more reliable, but the decision to stop will always be up to the researcher.
As already mentioned in section 3.1, in addition to the level of importance given by customers to each of the criteria, the company can use these same criteria and the same software to analyze which property is ideal for each type of customer. So, the software can have many uses, because it is based on a multicriteria method and ANN can be used to solve and classify any decision problem.
After evaluating the surveys, we conclude that MON-NA satisfactorily evaluates the alternatives by referring to different criteria. This is because when showing Pearson's coefficient of correlation, MONNA can present the level of the relationship between the variables that make up the survey. In this way, the researcher or decision-maker can structure the set of criteria and/or subcriteria in a coherent way, and is therefore able to reach more assertive results in a naturally subjective environment.
In the future, we intend to automate steps 9, 10, 11, and 12 so that the MONNA software finds the value of the correlation, previously established by the user, without the need for interference or new data insertion. Also as a future work suggestion is the use complete equations of the MOORA method, with the maximization and the minimization part. With the maximization and minimization the decision maker can add more variables and criteria to analyze and sort by the MONNA Software.