The Future of Evaluation of Lower Secondary Schools’ Management

Efficiency of educational institutions is a significant issue worldwide. However, there is no commonly accepted way to measure the quality of management of education services. Mainly because it too much depends on the national factors: political, economic or legal. Moreover, it has been proved that the efficiency of the school’s teaching is affected directly by the environmental factor represented by selected characteristics of students. Therefore, taking into account value-added students’ knowledge rather than absolute exam results changes evaluation of the schools’ efficiency. The article contribute in international discussions about future of evaluation the quality of management in educational sector. In the article the model of evaluation Polish lower secondary schools’ management taking into account local and environmental context is proposed. On the basis of Bialystok’s schools, author shows that the implementation of DEA could be useful and provides additional knowledge about the efficiency of management in educational institutions.


Introduction
Education is the foundation of civil society and a key condition for economic development. On the other hand, limited financial resources make the appropriate funding allocation extremely crucial. Efficiency evaluation of education institutions management is important at all levels: from nursery to university. However, there is no commonly accepted way to measure the quality of management of education services. Mainly because it too much depends on the national factors: political, economic or legal.
In Poland, there are no strong stimulators promoting high quality educational services. The basic education sector in majority belongs to the public finance sector. The competition on the market of educational services is only emerging. Rankings of schools Business, ManageMent and education ISSN 2029-7491 / eISSN 2029-6169 2015, 13(1): 112-125 doi:10.3846/bme.2015 of different levels are the evidence of that process. However, ranking of educational institutions in Poland are usually constructed on the basis of arbitrarily selected indicators and their weights. The most common measure of the school performance is its students' test results.
In 2005 a group of experts started developing a methodological and statistical background for the Polish version of Educational Value Added (EVA). EVD, while assessing the school on the basis of students' exam results, take into account the school's students prior scores on compulsory exam (Dolata et al. 2013). However, full assessment should, analyze all the activities of the school and consider other aspects (Chodakowska 2014).
The article presents the use of data envelopment analysis (DEA) in the process of assessing the lower secondary schools' efficiency in Poland. The model of evaluation the Polish lower secondary schools' management taking into account local and environmental context is proposed. An example of evaluation of schools management using DEA is done on the basis of schools in Bialystok.
The article consists of seven parts. After introduction, the concept of DEA methodology and mathematical formulations of DEA models are presented in brief. The DEA applications in the area of education that were carried out in the world are reviewed in the third part. The next part focuses on education system in Poland which introduction is necessary to understand the selection of assessment criteria. The proposed methodology and the process of efficiency evaluation of lower secondary schools in Bialystok are presented in the fifth part. Then the results of efficiency evaluation of Bialystok's using DEA method are presented. The article ends with conclusions that contain recommendation for application the DEA methodology to evaluate the performance of schools in Poland.

Data Envelopment Data Envelopment Analysis
Data Envelopment Analysis (DEA), developed by (Charnes et al. 1978), is a well-established method for evaluating the relative efficiency of a set of comparable entitiesdecision making units (DMUs). DEA method is derived from the Farrell's concept of efficiency, who proposed the relative efficiency measurement, indicating the ratio of inputs to outputs in relation to the maximum value achieved in the technological conditions (Farrell 1957).
Fully efficient units create an efficiency frontier. The possibility of improving the efficiency of inefficient units is determined by reference their results to the efficient frontier ( Fig. 1).
Determining the efficieny using DEA method is to find the optimal technology by solving the adequate linear programming task (Cooper et al. 2007). The efficiency ratio is obtained by comparing the optimal, virtual and empirical technology.
The basic input-oriented radial DEA model for measuring performance of (DMU j 0 ) can be written as (Cooper et al. 2007 λ j − intensity levels at which the production activities are conducted by the j-th DMUs; r = 1, 2, …, s -number of outputs; i = 1, 2, …, m -number of inputs; j = 1, 2, …, n -number of DMUs; 0 j θ -efficiency ratio taking values in the range <0.1>; 1 for fully effective entities.
The larger 0 j θ is, the better efficiency DMUj0 has.
Since DEA allows multi input and output variables, it can take into account many different areas of the school performance. In addition, output and input variables can be stated in different units of measurement. Because DEA is a non-parametric method it places no restriction on functional form of the production relationships between the outputs and inputs. Methodological extensions can adjust the DEA measure for exogenous environmental variables. Fig. 1. Production Frontier -a case of single output and single input (source: Coelli et al. 2002;Cooper et al. 2007)

Previous research
Data Envelopment Analysis (DEA) is more and more popular and widely used method for determining the effectiveness of both commercial and non-profit organizations. Vast and constantly growing number of publications on the subject is a proof. Education is one of the top five areas of application of DEA -over five percent of application embedded papers addresses education sector. Efficiency study in education focus mainly on higher education sector (Liu et al. 2013). The most recent work concerning universities are for example: Duh et al. (2012), Bayraktar et al. (2013), De Witte et al. (2013, Nazarko, Šaparauskas (2014). However, DEA method can be successfully used also to construct a synthetic indicator to measure primry and lower secondary school performance.
One of the first studies in this sector was done by (Bessent et al. 1982). This analysis was applied to 167 elementary schools in the Houston Independent School District. Due to this work DEA provides the management information. In developing an operating plan for the each school, the managers of a school can increase output goals if achievement is below the norm and will be able to request additional input resources.
The Data Envelopment Analysis proved to be an appropriate method to analyse efficiency of educational institutions. The results of the analysis show that some of the school and teacher characteristics significantly affect school productivity. The schools with similar characteristics and inputs display quite different results, so studying the reasons for these differences will help to design more effective educational policies. The researches indicate that the choice of inputs and outputs plays a key role in assessing the effectiveness of the education units by the DEA. Furthermore, the results highlight the importance of including environmental variables in both technical and allocative efficiency.

Education system in Poland
Compulsory education in Poland starts at the age of five or six, from the one-year preschool preparation, then from six or seven years of age, from the 1st grade of primary school. After 6 years of primary education, students take an external compulsory competence test and join the general lower secondary school for 3 years and at the end, take another compulsory exam. To ensure that every graduate of primary school finds a places in lower secondary schools, public lower secondary school must first take all the children from its catchment area, regardless of the student's competence test results.
Primary and lower secondary schools students graduate regardless of the score. Graduates from lower secondary school can continue their education in the different types of schools. Depending on scores on the exam they can choose: 3-year general upper secondary school, 3-year specialised upper secondary school, 4-year technical upper secondary school or 3-year basic vocational school.
In addition to the compulsory education until the age of 18 and the lack of possibilities for the public primary and lower secondary schools to choose students, what determines and limits the performance of schools the most are the responsibilities distributions and funding. The responsibility for the administration of public primary schools as well as lower secondary schools has been delegated to local governments. The school education part of the general subvention from the state budget is the main source of funding for the school education sector in Poland. However, subvention transferred from the state budget is insufficient to cover all the needs of schools and local governments have to finance education with additional sources. The largest part in the total expenditure on education are teachers' salaries with its derivatives. Salary standards in public schools is closely linked only to promotion system. Current government financing policy does not include the estimation of relative effectiveness of school performance and do not affect the selection of teaching staff in schools.
At the same time local governments responsible for the maintenance and financial management in primary and lower secondary schools are not responsible for quality management and pedagogical supervision over schools. It is the responsibility of regional superintendent authorities under the Ministry of National Education.
According to the author, the existing legal solutions and restrictions in no way should be a premise to abandon the comprehensive analysis and evaluation of the effectiveness of individual schools.

The proposed methodology of evaluation of lower secondary schools' management
The depth study of the literature on efficiency and DEA method and critical analysis of its applications allows to determine the stages of work necessary to assessment of the effectiveness of lower secondary schools. These included: analysis of the environmental context of secondary schools, the choice of variables, the estimation of efficiency and additional analysis based on selected models of DEA (Fig. 2).
The environmental context in which a lower secondary school operates has a significant impact on the evaluation of the efficiency of its performance. Knowledge about identical for all lower secondary schools operating conditions needs to be expanded with a detailed diagnosis of situation of the analysed units. Then variables to estimate efficiency can be select. It is worth to check the database which are keept by local govern-ments for the purpose of education management and may be useful for the evaluation of efficiencys of lowers secondary schools. Cooperation with departments of education in local governments, regional examination boards and regional superintendent authorities is necessary to properly define the results (outputs) and expenditures (inputs), as well as obtaining quantitative data. The selection of variables to DEA model is the most important stage because it determines significantly performance indicators (Chodakowska, Komuda 2010;Cook et al. 2014). Author advises to identify the main factors differentiating schools to use a questionnaire survey, correlation matrix and factor analysis. Then DEA can be implemented to evaluate the lower secondary schools' management. DEA scores should be corrected by regression analysis to take into account environmental context. At the end the additional analysis basis on the DEA method can be carried out: sensitivity of the model to data errors or changing with the efficiency during the time. This stage is particularry important because the occrrence of data interference, may distort the classification of the units and may cause misjudgement of their effectiveness (Nazarko, Urban 2007).
It must be emphasized that the implementation of a system to assess the effectiveness by DEA would raison d'être only if there is feedback. Drawing conclusions from the results obtained is a fundamental issue to implement each system of effectiveness measurement.

Efficiency evaluation of Bialystok's lower secondary schools
Efficiency evaluation of the Polish lower secondary schools' management was done on the basic of municipal district Bialystok.
Firstly based on the literature review and with cooperation with educators, managers, superintendents from Regional Educational Board in Lomza and Regional Superintendent Authority in Bialystok was prepared a list of over 100 potential variables describing the schools, their work and the effects they achieve. Then questionnaire study was carried out. The questionnaire study was rather an in-depth experts' interview. A questionnaire was a scenario conversation, the impulse to free discussion with expertsschool directors and inspectors, superintendents. However experts do not significantly reduce the number of variables and therefore statistical analysis was implemented.
Preliminary statistical analysis were subjected to a set of 47 variables. This was a result of the exclusion of insignificant variables according to the respondents, and the variables without reliable quantitative data in education databases. Correlation matrix, as well as on conclusions derived from questionnaire survey and literature review allow to include in further analysis set of 20 variables. Then factor analysis was performed.
The 6 factors explain a total of more than 84% of the variance. The first factor had high values of factor loadings for variables that can be classified as "value-add of knowledge". The second factor represented the type of school: public/non-public. The third factor was strongly correlated with variables that indicate the implementation of inclusive education in school. The fourth factor is the absolute exam results. The fifth factor captures sport performance. The sixth factor can be considered as an "environmental" factor, talking about the economic situation of the families of students and parents' educational ambitions. Two variables were not included in any factor. The percentage of students participating additional classes and the unexcused absence of more then 30% of classes. Six dimensions that were obtained following the procedure of factor analysis indicates a high complexity of the problem of description lower secondary schools. Confirmed that the practice of creating rankings of schools solely on the basis of exam results does not take into account other relevant aspects of lower secondary schools.To build the DEA model author decided to use the value of each factor, and compare the results obtained taking into account the representatives, that is, the variables most strongly correlated with the factors. By testing the six different models (Table 1) with different combinations of inputs, outputs and environmental variables and making multiple rankings of Bialystok lower secondary schools author attempted to define the relevant criteria of evaluation.
Comparative analysis of the results led the author to focus on two models: M1 and M2. These models differ in the way of inclusion one variable: a percentage of students with no unexcused absences of more than 30% of classes. The results of evaluation studies can be presented in the various charts and classification (e.g. Fig. 3). Author in the research used rare in the literature super efficiency DEA model (Andersen, Petersen 1993;Guzik 2009). To evaluate the efficiency and take into account the exogenous environmental factors in which the school operates, author used a combination of DEA and regression method (Cooper et al. 2004).
Additional analysis based on DEA models were done to authenticate the results of research through testing the stability and sensitivity of selected models. Among other things, resistance of classification to data errors was checked. It was be done by making efficiency calculations after random distorting the value of inputs by a noise (Nazarko, Urban 2007). The procedure was repeated several times with different levels of coefficient of variation and different number of inputs modified to determine the acceptable level of errors that does not undermine the stability of the models. Author also conducted analysis of efficiency changes over time using Malmiquist index. Consecutive, annual analysis of graduates is more objective because eliminates the risk of incidental Variable -a percentage of students with special education needs X X X Variable -a percentage of students with no unexcused absences of more than 30% of classes X X Variable -a percentage of students participating additional classes X X X X Factor -an implementation of inclusive education in school X X X

Outputs
Variable -an average score obtained from the humanistic part of the exam relative to the average score predicted by EVA

X X X
Variable -a number of points obtained in the sport competitions to a number of students X X X Factor -a relative change in the exam results X X X Factor -a sport performance X X X Environmental Variable -a number of rooms in a school to a number of students X X X Variable -a lower secondary school students average score on the compulsory test at the end of primary school X X X Variable -a percentage of students not repeated a year X X X Variable -a percentage of students with no unexcused absences of more than 30% of classes X X Factor -type of a school: public/non-public X X X Factor -absolute exam results X X X Factor -"environmental" X X X circumstances that may have caused for not appropriate evaluation of the schools' efficiency. At the end of the analyses, author proposed using results of the DEA method in benchmarking, understood as comparing in order to identify best practices, establish criteria to improve the performance and measure progress. Using the features of a DEA linear programming problem can be determined the excess in inputs and/or deficiency in outputs.
Author proposed a concept of implementing analytical DEA tools inside information system gathering, processing and reporting data on expenditures and achievement of schools (Fig. 4).

Results, discussion and limitations
Considerations carried out on the basis of expert opinions, and above all quantitative research, narrowed the set of varaibles to several indicators with significant measurable impact on school achievement. Absolute school exam results are determined mostly by the results achieved on previous stage of learning. Thus, it can be say that the social and family factors and individual characteristics of the student, included implicitly in its achievements in primary school, have a considerable impact on the assessment of the efficiency of lower secondary school (Fig. 5).
Most non-public schools select the candidates for students and enroll those with the results above average. Public schools have no such opportunity. Definitely, assessment of the value-added is a better alternative to the schools performance evaluation then absolute exam scores. Assessment of the efficiency changes also, but to a lesser extent, an explicit inclusion of environmental variable: the percentage of students repeating a class.
Law in Poland that result specific financial implications, but also the observed practice of financing education sector clearly show the lack of relationship between expenditure on education and the efficiency of the school. Particularly unacceptable is the lack of effect of teacher work in their incentive benefits. It should be noted, that even the current legal regulations do not prohibit local governments to take into account the evaluation of the efficiency of lower secondary schools in the rules for the financing of these schools. One reason for this state of affairs might be just the lack of appropriate tools for assessing performance.
According to the author, the future of evaluation schools' management is to taking into account many different areas of the school performance, incorporate into assessment local and environmental context. a useful tool to measure school performance. DEA uses mathematical programming techniques to evaluate the performance of a given unit. Since DEA allows multi input and output variables, it can take into account many different areas of the school performance. In addition, output and input variables can be stated in different units of measurement. Because DEA is a non-parametric method it places no restriction on functional form of the production relationships between the outputs and inputs. DEA does not require also specification or knowledge of a priori weights or prices for the outputs and inputs. The concept of evaluation of efficiency by Data Envelopment Analysis is showed in Figure 6.

Conclusions
In the article on the basis of Bialystok lower secondary schools, the author shows that the implementation of DEA could be useful and provides additional knowledge about the effectiveness of educational institutions. Through its versatility and flexibility DEA method is a useful tool for a multicriteria measuring and benchmarking analysis of lower secondary schools. DEA efficiency evaluation can be an imitation of competitiveness that could stimulate the enhancement of education quality.
The author is aware that the DEA is not universal remedy for the problem of performance evaluation of lower secondary schools. As a deterministic non-parametric method DEA has the drawback that there are no conventional tests of significance or methods for drawing inference. Moreover efficiency estimates can be affected by sample size (Johnes 2014). Particular care should be taken in choosing the inputs and outputs of any DEA model which should consistent with the production process being evaluated (Cook et al. 2014). DEA should be considered as an alternative better waywhich does not mean with no flaws -of using of exams and other results and allows to assess the contribution of the school to obtain them. The main objection in presented example of using DEA is that only quantitative data was used. While, from the point of view of learning outcomes, are also very important subjective information such as the atmosphere in the school, relationships between students and the teacher, etc. DEA, allowing to compare the unit's results with the results of competitors, is a very commonly used method to evaluate the efficiency of educational institutions around the world. In the article a possibility of implementing DEA in the process of evaluating the schools in Poland was presented. On the basis of Bialystok lower secondary schools it was demonstrated that it is worth extending the currently used methods of evaluation of schools of the implementation of the DEA method and obtain additional knowledge about the efficiency of each school. Reliable assessment of the education operations could help to optimize the economic activities of local governments, taking into account the human and social factor. The information obtained may help to promote the school and identify the most effective ways of development of weaker schools. This knowledge would help making strategic decisions in the sector, aimed at improving the efficiency of schools in terms of the expenditures and the results achieved.
According to the author, DEA is an interesting and valuable methodological proposal in the comparative evaluation of efficiency of schools. The analysis results can be complementary to the activities of local governments and regional superintendent authorities and assist the formulation of guidelines for the educational policy and thus contribute to the improvement of schools' management.