The relative efficiency of education and R&D expenditures in the new EU member states

The paper attempts to measure relative efficiency in utilizing public education and R&D expenditures in the new EU member states in comparison to the selected EU (plus Croatia) and OECD countries. As resources allocated to education and R&D sector are significantly limited, a special emphasis should be given to their efficient use regarding the institutional and legal constraints. By applying non-parametric methodology, i.e. Data Envelopment Analysis (DEA), a relative efficiency is defined as the deviation from the efficiency frontier which represents the maximum output/outcome attainable from each input level. An analysis of (output-oriented) efficiency measures shows that among the new EU member states Hungary, Estonia and Slovenia seem to be good benchmark countries in the field of primary, secondary and tertiary education, respectively. On the other hand, Cyprus and again Hungary dominate in the field of R&D sector, even if for different reasons. The empirical results also suggest that, in general, new EU member states show relatively high efficiency in tertiary education, while lag well behind in the R&D efficiency measures.


Introduction
Each nation's future wealth and competitive position in the globalised world depends increasingly on its ability to create and absorb knowledge. An essential feature of knowledge is that it requires human capital (educated persons) for both its production and its application. Indeed, long-term economic growth of the economy rests with its capacity to increase productivity through rapid technological progress. Therefore, the national systems of education and research and development (R&D) are the quintessential tools for the creation and application of knowledge. However, as most of the countries are faced with increasing demands on their limited (public) resources, there is an increasing pressure to improve resource allocation and utilisation. Accordingly, policy makers in a number of countries became increasingly concerned with measuring effi ciency. With education and R&D expenditures comprising a relatively important amount of national income, the interest in examining whether such expenditures are cost-effective has increased, recently.
The paper joins the efforts of other scholars in investigating education and R&D efficiency by applying a non-parametric methodology. Hence, the purpose of the paper is to review some previous researches on the effi ciency measurement of public education and R&D sectors as well as some conceptual and methodological issues of non-parametric approach. Most importantly, Data Envelopment Analysis (DEA) technique is presented and then applied to the wide range of the EU and OECD countries, including new EU member states 1 , to evaluate technical effi ciency within the both selected sectors. The importance of examining public sector expenditure effi ciency is particularly pronounced for emerging market economies where public resources are normally insuffi cient. When services are publicly provided, performance measurement becomes an inevitable management tool because when ineffi ciency continues, the constituents of that ineffi cient unit suffer. The government needs benchmarking tools to provide incentives to good performing sectors and to induce ineffi cient sectors to perform better. However, the focus of the paper is not on how to cut (public) expenditures, but rather more on investigating potential reserves to increase the value for money of public spending, i.e. how to make the most of limited public (and private) resources 2 .
The paper is organized as follows. In the next section we present a brief literature review of measuring public education and R&D expenditure effi ciency. Section 3 shows a theoretical background of non-parametric methodologies with special focus on Data Envelopment Analysis (DEA) and the specifi cations of the models. Section 4 outlines the results of the non-parametric effi ciency analysis of both, education and R&D sector. The fi nal section provides concluding remarks and some policy implications.

A brief literature review
Previous studies on the performance and effi ciency of the public sector (at national level) that applied non-parametric methods fi nd signifi cant divergence of effi ciency across countries. Studies include notably Fakin and Crombrugghe (1997) for the public sector, Gupta and Verhoeven (2001) for education and health in Africa, Clements (2002) for education in Europe, Aubyn (2003) for education spending in the OECD, Afonso et al. ( , 2006 for public sector performance expenditure in the OECD and in emerging markets, Afonso and St. Aubyn (2005, 2006a, 2006b for effi ciency in providing health and education in OECD countries. De Borger and Kerstens (1996), and Afonso and Fer-

Non-parametric methodology for assessing effi ciency in public sector
The measurement of effi ciency generally requires: (a) an estimation of costs; (b) an estimation of output; and (c) the comparison between the two. Applying this concept to the spending activities of governments, we can say that public expenditure is effi cient when, given the amount spent, it produces the largest possible benefi t for the country's population 3 . Often effi ciency is defi ned in a comparative sense: the relation between benefi ts and costs in country X is compared with that of other countries. This can be done for total government expenditure, or for expenditure related to specifi c functions such as health, education, poverty alleviation, building of infrastructures and so on. If in country X the benefi ts exceed the costs by a larger margin than in other countries, then public expenditure in country X is considered more effi cient. However, the measurement of public effi ciency is relatively complicated as comparison and measurement of both costs and benefi ts may be diffi cult. Defi cient budgetary classifi cations, lack of reliable data, diffi culties in allocating fi xed costs to a specifi c function, and failure to impute some value to the use of public assets used in the activity can also hamper the determination of real costs 4 . Figure 1 illustrates the link between input, output and outcome, the main components of effi ciency and effectiveness indicators. The monetary and non-monetary resources deployed (i.e. the input) produce an output. For example, education spending (input) affects number of students completing a grade (output). The input-output ratio is the most basic measure of effi ciency 5 . However, compared to productivity measurement, the effi ciency concept incorporates the idea of the production possibility frontier, which indicates feasible output levels given the scale of operations. The greater the output for a given input or the lower the input for a given output, the more effi cient the activity is. Productivity, by comparison, is simply the ratio of outputs produced to input used.
On the other hand effectiveness relates the input or the output to the fi nal objectives to be achieved, i.e. the outcome. The outcome is often linked to welfare or growth objectives and therefore may be infl uenced by multiple factors (including outputs but also exogenous 'environment' factors). The effectiveness is more diffi cult to assess than effi ciency, since the outcome is infl uenced by political choice. The distinction between output and outcome is often blurred and output and outcome are used in an interchangeable manner, even if the importance of the distinction between both concepts is recognized. For example, the outputs of a health system are often measured in terms of the number of operations performed or days spent in a hospital. The fi nal outcome, however, could be how many patients got well enough to return to an active life. Thus, the effectiveness shows the success of the resources used in achieving the objectives set.
A common approach to measure effi ciency is based on the concept of effi ciency frontier (productivity possibility frontier). There are multiple techniques to calculate or 4 More about measuring costs and effi ciency of public spending see Afonso et al. (2006). 5 When measuring effi ciency, a distinction can be made between technical and allocative effi ciency.
Technical effi ciency measures the pure relation between inputs and outputs taking the production possibility frontier into account. On the other hand, allocative ineffi ciency occurs if the distribution of particular public sector outputs is not in accordance with personal preferences (Bailey 2002: 119). Output Outcome

Effectiveness
Environmental factors e.g. Regulatory-competitive framework, socio-economic background, climate, economic development, functioning of the public administration estimate the shape of the effi ciency frontier. Most investigations aimed at measuring effi ciency are based either on parametric or non-parametric methods. The main difference between the parametric and the non-parametric approach is that parametric frontier functions require the ex-ante defi nition of the functional form of the effi ciency frontier. While a parametric approach assumes a specifi c functional form for the relationship between input and output, a non-parametric approach constructs an effi ciency frontier using input/output data for the whole sample following a mathematical programming method 6 . A calculated frontier provides a benchmark by which the effi ciency performance can be judged. This technique is therefore primary data-driven. Among the different non-parametric methods the Free Disposal Hull (FDH) technique imposes the fewest restrictions 7 . It follows a stepwise approach to construct the effi ciency frontier. Along this production possibility frontier one can observe the highest possible level of output/outcome for a given level of input. Conversely, it is possible to determine the lowest level of input necessary to attain a given level of output/outcome. This allows identifying ineffi cient producers both in terms of input effi ciency and in terms of output/ outcome effi ciency ).
An alternative non-parametric technique that has recently started to be commonly applied to (public) expenditure analysis is Data Envelopment Analysis (DEA) 8 . DEA is a non-parametric frontier estimation methodology originally introduced by Charnes, Cooper, and Rhodes in 1978 that compares functionally similar entities described by a common set of multiple numerical attributes. DEA classifi es the entities into "effi cient" or "performers" versus "ineffi cient" or "non-performers". According to DEA framework, the ineffi ciencies are the degrees of deviance from the frontier. Input ineffi ciencies show the degree to which inputs must be reduced for the ineffi cient country to lie on the effi cient practice frontier. Output ineffi ciencies are the needed increase in outputs for the country to become effi cient. If a particular country either reduces its inputs by the ineffi ciency values or increases its outputs by the amount of ineffi ciency, it could become effi cient; that is, it could obtain an effi ciency score of one. The criterion for classifi cation is determined by the location of the entities' data point with respect to the effi cient frontier of the production possibility set. The classifi cation of any particular entity can be achieved by solving a linear program (LP).
Various types of DEA models can be used, depending upon the problem at hand. The DEA model we use can be distinguished by the scale and orientation of the model. If one cannot assume that economies of scale do not change, then a variable returns-toscale (VRS) type of DEA model, the one selected here, is an appropriate choice (as opposed to a constant-returns-to-scale, (CRS) model). Furthermore, if in order to achieve better effi ciency, governments' priorities are to adjust their outputs (before inputs), then an output-oriented DEA model rather than an input-oriented model is appropriate. The way in which the DEA program computes effi ciency scores can be explained briefl y using mathematical notation (adapted from Ozcan 2007). The VRS envelopment formulation is expressed as follows: VRS p (Y 1 , X 1 , u 1 , v 1 ): min -(u 1 s + v 1 e); Y -s = Y 1 ; -X -e = -X 1 ; For decision making unit 1, x i1 ≥ 0 denotes the i th input value, and y i1 ≥ 0 denotes the r th output value. X 1 and Y 1 denote, respectively, the vectors of input and output values. Units that lie on (determine) the surface are deemed effi cient in DEA terminology. Units that do not lie on the surface are termed ineffi cient. Optimal values of variables for decision making unit 1 are denoted by the s-vector s 1 , the m-vector e 1 , and the n-vector  1 .
Although DEA is a powerful optimization technique that can assess the performance of each country, it has certain limitations. When one has to deal with large numbers of inputs and outputs, and a small number of countries are under evaluation, the discriminatory power of the DEA is limited. However, analysts can overcome this limitation by including only those factors (input and output) that provide the essential components of "production", thus avoiding distortion of the DEA results. This is usually done by eliminating one of a pair of factors that are strongly positively correlated with each other.
In the majority of studies using DEA, the data are analyzed cross-sectionally, with each decision making unit (DMU) -in this case the country -being observed only once. Nevertheless, data on DMUs are often available over multiple time periods. In such cases, it is possible to perform DEA over time, where each DMU in each time period is treated as if it were a distinct DMU. However, in our case the data set for all the tests in the study includes an average data for the 1999-2007 period (including PISA 2006 average scores) in order to evaluate long-term effi ciency measures as education and R&D processes are characterized by time lags in up to 37 EU (plus Croatia) and OECD countries. The program used for calculating the technical effi ciencies is the DEAFrontier software. The data are provided by Eurostat, OECD, UNESCO and the World Bank's World Development Indicators database.
The specifi cation of the outputs and inputs is a crucial fi rst step in DEA, since the larger the number of outputs and inputs included in any DEA, the higher will be the expected proportion of effi cient DMUs, and the greater will be the expected overall average efficiency (Chalos 1997). Common measures of teaching output in education used in previous studies are based on graduation and/or completion rates (see Johnes 1996;Jafarov, Gunnarsson 2008), PISA scores (see Afonso, Aubyn 2005;Jafarov, Gunnarsson 2008) pupil-teacher ratio and enrolment rate (see Jafarov, Gunnarsson 2008). On the other hand, the outputs of the R&D process are usually patents and publications (see Wang, Huang 2007;Sharma, Thomas 2008). Moreover, the literature shows that the specifi ca-tion of the inputs is generally in the form of domestic (public or total) expenditure (in % of GDP) (for education or R&D) or the number of teachers (or researchers) per million inhabitants. Nevertheless, these studies also demonstrate that DEA is an effective research tool for evaluating the effi ciency of education and R&D sectors, given varying input mixes and types and numbers of outputs.
Hence, similar to the former empirical literature, in this analysis the data set to evaluate education sector effi cency (at different levels) includes input data, i.e. (public) expenditure per student, tertiary (% of GDP per capita) or total expenditure on education (in % of GDP) and output/outcome data, i.e. school enrolment, tertiary (% gross), teacher/pupil ratio, primary completion rate, total (% of relevant age group), unemployment with tertiary education (% of total unemployment), labor force with tertiary education (% of total) and PISA 2006 average score. There are up to thirty-seven countries included in the analysis (selected EU (plus Croatia) and OECD countries). Different inputs and outputs/outcomes have been tested in four models (see Table 1). Moreover, to test a relative effi ciency of R&D sector, additional quantative input and output data is collected and processed. The inputs of the R&D process are total expenditure on R&D (as a % of GDP) and researchers in R&D (per million people). The output can be in the form of publications or patents (see Sharma, Thomas 2008), therefore the raw data for output employed in this study comprises total European patent applications (per million people), scientifi c and technical journal articles (per million people), and high-technology exports (% of manufactured exports). Table 2 shows the input and output/outcome data used in four different models.

Education effi ciency results
This subsection shows the empirical application of the Data Envelopment Analysis (DEA) 9 . When looking at the education results 10 , by using model 1 (see Table 1) and applying the DEA effi ciency frontier technique within a selected group of EU/OECD countries and Croatia to measure effi ciency of primary education, Denmark, Hungary and Portugal are seen as most effi cient. The effi cient countries are also Greece, Iceland and Romania, however, their primary expenditures per student (in % of GDP) is very low and have averaged less than 12% (the EU/OECD average is 18.7% in the considered period). One can also see that some countries come very close to the frontier (e.g. Czech R. and Italy), while the other countries are further away and therefore less effi cient (e.g. Turkey and Croatia) (see Table 3). Some less effi cient countries should signifi cantly decrease their input (primary expenditure per student) (e.g. Slovenia from 27.0% to 22.0%) and/or increase their outputs, i.e. school enrolment (e.g. Ireland and Poland), primary completion rate (Belgium) and teacher-pupil ratio (Turkey and Ireland) in order to become effi cient 11 . Interestingly, the new EU member states are, in general, 9 All the calculated results are available from the author on request. 10 All of the results relate to DEA with an output orientation, allowing for variable returns to scale (VRS). An output orientation focuses on the amount by which output quantities can be proportionally increased without changing the input quantities used. Using an input orientation approach leads to similar effi ciency results as those presented in the text. 11 The average output effi ciency score for primary education is 1.050, which means that the average country could increase the outputs/outcomes for about 5.0% if it were effi cient. The results also confi rm our expectations, that larger public sector increases the ineffi ciency in a primary education. relatively more effi cient than non-EU countries in the sample, however, they show relatively low effi ciency against the old EU-member states.
In terms of the effi ciency scores of secondary education, even ten analyzed countries are labeled as effi cient (see Table 3), however, only Romania and Slovakia represents new EU member states in this group of effi cient countries. The average output effi ciency score is 1.06715, which means that the average country could increase the outputs/ outcomes for almost 7.0% if it were effi cient. The worse performers are Mexico and Bulgaria with a well below average PISA scores (considerably less than 490), school enrolment (signifi cantly less than 103.6%) and teacher-pupil ratio (less than 0.086). Indeed, both countries should increase their outputs by more than 10% in order to become an effi cient (similar to the new EU member states average effi ciency, which is the least effi cient sub-group in the analysis).
When testing tertiary education effi ciency, eleven among the 37 countries analyzed within the formulation for tertiary education presented in Table 1 were estimated as effi cient. These countries are Canada, Czech R., Finland, Korea, Latvia, Lithuania, Poland, Russia, Slovakia, Slovenia and the United States. The results of the DEA analysis (Model 3) also suggest a relatively high level of ineffi ciency in tertiary education in a wide range of countries and, correspondingly, signifi cant room to rationalize public spending without sacrifi cing, while also potentially improving tertiary outputs and outcomes. Indeed, the countries under consideration could improve their effi ciency scores by decreasing their input (expenditure per student (in % of BDP)), in particular in Denmark and Switzerland. However, even more importantly, a signifi cant increase of outputs/outcomes is need in the form of school enrolment (in particular in Cyprus and Mexico), and in the form of labour force with tertiary education (in Portugal, Turkey and Romania). In general, output/outcome scores could be higher for about 6% on average. Interestingly, non-EU member states show signifi cantly worse DEA scores as they should increase their tertiary outputs/outcomes by more than 13% (in comparison to the old EU member states for about 7% and the new EU member states only for 1.4%).
Further empirical analysis, testing the effi ciency of the total expenditure on education (Model 4), shows that the worse effi ciency performers are Bulgaria, Romania and Portugal (see Table 4). Indeed, if these countries employed the resources in effi cient manner, they could increase their PISA scores by 19.5%, 15.6% and 13.6%, respectively. The main reason for the education ineffi ciency in these countries lies in transforming intermediate education outputs into real outcomes (see IMF 2008) (same problems have some other new EU member states, particularly Latvia, Lithuania and Hungary). The results also show that the best performers (in terms of effi ciency) seem to be Finland and Japan, while Greece presents a good effi ciency result due the lowest education spending (averaged only 3.6% of GDP in 1999GDP in -2007. Interestingly, output-oriented DEA results confi rm that Scandinavian countries could attain the same result with lowering their education expenditure by up to 2.3 percentage points (in Denmark). However, the new EU member states, in general, show the same effi ciency as the old EU member states (both groups could increase their PISA scores by around 10% on average). Notes: Relative effi ciency scores are based on models presented in Table 1. The countries are ranked from the most effi cient (e.g. Denmark ranks 1st for primary education) to the least effi cient (Belgium ranks 29th). Thirty-seven (or less) countries are included in the analysis (EU-27, OECD and Croatia).
The new EU member states are presented in italic.

R&D effi ciency results
The results of the output-oriented VRS formulation of DEA analysis (based on Models I-IV in Table 2) suggest a relatively high level of ineffi ciency in R&D sector in selected EU and OECD countries and, correspondingly, signifi cant room to rationalize this spending without sacrifi cing, while also potentially improving, R&D outputs and outcomes (see Table 5). Indeed, from the empirical results it may be seen that the total number of effi cient countries varies signifi cantly from one model to the other. There are only two technically effi cient countries in Model I, i.e. Cyprus and Switzerland.  . In order to enhance the reliability of the fi ndings, additional inputs and outputs/outcomes has been introduced, resulting in model II, III and IV (for details see also Table 2).
Adding another output in the form (Model II) of total number of scientifi c and technical journal articles (per million people), the results show Cyprus, Iceland and Switzerland to be technically most effi cient countries. Not surprisingly, the increasing number of the outputs in a relatively small sample leads to a higher number of effi cient countries. In general, the rankings remain relatively stable in comparison to the Model I (with Iceland 12 as only signifi cant exception). According to the presented empirical analysis, it is obvious that the R&D sector in many considered countries suffers from relatively low technical effi ciency. The ineffi ciency is particularly highlighted in the new EU member states (plus Croatia) and some less developed OECD members, i.e. emerging market economies (see Table 5). As most of these countries signifi cantly lag behind as far as total expenditure on R&D (in % of GDP), it will be crucial for them to increase these resources in an effi cient manner. Hence, the improvement of the sector's effi ciency, which can signifi cantly contribute to the development and the growth of the country, should therefore be a top priority practically for all countries in the near future.

Conclusion
In recent years, the debate of the role of the public sector has shifted signifi cantly towards empirical assessments of the effi ciency and usefulness of its activities. Indeed, tight budgets and demanding citizens put governments under increasing pressure to show that they are providing good value for money. Providing information about public sector performance can satisfy the public's need to know, and could also be a useful tool for governments to evaluate their performance. In this respect, the aim of the paper was to apply a common non-parametric method (Data Envelopment Analysis-DEA) to measure technical effi ciency in two extremely important sectors that signifi cantly determine long-run economic growth of the national economy, i.e. education and R&D. Moreover, in the paper the analysis also shows how DEA can be used for classifi cations and rankings of the countries in two highly important sectors for national economy.
The empirical results show that technical effi ciency in education and R&D sectors differs signifi cantly across the great majority of the EU (including new EU member states) and OECD countries. The analysis of different (output-oriented) effi ciency (under VRS framework) shows that Japan, Korea and Finland seem to be the most effi cient countries in the fi eld of education sector, while Switzerland and Netherlands dominate in the fi eld of R&D sector. When focusing only on the new EU member states, Hungary, Estonia and Slovenia seem to be good effi ciency performers in the fi eld of primary, secondary and tertiary education, respectively. On the other hand, Cyprus and Hungary dominate in the fi eld of R&D sector, even if for different reasons. The empirical results also suggest that, in general, new EU member states show relatively high effi ciency in tertiary education, while lag well behind in the R&D effi ciency measures. All in all, the analysis fi nds evidence that most of the new EU member states have a great potential for increased effi ciency in (public) spending of limited education and R&D resources.
However, a few limitations of the presented empirical study should be pointed out. Firstly, the applications of presented techniques are hampered by lack of suitable data to apply those techniques. Quality data are needed because the techniques available to measure effi ciency are sensitive to outliers and may be infl uenced by exogenous factors. Indeed, substantial ineffi ciency may be simply a refl ection of environmental factors (such as climate, socio-economic background, etc.). This also suggests applying a combination of techniques to measure effi ciency. Secondly, the precise defi nition of inputs, outputs and outcomes may signifi cantly infl uence the results. Finally, it seems important to bear in mind that by using a non-parametric approach, and in spite of DEA being an established and valid methodology, differences across countries are not statistically assessed, which can be considered as a limitation of such methodology. Hence, further research is clearly needed to eliminate the above defi ciencies, in particular to test the infl uence of the environmental factors on education and R&D sector effi ciency.