OPTIMALITY TESTING IN STOCHASTIC AND HEURISTIC ALGORITHMS

. In this paper we consider the application of order statistics to establish the optimality in stochastic and heuristic optimization algorithms. We suggest a method for the estimation of confidence intervals of minimum using order statistics which is implemented for optimality testing and stopping in stochastic approximation and Simulated Annealing algorithms. The efficiency of this approach is discussed using the results of application to continuous optimization and Bin-packing problem


Introduction
The stopping problem is topical in stochastic and heuristic optimization algorithms. In this paper we consider the application of order statistics to establish the optimality in stochastic and heuristic optimization algorithms and to stop the algorithm when the confidence interval of minimum becomes less than admissible value. Statistical inferences on the maximal (minimal) value of a function are described by Zilinskas and Zhigliavsky [1], and the application of the extreme order statistics in the estimation of the location of the maximum of a regression function can be found in Chen [2].

Method for testing the optimality by order statistics
The optimization problem is (minimization) ( ) min where ℜ → ℜ n : f is a function bounded from below, which elements are function values ) x ( f k k = η . We suggest a method for the estimation of confidence intervals of minimum according to order statistics when the number of iterations is finite. To estimate confidence intervals for minimum A of the objective function it suffices to choose from sample H only k+1 order statistic: ( ) [1]. Then the linear estimators for A can be as follows: where k is much smaller than N, a 0 ,…, a k are some coefficients satisfying the condition Let us consider simple sets of coefficients proposed by Hall [3]: , Λ is the symmetric matrix which elements are as follows: , α is the parameter of distribution of extreme values. We examine a choice of this parameter for continuous optimization β is the parameter of homogeneity of the function ( ) Then two-side confidence interval of the minimum of the objective function is as follows where The one-side confidence interval of the minimum of the objective function is as follows: The experimental research is a way of exploring the behaviour of stochastic and heuristic algorithms [5] and, thus, we investigate the implementation of the approach developed for several algorithms by computer modeling.

Optimality testing in stochastic approximation
Let us consider the application of this approach to Simultaneous Perturbation Stochastic Approximation (SPSA). Assume that function ( ) is a stochastic gradient, [6]. The smoothing is a standard way for the optimization of Lipshitz functions [6,7]. We consider a smoothed function where smoothing density p is Lipshitz function as well, 0 ≥ σ is the perturbation parameter. Functions smoothed by this operator are twice continuously differentiable [6]. The gradient of the smoothed function may be expressed as follows: x g and the stochastic gradient introduced: Let us introduce an optimising sequence: where i g is the value of the stochastic gradient at point i x , i ρ is a scalar multiplier, i σ is the value of the perturbation parameter in iteration i, 0 x is the initial point. This sequence used in the construction of sample (2) converges a.s. to the solution of the optimization problem under convergence conditions typical for Stochastic Approximation algorithms [6].

Experimental results for stochastic approximation
The proposed method was tested with functions: . The coefficients of sequence (7) were chosen according to the convergence conditions [6]: The estimate of minimum A, one-side and two-side confidence intervals of minimum A of the objective function, when coeficients a in (3) are computed according to all the cases described above, are given in Tables 1 and 2.
The upper and lower bounds of the confidence interval   (5) and (6) approximate the confidence interval of minimum value rather well and that the length of the confidence interval decreases when the number of iterations increases.
Thus, using formulas (5) and (6), we can introduce the stopping rule for the algorithm, namely, the algorithm stops when the length of the confidence interval becomes less than admissible value 0 > ε . We can see the asymptotic behaviour of the stopping rule in

Description of a simulated annealing algorithm
Let us consider the application of this approach to continuous global optimization by a simulated annealing algorithm (SA).
The procedure of SA algorithm for solving problem (1) is described as follows [8].
Step 1. Choose initial point , a kind of temperature-dependent generation prob-ability density function, a corresponding temperature updat-ing function, and a sequence } 0 Step 2. Generate random vector i z by using the generation prob-ability density function. If there exists   r  e  t  n  i  e  c  n  e  d  i  f  n  o  c  e  h  t  d  n  u  o  b  r  e  w  o  L  d  n  u  o  b  r  e  p  p  U  0  0  5  =  n  ,  2  =  n  4 Step 4. If the prescribed termination condition is satisfied, then stop; otherwise, update the value of the temperature by means of the temperature updating function, and then go back to Step 2.
The sequence { ; 0} i i ρ ≥ of positive numbers specified in Step 1 of the above SA algorithm is used to impose a lower bound on the random vector generated at each iteration for obtaining the random trial point. This lower bound should be small enough and monotonically decreasing as the annealing proceeds. Since the temperature-dependent generation probability density function is used to generate random trial points, and since only one trial point is generated at each temperature value, SA algorithm considered is characterized by a nonhomogeneous continuous-state Markov chain. By applying the generation mechanism and the Metropolis acceptance criterion, SA algorithm produces three sequences of random variables. These are sequence { ; 0} i z i ≥ of random vectors generated by the generation probability density function, sequence { ; 0} i y i ≥ of trial points generated by (8), and sequence { ; 0} i x i ≥ of iteration points deter-mined by applying the Metropolis acceptance criterion as described in Step 3. These three sequences { ; 0} i T i ≥ of random variables are all dependent on temperature sequence determined by the temperature updating function.
The next conditions yield the global convergence property of the objective value sequence induced by SA algorithm as described above for solving problem (1) (see Theorem 1 in [8] ).
These conditions indicate that under suitable conditions an appropriate choice of the temperature updating function ensures the convergence of SA algorithm to the global minimum of the objective function over the domain of interest.
The temperature updating functions are given below corresponding to different kinds of generation probability density functions to guarantee the global convergence of SA algorithm.
Temperature updating 1. Suppose that the assumptions described above hold. Let n r ℜ ∈ , with components is the sequence used to impose lower bounds on the random vectors generated in SA algorithm.
Let the temperature-dependent generation probability density function ) , ( i T p ⋅ be given by Then, for any initial point is the sequence used to impose lower bounds on the random vectors generated in SA algorithm. Let the temperature-dependent generation probability density function ) , ( i T p ⋅ be given by Then, for any initial state is determined by the temperature updating function: is the sequence used to impose lower bounds on the random vectors generated in SA algorithm. Let the temperature-dependent generation probability density function ) , ( i T p ⋅ be given by is the initial temperature value and 0 > l is a given real number.
The descriptions above indicate that a different form of the temperature updating function has to be used with respect to a different kind of the generation probability density function in order to ensure the global convergence of the corresponding SA algorithm. Furthermore, the flatter tail of the generation probability function implies the faster decrement of the temperature sequence determined by the temperature updating function.

Experimental results for simulated annealing
Experimental research is a way to explore the behaviour of stochastic and heuristic algoritms (Pardalos (2000)). The proposed method was tested with Beale functions: where the search domain is: and global minimum point is: (x 1 , x 2 ) * =(3, 0.5) and minimum function value is f((x 1 , x 2 ) * )=0. The extreme value distribution parameter a was chosen according to the homogeneity rule (4). From Figs 3 and 4 we can see the estimate of minimum A, confidence bounds of this minimum value, the hitting probability to the confidence interval of minimum A and its confidence bounds. The results presented in these pictures have been obtained using temperature updating 3. Similar results were obtained for temperature updating 1 and 2.

Experimental results for Bin-packing
Modification of the Simulated Annealing method described above was applied to Bin-packing problem.
The objective function was as follows: . Thus, set D consists of vertices of the n-dimensional cube.  For comparison the real minimum value was established by the full selection of vertices D.

Conclusions
The linear estimator for the minimum value of function is proposed using the theory of order statistics and is studied in an experimental way. The estimators proposed are simple and depend only on the parameter of the extreme value distribution a. Parameter a is easily estimated using the parameter of homogeneity of the objective function or in a statistical way. Theoretical considerations and computer examples have shown that the confidence interval of the function minimum can be estimated with admissible accuracy when the number of iterations is increased. The stopping rule using the minimum confidence interval has been proposed and implemented in the Stochastic Approximation and Simulated Annealing. The optimality testing approach was explored by Monte-Carlo method. The objective function was minimized by SA method, where the numbers i y were randomly generated with uniform distribution ( ) in each trial. Since in the latter case the value of ( ) x f is asymptotically Gaussian, the extreme value distribution parameter α is equal to 1.
In Figs 5 and 6 we can see the estimate of minimum A, confidence bounds of this minimum value, the hitting probability to the confidence interval of minimum A and its confidence bounds as well. The results presented in these pictures have been obtained using temperature updating 1.