New Rule for Choice of the Regularization Parameter in (Iterated) Tikhonov Method

. We propose a new a posteriori rule for choosing the regularization parameter α in (iterated) Tikhonov method for solving linear ill-posed problems in Hilbert spaces. We assume that data are noisy but noise level δ is given. We prove that (iterated) Tikhonov approximation with proposed choice of α converges to the solution as δ → 0 and has order optimal error estimates. Under certain mild assump-tion the quasioptimality of proposed rule is also proved. Numerical examples show the advantage of the new rule over the monotone error rule, especially in case of rough δ


Introduction
In this paper we consider linear ill-posed problems Au = f, (1.1) where A ∈ L(H, F ) is a bounded operator with non-closed range R(A) of A and H, F are infinite-dimensional real Hilbert spaces. We assume that instead of exact data f there are given noisy data f δ ∈ F with f δ − f ≤ δ and known noise level δ. The well-known methods for solving ill-posed problems are the Tikhonov method and iterated variant of this method u α = u α,m , u α,i = (αI + A * A) −1 (αu α,i−1 + A * f δ ), (i = 1, 2, . . . , m). (1.3) Note that approximation (1.2) is a special case of (1.3), where m = 1 and for initial approximation u α,0 = 0 is used. For applying methods (1.2), (1.3) the important problem is a proper choice of the regularization parameter α = α(δ).
Well-known rules for parameter choice in methods (1.2), (1.3) are the discrepancy principle [8,19,20], the modified discrepancy (MD) rule [1,11,12], the monotone error (ME) rule [18], rule R1( [13,14,15]). In this paper we propose a new rule. In numerical experiments we get typically smaller error of approximations (1.2), (1.3) than by using the other rules, particularly in case of rough estimates of the noise level. Both, the rule R1 and the proposed rule are applicable also in the case, when problem (1.1) has solution only in the least squares sense (f ∈ R(A), but A * f ∈ R(A * A)).
In this paper in Section 2 we consider rules for a posteriori choice of α. For the proposed rule the convergence and order optimality are proved in Section 3. The quasioptimality properties of the new rule are studied in Section 4. In the final section numerical results are given.

Rules for Parameter Choice
Before considering parameter choice rules we remind, that approximations (1.2), (1.3) have the following form where the generating function Here γ p = p m p 1 − p m m−p and γ * = 1/2 for the Tikhonov method and γ * = √ m in case m ≥ 2. We introduce notations In the following we consider rules for the choice of the regularization parameter α = α(δ) for guaranteeing the convergence of the regularized solution where u * is a solution of problem (1.1) nearest to the initial approximation u 0 .
Let b = const ≥ 1. Then in the discrepancy principle, in MD rule and in ME rule the corresponding regularization parameters α D , α MD and α ME are chosen as solutions α = α(δ) of equations respectively. The ME rule has the properties therefore α ME ≥ α opt := arg min{ u α − u * , α > 0}. Usually α ME < 1 and α opt = α c ME with some c ≥ 1. Our extensive numerical experiments presented in [2] have suggested the use of MEE rule with parameter α MEE = α 1.1 ME and we have obtained the average error ratio u αMEE − u * / u αME − u * ≈ 0.8.
If u * − u 0 ∈ R((A * A) p/2 ), the order optimal error estimate u * − u α = O(δ p(p+1) ) holds true for the range p ∈ (0, 2m − 1], if α is chosen by the discrepancy principle, and for the larger range p ∈ (0, 2m], if α is chosen by the MD, ME or by the R1 rule.
To introduce a new rule, we denote We choose the regularization parameter α = α(δ) as the smallest solution of the equation To show that this equation has a solution, let us consider function d(α). It is obvious that lim On the other hand, since κ α D 1/2 α In order to avoid the instability in computing problems in finding the smallest solution of (2.4) for all α ≥ 0, we recommend to find the smallest solution From the following inequality Note that for numerical realization of rule R2 additionally iterated approximations u α,m+1 and u α,m+2 may be replaced by linear combinations of approximations u α,m with different parameters α. These ideas for rules MD, ME and R1 are realized in [16].

Convergence and Order Optimality
To prove this theorem we need the following auxiliary result, which was proved in [15] (Lemma 3).
Lemma 1. Let α(δ) be such a parameter that for each α ≤ α(δ) the inequality holds. Then α(δ) ≤ α 0 , where α 0 is a parameter, for which the function it follows from inequality (2.6) that Analogously to the proof of convergence and order optimality in discrepancy principle (see [19,20]) the last inequality gives

⊓ ⊔
We note that numerical examples and results of Section 4 show that probably the estimate (3.1) holds with 0 < p ≤ 2m.

On the Quasioptimality of Rule R2
In this section we show that the choice of α by rule R2 in methods (1.2), (1.3) is under certain conditions weakly quasioptimal (see [15]) in sense that the error estimate (3.2) also holds, if the right hand side there is replaced by its infimum over α ≥ 0, multiplied by a constant.
We denote Let {F λ } denote the spectral family of the operator AA * . From (2.1) follows that then the function t c (α) is monotonically increasing in the interval [α 0 , α 1 ].
Proof. From (2.1) follows that d dα g α,m (λ) = − m α 2 α α+λ m+1 . Using this equality and the equality g α,m (A * A)A * = A * g α,m (AA * ) (see [19,20]) we have This equality and the inequality g α,m (λ) ≤ m α α α+λ (see (2.1)) give This inequality and equality (4.1) give Therefore in case (4.2) the function t c (α) monotonically increases in the interval In the following, we need the condition This condition is satisfied in special cases, if the error of the right hand side Numerical experiments show that for most severely ill-posed problems in case of uniform distribution of error f the condition D Proof. From the equality This inequality together with condition (4.3) gives (4.4) Assume now that α ≥ α(δ). Then from (2.4), (4.4) we conclude that To continue this estimation, we use the inequalities B α ≤ 1, D α ≤ 1,   Denoting c = min(c, 1) and c = max(c, 1) we have To estimate the first summand in (4.6), we use the inequality (2.6) which allows us to use Lemma 1 with c = bκ −1 α + γ 1/2,2m+1 , w = u and we get

Summation of the last inequality and (4.7) gives asserted estimate (4.5). ⊓ ⊔
The quasioptimality of the rule (2.4) is proved under the special condition (4.3) for f , but remains an open problem in the general case. Note, that quasioptimality guarantees best convergence rates on all solution sets, including sets from [5,6,7,9,10,17].
The results of Theorems 1, 2 were proved for constant b > 2. The following Theorem 3 shows, that under certain additional conditions the quasioptimality of Rule R2 holds also for smaller b. Numerical experiments show that for constant b smaller values than indicated in Rule R2 may be used.

Numerical Experiments
We solved test problems from [4] with discretization parameter 100, the other parameters had standard values. Instead of exact data f , a randomly perturbed data f δ was used with error f − f δ = dδ, where the values of δ were 10 −(i+1)/2 f , i = 1, . . . , 11 and the values of d were 1, 0.7 and 0.3. In case d < 1 the noise was overestimated: actual noise f − f δ was smaller than its estimate δ. The problems were regularized by the Tikhonov method using αvalues α i = q i , i = 0, 1, . . ., q = 0.9. In the model equations the exact solutions are known. We found α opt as α with the smallest error: We solved these problems 10 times using ME rule with constant b = 1.01, the MEE rule and rule R2 with constants b = b1 := 1.01 γ 2 1/4,1 = 1.01 2 3 √ 3 and b = b2 := 0.7 γ 2 1/4,1 . We also used the combination of the ME and R2 rules with b = b2 taking the minimal value from these parameters: α MER2 (δ) = min{α ME (δ), α R2 (δ)}. In Tables 1-3 the averages and maxima (over all δ and 10 runs) of the error ratios r = u α(δ) − u * / u αopt − u * are given with parameter in numerator from the corresponding rule.
In case of exactly given noise level δ (see Table 1) the MEE rule gave the best results, the rule R2 with b = b1 gave worse results than the ME rule and the rule R2 with b = b2 gave in half of our problems better results than the ME rule. The combination of the ME and R2 rules gave better results than ME and R2 rules. In case of overestimated δ (see Tables 2, 3) the rule R2 with both constants b is better than the ME rule and MEE rule (if d > 1, parameter α c ME with c > 1.1 would be better than α 1.1 ME ) and gave in most cases a smaller parameter than the ME rule.
We recommend the new rule (or its combination with ME rule or with MEE rule) especially in case of roughly estimated noise level δ.