Question

2. Steepest descent for unconstrained quadratic function minimization The steepest descent method for minimize f(x) is the gradient descent method using exact line search, that is, the step size of the kth iteration is chosen as Ok = argmin f(x“ – av f(x)). a20 (a) (3 points) Consider the objective function f(x):= *xAx- Ax - c^x + d. where A e RrXnCER”, d E R are given. Assume that A is symmetric positive definite and, at xk, f(x) = 0. Give a formula of or in terms xk, A, c, d. (b) (5 points) Consider R2. Starting from xº = (0,0), perform two iterations of the steepest descent method for minimize f (11,22) := 5(+ zì) +222 +1. +

Answer 1

Steepest Descent Method Cik = argmin f. Gacka vf69) azo + f(x) = 1 x Ax- c T x + d Calcution of derivative (ie, I f(x)) f(x+3) = fou + le (E) +0 (11Z!!) where, Ix (Z) = f(x) Bor 3=20. (small 3) f(x+3) = + (x+33TA (644) – c" (k+3) +d = (x++37) A (x+3) - C + 3) + 200 C (A+B)T= ATART) (XA (x+3) + 3T A(+ 3)) - Čxels 2 td

- 1 (&? Axe + xA3 +37A3C+3 A3) -ca-c3 Separate the variables, = ( x A x - Ex+b) + (x Az + 32 12 - C3) + 3AX 2 (10) + fx 'as t3 TA3-cs) + 3' = f) + Exc"A +43 A -(")3 +13TAX a This is not correct go back one step up that f(x) + XA3 +3 A3-03 +13TAX collect

for fx-CP) A3 +$"AX) 2 op compare equation 6 and 1 la lx 3= EXTA-CT) Z > veflr) = 420A-"] Che = estre) we have got tlf (C) For minima If(x) = 0. XTA = CT A x = 20 x =2(AT J7 C argmin (f (99) Ollique

4 Now Tf lack) = ! (CK) TACT a flak) = a ( 2 (CK) TA (7) Dek_ o f@k) = x - x 6(xK) TA-CT) . flok_d of (@K)= f(x50 €125 TA -CT)) 3. flores et XTAX_ctx td given f(xX ac ffx* TA~(7)) = ax = next line * (** te STA-C73) A (****! A =CT)) c? (xta xxx A-(T)) to T - A CATA"_()) A (x² a Čaura - (?) z ہو ( مورچه ای TCے ۔ А

Ov=4 (25% dimnxk.c)A (see algatae) -c" (xk er fuckTA - (0) 14 d This is you ck. It is dificult please consider this as your answer. The answer is conceptually and mathematically Correct. where, a= 2K-2 (AT) - C to reduce it 1/2 xk A-CT

calculation of 'a. xk - au flack) = 2. (ATTC x* 2 CAT/C I f(x) T 7.ffxK) = 2 x 5A - c x - 2 CATIC 2 xk A - CT d = 2

7 xº = (0,0) = (x2) minimize f (x + x2 ) = (x + x) + 2x2+1 The column vector for f (x2+xz²) is af C = azt, af 22 J X2+2 C = X + 2

8 ) c-7fc x₂ +2 for Clo x = 0, y =ó (given) = O 2 Normalized c (6) 20 says Steepest descent method x6 = x()-OCCO where 20) and 26. [] calculated

first iteration 200 zelo + a C(O) say for small a = 0.01 o it (0.4) 1 061) +0.61 Second iteration xle) Dell + ac) C(1) - Loom)- 1600 2+2 0.0172 12.01 Ć (1) - - 6 (2001) 2.0) IN 1

x(2) 0 7 0.01 ) x(2) 8.02 f (x1); 120) = 21070²) + 2x0+ f(0, 2) = f($",x2")= 100*7.60.0132) +2 X0-01+) I X164 + 2r0.01 t/ 2 f 67,0 x = 1.020056 f(, ) = (oP+ (0,02%) +276:62 + 1 1.0402