Question

Question 1 [22 marks] (Chapt ers 2, 3, 4, 5, and 6) Let A e Rn be an (n x n) matrix and be R. Consider the problem 1 (P2) min2+ s.t. xe R" 1Ax-bil2 1 where & > O is fixed and Il IIl denot es the 2-norm. Call g.(x)=l|2 the objective function of problem (P2) 1Ax-bl2 i) [3 marks] Compute the gradient of g, and use it to show that the solution xi of this problem verifies (I+EATA)(x) = £A*b where I is the nx n identity matrix ii) [3 marks] Compute the Hessian of g and use it to prove that g. is strictly convex for every E > O. Is ge still strictly convex if = 0? iii) [3 marks] Explain why (IEATA) is invertible for every & > O and use this fact to express x (the solution obtai ned in part (i)) in terms of A, b and iv) [3 marks] Show that g, is coerci ve for every & > 0. Is g still coercive if 0? v) [5 marks] Given an initial iterate Xo, write down the next iteratex, produced by the pure Newt on method, and check that it converges to the solution x in a single iteration for any initial point. vi) [5 marks] Take Xo 0 and perform a single gradient step with a generic stepsize t 0. Show that the next iterate x, verifies + x1 t (EATA)(x;). Hint: Recall that xo = 0, so the expression for x, simpl ifies. You may also use parts (i) and (iii) w IN w1N

Answer 1

be Carsiden the mXm madrix (Pa) amd um 6.t. xIRm where O is Aixed ad 1111 deroteg the Call &l) 2-marun 11 AM-12 objedtive Arincttan the (P2) fproblem ol (1 the Finst anden mecenny eamditany is Forr mimimmum EAb. Mxm dentrty mabux I is the Where ()Henian Know AA is positive dehmite rur eny 0. > 0.and Simee is Fon dtrictly Comex madrix identi ty madrix whieh i an still auctly clenty po itne detiate, Se ie =0 iJr Convex on Pi >0 ill) Tt) inwatible js a At asitive mot eaual 0. ATA Since det (h'ht debimite So, eigenvalue. o So T+e) E ATb imvestidle to Tany Sa)

JAM Clandy So e 1s Coercive Th O. euy 11201 t Ko-(ItE (o teA(-)) Ko-(1+eMo (T )-h (TAEA'nEATb im a dingl solatoen the So it Coverge t i toatun n any imital fnt.