Question

1. Neural networks often have many parameters that need to be optimised. Suppose that in a simple case a particular neural network has just two parameters x and y that satisfy y and x2 + y2 25. An analyst establishes that the performance function of the network is f(x, y)-(x2 + y2)3/2-6(x2 + y2) + 9y. (a) Find ▽f(x,y). (b) Find the Hessian matrix H(x, y) for f (, y (c) Locate and classify all stationary points of f(x, y) (d) At what values of x, y is the network performance maximised? (e) At what values of x, y is the network performance minim ised? (f) In which direction if the function f decreasing most rapidly when (x, y) (,1)?

Answer 1

f(x, y) = (x^2 + y^2)^3/2 - 6(x^2 + y^2) + 9y (a) sf/sx = 3/2 (x^2 + y^2)^1/2 2x - 12x = 3x{(x^2 + y^2)^1/2 - 4 sf/sy = 3/2 (x^2 + y^2)^1/2 2y - 12y + 9 = 3{y(x^2 + y^2)^1/2 - 4y + 3} so, delta f(x, y) = (sf/sx, sf/sy) = (3x (x^2 + y^2)^1/2 - 12, 3y(x^2 + y^2)^1/2 - 12y + 9) (b) H(x, y) = (s^2 f/sx^2 s^2 f/sys x s^2 f/sx sy s^2 f/sy^2) Now s^2 f/sx^2 = 3(x^2 + y^2)^1/2 + 3. 1/2 x (x^2 + y^2)^-1/2 2x - 12 = 3(x^2 + y^2)^1/2 + 3x^2 (x^2 + y^2)^-1/2 - 12 s^2 f/sys x = 3xy (x^2 + y^2)^-1/2 and s^2 f/sx sy = 3xy (x^2 + y^2)^-1/2 s^2 f/sy^2 = 3(x^2 + y^2)^1/2 + 3y^2 (x^2 + y^2)^-1/2 - 12 therefore H(x, y) = (3(x^2 + y^2)^1/2 + 3x^2 (x^2 + y^2)^-1/2 - 12 3xy (x^2 + y^2)^-1/2 3xy (x^2 + y^2)^-1/2 3(x^2 + y^2)^1/2 + 3y^2 (x^2 + y^2)^-1/2 - 12