Question

In the attached image you can see a problem. It would be very helpful if the problem could be broken down and explained, especially the symbols used and what they mean.

Your help will be acknowledged

4.5 Example: Linear Least Squares Suppose we want to find the value of z that minimizes (4.21) Specialized linear algebra algorithms can solve this problem efficiently; however, an also explore how example of how these techniques work. First, we need to obtain the gradient: (4.22) We can then follow this gradient downhill, taking small steps. Sce algorithm 4.1 with respect to r for details. Algorithm 4.1 An algorithm to minimize ()1A-b using gradient descent, starting from an arbitrary value of x Set the step size (e) and tolerance (8) to small, positive numbers. while IIA"An-A"bllz > δ do end while

Answer 1

So we have a function f(x) = 1/2 Norm(Ax-b)^2

This basically means that you want to minimize the length of the vector Ax-b, which is the same as finding the closest approximation to a solution of Ax=b. Even if this equation doesn't have a solution, we try to minimize the value of Ax-b to make it close to 0.

The algorithm given is one method of minimizing this function.

The tolerance delta is basically how small of an error you will be happy with. The smaller you set delta, the closer your approximation to the actual minimum. epsilon is the step size, which is how much distance you move everytime you iterate your loop

The given expression x(new) = x(old) - epsilon(....)

The vector in brackets is the direction in which f decreases the fastest. It is the gradient of f

Hope that helps. :)