Question

Write a tutorial that compares and contrasts a number of different methods for computing a locally optimal solution of a smooth unconstrained minimization problem.

Answer 1

Unconstrained optimization problems -

consider the problem of minimizing an objective function that depends on real variables with no restrictions on their values. Mathematically, let x∈Rnx∈Rn be a real vector with n≥1n≥1 components and let f:Rn→Rf:Rn→R be a smooth function. Then, the unconstrained optimization problem is

minxf(x).minxf(x).

Unconstrained optimization problems arise directly in some applications but they also arise indirectly from reformulations of constrained optimization problems. Often it is practical to replace the constraints of an optimization problem with penalized terms in the objective function and to solve the problem as an unconstrained problem