Question

In this problem, you will get more experience with taking derivatives with respect to vectors by proving

common identities. In the following, it will be useful to remember that if x = (x1, . . . , xn)^⊺ and y =(y1, . . . , yn)^⊺ are vectors, then the dot product x^⊺y is a scalar equal to

In this problem, you will get more experience with taking derivatives with respect to vectors by proving common identities. In the following, it will be useful to remember that ifx = (z, ,zn)T and y = (vi,..., Jn)T are vectors, then the dot product zTy is a scalar equal to 1iy Similarly, if A is a matrix and x is a vector, then the result of Ax is a vector whose ith entry is given by Σ.1 Aijzj. Suppose f is a scalar-valued function of a vector z. The derivative of f with respect to the vector z (also called the gradient of f w.r.t. x) is the vector whose ith component is df/dz a) Let x be a vector. Show that (gTz) = 2x b) Let A be a symmetric n × n matrix and x be a vector with n elements. Show that (zTAz) = 2Ax

Answer 1

$(a)~\\ {\bf x}^\prime{\bf x}=(x_1,x_2,...,x_n)^\prime \begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{pmatrix}=\sum_{i=1}^nx_i^2=x_1^2+x_2^2+....+x_n^2\\ \frac{\mathrm{d} {\bf x}^\prime{\bf x}}{\mathrm{d} {\bf x}}=\begin{pmatrix} \frac{\partial {\bf x}^\prime{\bf x}}{\partial x_1}\\ \frac{\partial {\bf x}^\prime{\bf x}}{\partial x_2}\\ \vdots\\ \frac{\partial {\bf x}^\prime{\bf x}}{\partial x_n} \end{pmatrix}=\begin{pmatrix} 2x_1\\ 2x_2\\ \ldots\\ 2x_n \end{pmatrix}=2\begin{pmatrix} x_1\\ x_2\\ \ldots\\ x_n \end{pmatrix}=2{\bf x}\\$

$(b)~\\ {\bf x}^\prime{\bf A}{\bf x}=(x_1,x_2,...,x_n)^\prime {\bf A} \begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{pmatrix}=(x_1,x_2,...,x_n)^\prime \begin{pmatrix} a_{11} &a_{12} & \ldots & a_{1n}\\ a_{21} &a_{22} &\ldots &a_{2n} \\ \vdots & \vdots & \vdots &\vdots \\ a_{n1} &a_{n2} &\ldots & a_{nn} \end{pmatrix} \begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{pmatrix}\\ =(x_1,x_2,...,x_n)^\prime \begin{pmatrix} a_{11}x_1+a_{12}x_2+ \ldots+ a_{1n}x_n\\ a_{21} x_1+a_{22} x_2 +\ldots +a_{2n}x_n \\ \vdots \\ a_{n1}x_1+a_{n2}x_2+\ldots+ a_{nn}x_n \end{pmatrix}=\sum_{i=1}^nx_iy_i$

$where,~{\bf y}=\begin{pmatrix} y_1\\ y_2\\ \ldots\\ y_n \end{pmatrix}= \begin{pmatrix} a_{11}x_1+a_{12}x_2+ \ldots+ a_{1n}x_n\\ a_{21} x_1+a_{22} x_2 +\ldots +a_{2n}x_n \\ \vdots \\ a_{n1}x_1+a_{n2}x_2+\ldots+ a_{nn}x_n \end{pmatrix}=\begin{pmatrix} a_{11} &a_{12} & \ldots & a_{1n}\\ a_{21} &a_{22} &\ldots &a_{2n} \\ \vdots & \vdots & \vdots &\vdots \\ a_{n1} &a_{n2} &\ldots & a_{nn} \end{pmatrix} \begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{pmatrix}\\={\bf A}{\bf x}\\$

$\frac{\mathrm{d} {\bf x}^\prime{\bf A}{\bf x}}{\mathrm{d} {\bf x}}=\begin{pmatrix} \frac{\partial {\bf x}^\prime{\bf A}{\bf x}}{\partial x_1}\\ \frac{\partial {\bf x}^\prime{\bf A}{\bf x}}{\partial x_2}\\ \vdots\\ \frac{\partial {\bf x}^\prime{\bf A}{\bf x}}{\partial x_n} \end{pmatrix}=\begin{pmatrix} y_1+a_{11}x_1+a_{12}x_2+\ldots+a_{1n}x_n\\ y_2+a_{21} x_1+a_{22} x_2 +\ldots +a_{2n}x_n\\ \ldots\\ y_n+a_{n1} x_1+a_{n2} x_2 +\ldots +a_{nn}x_n \end{pmatrix},\\ ~using~the~fact~that~{\bf A}~is~symmetric~i.e.~a_{ij}=a_{ji},~i\neq j\\ =\begin{pmatrix} 2y_1\\ 2y_2\\ \vdots\\ 2y_n \end{pmatrix}=2{\bf A}{\bf x}$