Question

1). The weight of w12 is damaged. Before this power failure the output of the network is 0.92129 when input x was applied. Compute the value of w12 weight supposing that the activation function is the logsig
(Ans: w12 = 7.5)

2 -3 Zx-X и W. W.. 12 13 1B = [-12 8.2 5.5] X3L5.3 W11 Neuron (fy 202 W13 X30 1 Bias

2). A power failure damaged weights w11 and w12. Before the damage the output network was 0.539915 when the first column of x was applied and 0.327393 for the second column of x. Compute the values of w11 and w12.
(Ans: w11 = -1 and w12 = 9.1)

3). The network has 2 hidden layers. Compute the size of the matrices: W1, W2 & W3 AND compute the number of weights of this artificial neural network

4). What are other activation functions apart from logsig and tanh

Please answer ALL FOUR QUESTIONS

Answer 1

Q1:

Z logsig (y) 1ey

yIn(

Now from the network given:

y1 = (w11 * x1) + (w12 * x2) + (w13 * x3) + w1B

y1 = (-12 * 2) + (w12 * -3) + (8.2 * 5.3) + 5.5 = 24.96 - 3*w12

here z1 = 0.92129

y1 = ln(0.92129 / (1-0.92129)) = 2.46

=> 24.96 - 3*w12 = 2.46

=> w12 = 7.5

################################################################################

Q2:

Similarly we will make 2 equations and then solve them to get the values of weights

For the first column:

y1 = (w11 * x1) + (w12 * x2) + (w13 * x3) + w1B

y1 = (w11 * 12) + (w12 * -13) + (15.2 * 7.3) + 19.5 = 130.46 + 12*w11 - 13*w12

here z1 = 0.539915

y1 = ln(0.539915 / (1-0.539915)) = 0.16

=> 130.46 + 12*w11 - 13*w12 = 0.16

=> 12*w11 - 13*w12 + 130.3 = 0 ---------------------- equation1

For the second column:

y1 = (w11 * x1) + (w12 * x2) + (w13 * x3) + w1B

y1 = (w11 * 2.2) + (w12 * -11) + (15.2 * 5.4) + 19.5 = 101.58 + 2.2*w11 - 11*w12

here z1 = 0.327393

y1 = ln(0.327393 / (1-0.327393)) = -0.72

=> 101.58 + 2.2*w11 - 11*w12 = -0.72

=> 2.2*w11 - 11*w12 + 102.3 = 0 ---------------------- equation2

Now we will solve equation1 and equation2 by using operation (11 * equation1 - 13*equation2)

We get:

(132*w11 - 143*w12 + 1433.3) - ( 28.6*w11 - 143*w12 + 1329.9) = 0

=> 103.4*w11 + 103.4 = 0

=> w11 = -1

now from equation2:

2.2 * -1 - 11*w12 + 102.3 = 0

=> w12 = 9.1

################################################################################

Q3:

Bias can also be considered as weights because they are learnable parameters.

1. For first layer of weights W1, input layer has 3 elements and "Hidden Layer 1" has 3 neurons. Therefore W1 has dimentions 3x3 = 9 weights. If we consider bias also as weights we get 9+3 = 12 weights.

2. For second layer of weights W2, "Hidden Layer 1" has 3 neurons and "Hidden Layer 2" has 2 neurons. Therefore W2 has dimentions 2x3 = 6 weights. If we consider bias also as weights we get 6+2 = 8 weights.

2. For third layer of weights W3, "Hidden Layer 2" has 2 neurons and "Output Layer" has 2 neurons(outputs). Therefore W3 has dimentions 2x2 = 4 weights. If we consider bias also as weights we get 4+2 = 6 weights.

Therefore total number of weights in ANN are:

6 + 4 + 9 = 19 (Excluding bias)

12 + 8 + 6 = 26 (including bias)

##############################################################################

Q4:

Basically activation functions have 3 properties:

1. Nonlinearity

2. Continuously Differentiability

3. Monotonicity

There are many other activation functions available. some of them are :

1. ReLU (Rectified Linear Unit): ReLU(y) = max(y, 0)

2. Leaky ReLU: LReLU(y) = max(y, alpha * y), where alpha is a small number e.g. 0.0001

3. ELU (Exponential Linear Unit): = x when x>=0 and alpha*(exp(x) - 1) when x<0