Question

will give thumbs up to 3/5 answers to question

1. Select all reasonable methods for handling local minima when training an ANN (Artificial Neural Networks):

1. restart the training several times from the same initial state
2. use simulated annealing
3. perturb the weight matrix slightly and continue the training
4. use a momentum term
5. use full gradient descent
6. add an additional hidden layer

2. Select all that are true in regard to the hidden units of a fully-connected ANN:

1. unlike decision tree nodes, ANN nodes have a meaning that can be clearly interpreted
2. the nodes in each successive layer represent higher level abstractions
3. a given learned feature is always represented by a single node in the network
4. there must be the same number of units in each hidden layer in the network

3. Select the reasons that the computational complexity of the convolutional NN (CNN) allows researchers to create a 313 layer network to train on ImageNet (millions of images) within a reasonable time.

1. Images are down sampled to a much smaller size.
2. For one layer, a single k x k convolutional weight matrix (also know as a kernel) is learned, rather than separate matrices for each k x k grid in the image.
3. The pooling layers generally down sample the features, for example, by replacing a 2x2 grid with its max value.
4. Shift invariance always allows the CNN to learn more quickly
5. They use stochastic gradient descent rather than the computationally complex full gradient descent.
6. They use sparse input matrices.

4. On which of the following data is it possible to use CNNs?

1. 1D vector data
2. 2D image data
3. 3D sculpture data
4. 4D air readings (3D space plus time)
5. Sequence-based data

5. Select all of the disadvantages to performing evaluation by using k-fold CV versus using separate training and test datasets?

1. When you tune hyper-parameters, you are tuning effectively them to the test set; so you will overestimate performance (such as accuracy) that you expect on unseen/production data.
2. It is not possible to use as much training data in k-fold CV. Therefore, it is unlikely the classifier will perform as well.
3. The effective size of your test set is much smaller in k-fold CV.

Answer 1

Solution:

Select all reasonable methods for handling local minima when training an ANN (Artificial Neural Networks):
a. restart the training several times from the same initial state
b. use simulated annealing
c. perturb the weight matrix slightly and continue the training

Select all that are true in regard to the hidden units of a fully-connected ANN:
a. unlike decision tree nodes, ANN nodes have a meaning that can be clearly interpreted
d. there must be the same number of units in each hidden layer in the network

Select the reasons that the computational complexity of the convolutional NN (CNN) allows researchers to create a 313 layer network to train on ImageNet (millions of images) within a reasonable time.
a. Images are down sampled to a much smaller size.
b. For one layer, a single k x k convolutional weight matrix (also know as a kernel) is learned, rather than separate matrices for each k x k grid in the image.
c. The pooling layers generally down sample the features, for example, by replacing a 2x2 grid with its max value.

On which of the following data is it possible to use CNNs?
e. Sequence-based data

Select all of the disadvantages to performing evaluation by using k-fold CV versus using separate training and test datasets?
a. When you tune hyper-parameters, you are tuning effectively them to the test set; so you will overestimate performance (such as accuracy) that you expect on unseen/production data.
c. The effective size of your test set is much smaller in k-fold CV.