8 of the 30 questions in this set, with the correct answer marked and every option explained.
1. What is the primary function of an activation function in a neural network?
✗To initialize the weights of the network.
Weight initialisation is a separate concern, handled by schemes such as Xavier or He initialisation before training starts.
✓To introduce non-linearity into the network.
Activation functions introduce non-linearity into the neural network, allowing it to learn complex patterns. Without activation functions, the network would behave like a linear regression model, regardless of the number of layers.
✗To compute the loss during training.
That is the loss function's job. The activation function transforms a neuron's output; the loss compares the network's final prediction against the target.
✗To prevent overfitting by regularization.
Regularisation comes from dropout, weight decay or early stopping. Without a non-linear activation, stacking layers would collapse into a single linear transformation no matter how deep the network.
2. Which of the following neural network architectures is specifically designed for sequential data processing?
✗Radial Basis Function Networks
Use distance from learned centres as their activation, typically for function approximation over fixed-size inputs. They carry no state between inputs.
✓Recurrent Neural Networks (RNN)
Recurrent Neural Networks (RNNs) are designed to handle sequential data by maintaining a 'memory' of previous inputs through their recurrent connections. This makes them suitable for tasks like language modeling and time series prediction.
✗Feedforward Neural Networks
Information moves in one direction with no cycles, so the network has no memory of what it saw previously. Each input is processed in isolation.
✗Convolutional Neural Networks (CNN)
Built for grid-structured data using spatially local filters. 1D convolutions can in fact process sequences and are used for that, but a CNN has a fixed receptive field rather than the recurrent state that lets an RNN carry information across arbitrary time steps.
3. What is 'backpropagation' in the context of training neural networks?
✗A method for initializing the weights of a neural network.
Initialisation sets the starting weights. Backpropagation is what updates them afterwards, by telling the optimiser which direction reduces the loss.
✗A technique for reducing the dimensionality of input data.
Dimensionality reduction is the job of PCA or an autoencoder. Backpropagation changes weights, not the shape of the input.
✗A regularization technique to prevent overfitting.
Backpropagation is neutral on overfitting — it computes gradients faithfully whether the model is generalising well or memorising.
✓An algorithm for computing the gradient of the loss function with respect to the weights.
Backpropagation is an algorithm used to compute the gradient of the loss function with respect to each weight in the network, allowing for efficient updating of the weights during training.
4. Which optimization algorithm adapts the learning rate for each parameter individually by keeping track of an exponentially decaying average of past squared gradients?
✗Momentum
Accumulates an exponentially decaying average of past gradients themselves, not their squares, to build velocity through flat regions. It accelerates descent but keeps one global learning rate.
✗Stochastic Gradient Descent (SGD)
Applies a single learning rate to every parameter and adapts nothing. It is the baseline the adaptive methods improve on.
✗Adagrad
Very close, and it does adapt the rate per parameter using squared gradients. The difference is that Adagrad accumulates the running sum of all past squared gradients, so the denominator only ever grows and the learning rate decays monotonically until learning stalls. RMSProp uses a decaying average instead, which lets the rate recover.
✓RMSProp
RMSProp is an optimization algorithm that adjusts the learning rate for each parameter by dividing the learning rate by an exponentially decaying average of squared gradients. This helps in dealing with non-stationary objectives and speeds up convergence.
5. In Convolutional Neural Networks (CNNs), what is the purpose of the pooling layer?
✓To reduce the spatial dimensions of the input volume.
Pooling layers reduce the spatial dimensions (width and height) of the input volume, which helps in reducing computation and controlling overfitting. Common pooling operations include max pooling and average pooling.
✗To normalize the input data.
Normalisation is handled by batch normalisation or by preprocessing the inputs. Pooling changes resolution, not scale.
✗To convolve the input with learned filters.
That is the convolutional layer. Pooling has no learnable parameters at all — it simply aggregates over a window.
✗To apply non-linear activation functions.
Max pooling is technically a non-linear operation, which makes this tempting, but its purpose is downsampling. Non-linearity is supplied by activation functions such as ReLU.
6. Which activation function is defined as f(x) = max(0, x) and is commonly used in deep neural networks?
✗Hyperbolic Tangent (tanh) Function
An S-shaped curve squashing inputs into the range -1 to 1. It saturates at both ends, which is exactly the vanishing-gradient problem ReLU was introduced to avoid.
✓Rectified Linear Unit (ReLU)
The Rectified Linear Unit (ReLU) activation function is defined as f(x) = max(0, x). It introduces non-linearity and helps mitigate the vanishing gradient problem, making it popular in deep learning architectures.
✗Sigmoid Function
Squashes inputs into 0 to 1 following 1/(1+e^-x). Also saturating, and its outputs are not zero-centred.
✗Softmax Function
Normalises a whole vector into a probability distribution summing to one. It operates across a layer's outputs rather than element-wise, and is used at the output of a multi-class classifier.
7. What is the main purpose of applying dropout in training neural networks?
✗To improve the model's accuracy on the training data.
Dropout usually makes training accuracy worse, and that is the point. It sacrifices fit on the training set to buy generalisation to unseen data.
✗To increase the training speed by reducing the number of parameters.
No parameters are removed. Units are deactivated randomly on each forward pass but all weights remain and continue to be trained.
✓To prevent overfitting by randomly dropping units during training.
Dropout is a regularization technique that prevents overfitting by randomly setting a fraction of the neurons' outputs to zero during training. This discourages the network from relying too heavily on any one neuron.
✗To adjust the learning rate during training.
That is a learning-rate schedule or an adaptive optimiser. Dropout never touches the learning rate.
8. Which loss function is most appropriate for a binary classification problem?
✗Mean Absolute Error (MAE)
Measures the average absolute distance between predicted and actual numbers. A regression loss, with no notion of predicted probability.
✗Mean Squared Error (MSE)
Also a regression loss. It can technically be applied to classification, but paired with a sigmoid it produces very small gradients when the model is confidently wrong, so learning stalls exactly when correction is most needed.
✗Hinge Loss
A legitimate classification loss, used by support vector machines to maximise the decision margin. It optimises for margin rather than calibrated probabilities, so cross-entropy is the standard choice for a network with a probabilistic output.
✓Cross-Entropy Loss
Cross-Entropy Loss, also known as Binary Cross-Entropy when used for binary classification, measures the performance of a classification model whose output is a probability between 0 and 1.
22 more questions in the app
Practise the full 30-question set with a timer, scoring and progress tracking.