Top 30 Deep Learning Interview Questions and Answers
This set of 30 tricky deep learning interview questions covers neural networks, activation functions, backpropagation, optimization, regularization, CNNs, self-attention, transformers, embeddings, loss functions, and model evaluation.
30 Tricky Deep Learning Interview Questions
1. What is the primary reason for using non-linear activation functions in deep neural networks?
- A. To make the network deeper
- B. To enable the network to learn complex patterns
- C. To reduce the number of parameters
- D. To guarantee faster convergence
Correct Answer: B
Non-linear activation functions like ReLU or tanh allow neural networks to learn complex, non-linear mappings between inputs and outputs. Linear activations restrict the network to linear combinations, regardless of depth.
Why others are incorrect: A is wrong because depth doesn't require non-linearity; C is unrelated; D is not guaranteed by non-linearity.
2. Given the following code, what will be the output?
import torch
x = torch.tensor([-2.0, 0.0, 2.0])
y = torch.relu(x)
print(y)
- A. tensor([0., 0., 0.])
- B. tensor([-2., 0., 2.])
- C. tensor([0., 0., 2.])
- D. tensor([-2., 0., 0.])
Correct Answer: C
ReLU(x) returns max(0, x) elementwise. So, [-2, 0, 2] maps to [0, 0, 2].
3. Which of the following best describes the vanishing gradient problem?
- A. Gradients become excessively large during backpropagation
- B. Gradients become very small, causing slow or stalled learning
- C. Gradients are always zero
- D. Gradients alternate in sign
Correct Answer: B
In deep networks, repeated multiplication by small derivatives (from, e.g., sigmoid or tanh) leads to gradients approaching zero, stalling learning in early layers.
4. What will be the output of the following code?
import torch.nn.functional as F
import torch
x = torch.tensor([[-1.0, 0.0, 1.0]])
y = F.softmax(x, dim=1)
print(y)
- A. tensor([[0.0, 0.0, 1.0]])
- B. tensor([[0.0, 0.5, 0.5]])
- C. tensor([[0.0900, 0.2447, 0.6652]])
- D. tensor([[0.2689, 0.7311, 0.0000]])
Correct Answer: C
Softmax computes \(\frac{e^x}{\sum e^x}\) for each element:
- \(e^{-1} ≈ 0.3679\)
- \(e^{0} = 1\)
- \(e^{1} ≈ 2.7183\)
- Sum: 0.3679 + 1 + 2.7183 ≈ 4.0862
- So, [0.3679/4.0862, 1/4.0862, 2.7183/4.0862] ≈ [0.0900, 0.2447, 0.6652]
5. What is the main benefit of using dropout during training?
- A. It increases the model size
- B. It reduces overfitting by randomly deactivating neurons
- C. It speeds up inference
- D. It guarantees 100% accuracy
Correct Answer: B
Dropout randomly disables units during training, preventing co-adaptation and reducing overfitting.
6. In convolutional neural networks, what is the primary effect of increasing the stride in a convolutional layer?
- A. Increases output spatial dimensions
- B. Reduces output spatial dimensions
- C. Increases number of parameters
- D. Does not affect output dimensions
Correct Answer: B
A higher stride skips positions, shrinking the output size (feature map) spatially.
7. Which of the following is not a commonly used pooling operation in CNNs?
- A. Max pooling
- B. Average pooling
- C. Min pooling
- D. Sum pooling
Correct Answer: D
Sum pooling is rarely used; max and average pooling are standard. Min pooling is less common but does exist.
8. Given a 32x32 input image and a 5x5 convolutional filter, no padding, stride=1, what will be the output feature map size?
- A. 32x32
- B. 28x28
- C. 27x27
- D. 26x26
Correct Answer: B
Output size = \((N - F) / S + 1\) where N=32, F=5, S=1: (32-5)/1 + 1 = 28.
9. What is the main difference between batch normalization and layer normalization?
- A. Batch norm normalizes across features for each sample; layer norm normalizes across samples for each feature
- B. Batch norm normalizes across samples for each feature; layer norm normalizes across features for each sample
- C. Both normalize across the same dimension
- D. Batch norm is only used in CNNs
Correct Answer: B
Batch norm: across batch (samples) for each feature; Layer norm: across features for each sample.
10. What is the effect of L2 regularization (weight decay) on the weights of a neural network?
- A. Encourages weights to be exactly zero
- B. Encourages large weights
- C. Penalizes large weights, pushing them toward zero (but not exactly zero)
- D. Has no effect on weights
Correct Answer: C
L2 adds a penalty proportional to the square of the weights; it shrinks weights but doesn't enforce exact zeros.
11. What is a potential issue when stacking many convolutional layers in a CNN?
- A. Decreased receptive field
- B. Increased parameter sharing
- C. Vanishing/exploding gradients
- D. Reduced expressiveness
Correct Answer: C
Deeper networks can suffer vanishing or exploding gradients, making training unstable.
12. You have a classification dataset with highly imbalanced classes. Which loss function is most suitable?
- A. Mean Squared Error (MSE)
- B. Unweighted Cross-Entropy Loss
- C. Weighted Cross-Entropy Loss
- D. Hinge Loss
Correct Answer: C
Weighted cross-entropy helps by upweighting rare classes, handling imbalance.
13. Consider the following code. What is the output?
import torch
import torch.nn as nn
m = nn.Embedding(4, 2)
input = torch.LongTensor([1, 2, 1, 3])
print(m(input))
- A. 4x4 tensor
- B. 4x2 tensor
- C. 2x4 tensor
- D. 1x4 tensor
Correct Answer: B
Embedding(4,2) maps each integer (0-3) to a 2D vector. Four indices in input yield a 4x2 output.
14. What is the main role of the softmax function in a neural network?
- A. To produce binary outputs
- B. To convert logits to probabilities that sum to 1
- C. To increase non-linearity
- D. To reduce overfitting
Correct Answer: B
Softmax turns logits into a probability distribution over classes.
15. Which of the following is a misconception about the use of ReLU activation?
- A. ReLU is computationally efficient
- B. ReLU can cause dying neurons
- C. ReLU always solves the vanishing gradient problem completely
- D. ReLU outputs are zero for negative inputs
Correct Answer: C
ReLU alleviates but does not entirely solve vanishing gradients; it can still suffer from dying neurons.
16. In backpropagation, which variable is updated during training?
- A. Input data
- B. Output predictions
- C. Model weights/parameters
- D. Loss function
Correct Answer: C
Weights/parameters are updated using gradients computed by backpropagation.
17. What is the key advantage of using residual connections in deep networks?
- A. They decrease the number of layers
- B. They allow gradients to flow better, enabling training of very deep networks
- C. They increase non-linearity
- D. They decrease model accuracy
Correct Answer: B
Residual (skip) connections help gradients propagate, mitigating vanishing gradients and enabling deeper networks.
18. What is the main purpose of attention mechanisms in deep learning models?
- A. To increase model size
- B. To focus on relevant parts of the input when producing each output
- C. To pool features spatially
- D. To reduce the number of parameters
Correct Answer: B
Attention mechanisms assign weights to input parts, allowing the model to concentrate on relevant information dynamically.
19. In the transformer architecture, what is the role of positional encoding?
- A. To make the model non-linear
- B. To provide information about the order of sequence elements
- C. To normalize activations
- D. To share parameters across layers
Correct Answer: B
Transformers lack recurrence; positional encoding injects order information into the sequence.
20. Which optimizer adapts the learning rate for each parameter and accumulates squared gradients?
- A. SGD (Stochastic Gradient Descent)
- B. Adam
- C. Adadelta
- D. RMSprop
Correct Answer: B
Adam combines momentum and adaptive learning rates (using squared gradients).
21. Which of the following is a trade-off of using a larger batch size during training?
- A. Always improves model generalization
- B. Reduces computation per epoch
- C. Provides more stable gradient estimates but may reduce generalization
- D. Makes training slower
Correct Answer: C
Larger batch sizes give more accurate gradients but may lead to worse generalization compared to smaller batches.
22. Predict the output of the following code snippet:
import torch
import torch.nn.functional as F
x = torch.tensor([[1.0, 2.0, 3.0]])
y = F.layer_norm(x, x.shape)
print(y)
- A. tensor([[0., 0., 0.]])
- B. tensor([[-1.2247, 0.0, 1.2247]])
- C. tensor([[1.0, 2.0, 3.0]])
- D. tensor([[0.5, 0.5, 0.5]])
Correct Answer: B
Layer norm normalizes to zero mean and unit variance:
- Mean = 2.0
- Std = sqrt(((1-2)^2 + (2-2)^2 + (3-2)^2)/3) = sqrt((1+0+1)/3) = sqrt(2/3) ≈ 0.8165
- So: [(1-2)/0.8165, (2-2)/0.8165, (3-2)/0.8165] ≈ [-1.2247, 0, 1.2247]
23. Which technique can help a model learn better representations for rare words in NLP tasks?
- A. One-hot encoding
- B. Static word embeddings
- C. Subword/token embeddings (e.g., Byte Pair Encoding)
- D
23. Which technique can help a model learn better representations for rare words in NLP tasks?
- A. One-hot encoding
- B. Static word embeddings
- C. Subword/token embeddings (e.g., Byte Pair Encoding)
- D. Dropout regularization
Subword embedding techniques like Byte Pair Encoding (BPE) or WordPiece break rare words into smaller, more frequent subword units, allowing the model to compose representations for unseen or rare words based on known components.
Why others are incorrect: One-hot encoding cannot generalize to rare or unseen words. Static embeddings (e.g., word2vec) treat each word as atomic, so rare words are still poorly represented. Dropout is a regularization technique, not an embedding strategy.
24. In multi-class classification with softmax, what is the cross-entropy loss formula for a single sample with true class \( y \) and predicted probabilities \( \mathbf{p} \)?
- A. \( -\log(p_y) \)
- B. \( \sum_{i} (y_i - p_i)^2 \)
- C. \( \prod_{i} p_i \)
- D. \( \max(p_i) \)
For a single sample, cross-entropy loss is \( -\log(p_y) \), where \( p_y \) is the predicted probability for the true class \( y \).
Why others are incorrect: B is the Mean Squared Error. C and D do not represent loss functions for classification.
25. Which of the following can help alleviate overfitting in deep neural networks?
- A. Using a very large model
- B. Reducing the amount of training data
- C. L1/L2 regularization and data augmentation
- D. Training for more epochs without early stopping
L1/L2 regularization penalizes large weights, and data augmentation increases the effective size and diversity of the dataset, both helping to prevent overfitting.
Why others are incorrect: A increases risk of overfitting; B reduces generalization; D can worsen overfitting.
26. Scenario: You train a deep network and notice the training accuracy is high, but validation accuracy is much lower. What is the most likely cause?
- A. Underfitting
- B. Overfitting
- C. Proper generalization
- D. Good regularization
High train but low validation accuracy is a classic sign of overfitting.
Why others are incorrect: Underfitting would show low accuracy on both; C and D do not match the scenario.
27. What problem does the “label smoothing” technique address in classification tasks?
- A. Makes the model output exactly one-hot probabilities
- B. Helps prevent the model from becoming overconfident in its predictions
- C. Forces the model to always predict the majority class
- D. Decreases the learning rate
Label smoothing replaces the hard 0 and 1 targets with softer values (e.g., 0.9 for the true class, 0.1 distributed among others), reducing overconfidence and often improving generalization.
Why others are incorrect: A is the opposite of smoothing; C and D are unrelated.
28. Consider the following code. Which pooling strategy is used?
import torch import torch.nn.functional as F input = torch.tensor([[1., 3., 2., 4.]]) output = F.max_pool1d(input.unsqueeze(0), kernel_size=2, stride=2) print(output)- A. Average pooling
- B. Max pooling
- C. Global pooling
- D. Min pooling
F.max_pool1dperforms max pooling; with kernel size 2 and stride 2, the output takes the max of each non-overlapping window: max([1,3]) = 3, max([2,4]) = 4. Output: [[3, 4]].
29. Scenario: You are using Adam optimizer, but your validation loss oscillates wildly. What adjustment might help?
- A. Increase the learning rate
- B. Decrease the learning rate
- C. Switch to SGD without momentum
- D. Remove weight decay
A high learning rate can cause unstable training and oscillating loss. Lowering it often stabilizes convergence.
Why others are incorrect: Increasing the learning rate can worsen the oscillation. C and D are less directly related to the symptom.
30. Which metric is most appropriate for evaluating a binary classifier on a highly imbalanced dataset?
- A. Accuracy
- B. Precision, Recall, or F1-score
- C. Mean squared error
- D. Top-5 accuracy
Precision, recall, or F1-score are more informative for imbalanced data, as accuracy can be misleading (e.g., always predicting the majority class yields high accuracy but poor minority-class performance).
Why others are incorrect: Accuracy ignores class imbalance; MSE is for regression; Top-5 accuracy is for multi-class tasks.
Summary Table: Key Concepts & Trade-offs
Concept Core Idea Common Pitfall / Trade-off Activation Functions Introduce non-linearity Vanishing gradients (sigmoid/tanh), dying neurons (ReLU) Regularization Reduces overfitting Too much can underfit Batch Size Larger = stable gradients Can hurt generalization Optimizers Adapt learning rates Adam can overfit, SGD slower but may generalize better Model Depth Enables complex representations Vanishing/exploding gradients Attention/Transformers Dynamic focus, parallelism High computation/memory Evaluation Metrics Match metric to data/task Accuracy fails on imbalance
Conclusion
Mastering deep learning interview questions requires not just rote memorization, but a deep conceptual understanding and the ability to reason through code and real-world scenarios. Use these 30 tricky questions to test your knowledge, identify gaps, and gain confidence for your next interview. Keep practicing with code, explore edge cases, and always ask: “Why does this work?” and “What happens if I change this?”—that’s how you’ll excel as a deep learning practitioner.
