8 of the 15 questions in this set, with the correct answer marked and every option explained.
1. A 70-billion-parameter model will not fit in a single GPU's memory. The team splits individual weight matrices across four GPUs so that each device computes a slice of every layer, exchanging partial results at each step. Which parallelism strategy is this?
✓Tensor parallelism
Tensor parallelism partitions the weight matrices within each layer across devices, so every GPU participates in every layer and they exchange partial activations. It is the standard way to fit a model whose individual layers are too large for one device.
✗Data parallelism
Data parallelism replicates the *entire* model on every GPU and splits the batch. It does nothing for a model that will not fit on one device, because each replica still needs the full parameter set.
✗Pipeline parallelism
Pipeline parallelism assigns different *layers* to different GPUs and passes activations along the chain. That also splits a large model, but the description here — slicing within each layer, with every GPU computing a slice of every layer — is tensor parallelism specifically.
✗Gradient accumulation
Gradient accumulation simulates a larger batch across sequential micro-batches on the same hardware. It reduces memory pressure from batch size, not from parameter count, and involves no splitting across devices.
2. Training runs out of GPU memory during the backward pass. The team wants to trade compute time for memory rather than reduce model size. Which technique fits?
✓Activation checkpointing — discard intermediate activations during the forward pass and recompute them when needed during the backward pass.
This is precisely a compute-for-memory trade. Storing every intermediate activation for the backward pass is often the dominant memory cost; recomputing them costs extra forward work but can cut activation memory dramatically.
✗Increasing the batch size to improve GPU utilization.
A larger batch increases activation memory proportionally, making the out-of-memory failure worse rather than better.
✗Switching the optimizer from Adam to a higher-momentum variant.
Optimizer choice affects convergence and optimizer-state memory, but simply choosing a different momentum setting does not systematically trade compute for memory the way checkpointing does. Moving to a lower-state optimizer would reduce memory, but that is not what 'higher momentum' means.
✗Disabling gradient computation for all layers.
That stops training altogether — with no gradients there is nothing to update. It solves the memory error by not training.
3. A team needs to adapt a large base model to a specialized domain but cannot afford to update and store a full copy of all parameters for each of their twelve customers. Which approach best fits?
✓Parameter-efficient fine-tuning such as LoRA, training small low-rank adapter matrices while the base weights stay frozen.
LoRA-style adapters train a tiny fraction of the parameters and produce small per-customer artefacts that layer onto one shared frozen base model. Twelve adapters cost far less to train and store than twelve full fine-tunes, which is exactly the constraint given.
✗Full fine-tuning of all parameters, once per customer.
This is the option the constraint explicitly rules out — twelve full copies of a large model is precisely the storage and compute burden being avoided.
✗Continued pretraining on the full original corpus plus customer data.
Continued pretraining is even more expensive than fine-tuning and still yields a full model copy per customer. It is aimed at broad domain shift, not efficient per-tenant specialization.
✗Increasing the context window so customer data can be pasted into every prompt.
This is in-context learning rather than fine-tuning. It can work for small amounts of context but costs tokens on every single request, does not scale to substantial domain knowledge, and is a different technique from what is being asked about.
4. After fine-tuning on a narrow domain dataset, a model performs well on the new domain but noticeably worse on general tasks it previously handled. What is this phenomenon called, and what is a standard mitigation?
✓Catastrophic forgetting; mitigate by mixing general-domain data into the fine-tuning set or by using parameter-efficient methods that leave base weights intact.
Catastrophic forgetting is the loss of previously learned capability when weights are pushed hard toward a new distribution. Replaying general data during fine-tuning, or freezing the base and training adapters, are the two standard mitigations.
✗Overfitting; mitigate by increasing the learning rate.
The name is wrong and the mitigation is backwards. Overfitting means memorizing the training set and failing to generalize *within* the task; raising the learning rate would worsen instability, not fix it.
✗Gradient explosion; mitigate by adding more layers.
Gradient explosion is a numerical instability during training that manifests as diverging loss, not as degraded general performance after a successful run. Adding layers does not address it — gradient clipping does.
✗Quantization error; mitigate by retraining in higher precision.
Quantization error comes from reducing numeric precision of weights. Nothing in the scenario mentions quantization, and the degradation described is domain-specific rather than a uniform precision loss.
5. A model gets multi-step arithmetic and logic questions wrong when asked for a bare answer. Which prompting change most directly addresses this?
✓Chain-of-thought prompting — instruct the model to work through intermediate reasoning steps before giving the final answer.
Forcing intermediate steps gives the model tokens in which to carry out the computation rather than having to produce the answer in a single step. This measurably improves multi-step arithmetic and logical reasoning, and is named explicitly in the blueprint.
✗Reducing the temperature to zero and nothing else.
Lower temperature makes output more deterministic and repeatable, but a deterministic wrong answer is still wrong. It does not give the model room to reason.
✗Shortening the prompt to reduce token count.
Brevity is good practice generally but does not help a reasoning failure. Here the model needs *more* space to reason, not less.
✗Asking the same question repeatedly in one prompt.
Repetition adds no new information or reasoning structure. Self-consistency — sampling several independent chains and taking the majority — is a real technique, but that is not what naive repetition in a single prompt does.
6. A downstream service must parse an LLM's output programmatically, but occasionally the model returns prose around the data. Which approach most reliably solves this?
✓Constrain output format explicitly — request a defined schema and use structured output or function-calling features that enforce it, rather than relying on the prompt alone.
Enforced structured output constrains decoding so the response conforms to the schema by construction. Prompt instructions alone are probabilistic and will occasionally be ignored, which is exactly the intermittent failure described.
✗Raise the temperature so the model explores more formats and eventually finds the right one.
Higher temperature increases variability, making format deviation more frequent, not less.
✗Post-process with a regular expression and accept occasional failures.
This is a mitigation rather than a fix, and a brittle one — regex parsing of free-form LLM output fails in ways that are hard to anticipate. Constrain generation first, then parse.
✗Increase the maximum output token limit.
A higher limit prevents truncation but has no bearing on whether the model wraps its answer in prose.
7. Why does the key-value (KV) cache matter for transformer inference performance?
✓It stores the key and value tensors already computed for previous tokens so each new token attends over them without recomputing the whole sequence, turning generation from quadratic to roughly linear per-token work.
Without caching, generating token n would require recomputing attention keys and values for all preceding tokens every step. The KV cache makes autoregressive decoding practical, at the cost of memory that grows with sequence length and batch size — which is why KV cache size is a central serving constraint.
✗It caches the model weights in GPU memory so they need not be re-read from disk each request.
Weights are loaded once into GPU memory at startup; that is not what the KV cache holds. The KV cache holds per-request, per-token attention state.
✗It stores previous users' responses so identical prompts can be answered without inference.
That describes a response or prompt cache, a different optimization operating at the application layer. The KV cache is internal to a single generation.
✗It compresses the vocabulary embedding table to reduce model size.
Embedding table size is a static property of the model. The KV cache is dynamic per-request state and does not alter the embeddings.
8. A team quantizes a model from FP16 to INT8 for serving. Which statement best describes the trade-off?
✓Memory footprint and bandwidth requirements drop and throughput usually rises, at the cost of some numerical precision that may slightly degrade output quality and must be validated.
That is the honest characterization. Lower precision means smaller weights, less memory traffic and faster arithmetic on hardware with INT8 support, but quantization introduces error. Whether that error is acceptable is an empirical question, which is why post-quantization evaluation is mandatory rather than optional.
✗Quality is mathematically guaranteed to be unchanged because quantization is lossless.
Quantization to INT8 is lossy by definition — a continuous range is mapped onto 256 levels. Techniques like calibration and quantization-aware training reduce the damage but do not eliminate it.
✗Memory use falls but throughput always decreases because of dequantization overhead.
On hardware with native INT8 tensor operations, throughput generally improves substantially. Dequantization overhead exists but is normally far outweighed by the gains.
✗Quantization only affects training and has no effect on inference.
Inference is the primary target of quantization. Serving is where the memory and latency benefits are realized.
7 more questions in the app
Practise the full 15-question set with a timer, scoring and progress tracking.