8 of the 60 questions in this set, with the correct answer marked and every option explained.
1. A company wants to assess the performance of a foundation model (FM) for text generation. Which technique or metric will meet these requirements?
✓Recall-Oriented Understudy for Gisting Evaluation (ROUGE)
ROUGE is a metric that you can use to evaluate the quality of text summarization and text generation. You can use ROUGE to assess the performance of an FM for text generation.
✗F1 score
F1 balances precision and recall for classification, where an answer is right or wrong. Generated text has no single correct output, so F1 does not apply.
✗Reinforcement learning
A training technique, not an evaluation metric. RLHF shapes model behaviour during training; it does not measure output quality afterwards.
✗Fine-tuning
Also a training technique. Fine-tuning adapts a model to a task; assessing how well it performs is a separate step.
2. What is a foundation model (FM) in the context of generative AI?
✗A basic architecture that serves as a starting point to design more complex neural networks.
This describes a base architecture such as a transformer block. A foundation model is a trained model, not a design pattern.
✗A theoretical framework to understand how different types of models learn representations.
Foundation models are concrete pre-trained artefacts you can call, not a conceptual framework.
✗A task-specific model that is trained on a narrow domain, such as finance or medicine, to serve as a foundation in that area.
The opposite of a foundation model. Breadth is the defining property — narrow domain models are what you get after fine-tuning one.
✓A large, general-purpose model that is pre-trained on diverse datasets that can be fine-tuned for downstream tasks.
FMs are large models that are pre-trained on a vast amount of data and that can perform several tasks. FMs can be fine-tuned for downstream tasks by using smaller datasets.
3. A company is building a generative AI application by using a foundation model (FM). The company decides to customize its own FM by using proprietary datasets instead of using a pre-trained FM. What are the tradeoffs of customizing the FM? (Select TWO.)
✗Increased risk of hallucination
Retrieval-augmented generation typically reduces hallucination, because the model answers from retrieved source material rather than from parameters alone.
✓Higher implementation complexity
Re-training an FM requires a higher budget than using the public pre-trained FM. It also involves more implementation complexity, requiring skilled ML engineers for data preparation, model re-training, and evaluation.
✓Higher cost
✗Higher latency
RAG does add a retrieval step, but the question asks for the drawbacks that most define the trade-off; latency is usually modest relative to generation time.
✗Reduced accuracy
Grounding responses in retrieved documents generally improves factual accuracy, which is the main reason to adopt RAG.
4. A company wants to record API calls that are made to Amazon Bedrock. For compliance purposes, the company wants these logs to include the API call, the user who made the call, and the time that the call was made. Which AWS service will meet these requirements?
✗Amazon Inspector
Inspector scans workloads for software vulnerabilities and unintended network exposure. It does not record API activity.
✗AWS Trusted Advisor
Trusted Advisor gives cost, performance and security recommendations against best practices. It is advisory, not an audit log.
✓AWS CloudTrail
AWS CloudTrail records API calls, including the user who made the call and the time. This helps ensure compliance and auditability.
✗Amazon CloudWatch
CloudWatch collects metrics and application logs. Who called which API, when, and from where is CloudTrail's job.
5. What is a valid data format for instruction-based fine-tuning?
✗Audio files with transcriptions
That is training data for speech recognition. Instruction fine-tuning of a text model needs text in, text out.
✓Prompt-response text pairs
Instruction-based fine-tuning uses labeled examples formatted as prompt-response pairs to improve the performance of a pre-trained foundation model on domain-specific tasks.
✗Images that are labeled with categories
Labelled images train a vision classifier, not an instruction-following language model.
✗Playlists that are curated with recommended music
Interaction data for a recommender system. It carries no instruction-and-response structure.
6. A company wants to use generative AI to create product descriptions on its website. What is a limitation of generative AI that the company should be aware of?
✓Generative AI models might produce biased or inappropriate content that requires human review and editing.
Generative AI models can produce human-like text but may also generate biased or inappropriate content, which requires human oversight and editing.
✗Generative AI cannot generate text in the multiple languages that is required for an ecommerce website.
Modern foundation models handle many languages well. This is not a real limitation.
✗Generative AI models lack the ability to understand and incorporate product specifications and details.
Specifications can be supplied in the prompt or retrieved via RAG, so the model can absolutely incorporate them.
✗Generative AI cannot handle the large volumes of data that is required for product descriptions.
Volume is a throughput and cost question, readily solved by batching. It is not a capability limit.
7. A travel company wants to use a pre-trained generative AI model to generate background images for marketing materials. The company does not have ML expertise. Additionally, the company does not want to customize and host the ML model. Which AWS service will meet these requirements?
✗Amazon Rekognition
Rekognition analyses existing images — detecting objects, faces and text. It does not generate new ones.
✓Amazon Bedrock
Amazon Bedrock is a fully managed service that provides a unified API to access popular foundation models without the need to train, host, or manage ML models.
✗Amazon SageMaker JumpStart
JumpStart can deploy image generation models, but it hands you infrastructure to manage. Bedrock offers the same models as a serverless API, which fits 'pre-trained model, minimal effort'.
✗Amazon Personalize
Personalize builds recommendation systems from user interaction data. It has no image generation capability.
8. A data scientist notices that a model has high accuracy on training data, but has low accuracy on testing data. What is causing these results?
✓Overfitting
Overfitting occurs when a model learns the training data too well, causing it to perform poorly on new data. This explains why the model has high training accuracy but low testing accuracy.
✗Underfitting
Underfitting shows as poor accuracy on training data too. Here training accuracy is high, so the model has learned — just not generalisably.
✗Not enough training time
Insufficient training produces underfitting. This model has arguably trained too long on the same data.
✗Too much training data
More data almost always improves generalisation. Overfitting is usually a symptom of too little data relative to model capacity.
52 more questions in the app
Practise the full 60-question set with a timer, scoring and progress tracking.