AWS Certified Machine Learning Engineer Associate Training

Who Should Attend

• ML engineers and MLOps engineers.
• Data scientists who deploy their own models.
• Developers and data engineers adding ML to their work.
• AI Practitioner holders moving into hands-on ML.

Prerequisites

• AWS doesn't require another certification first, and we don't have enrollment prerequisites either.
• AWS recommends at least 1 year of experience with Amazon SageMaker AI, Amazon Bedrock and other ML engineering services.

Domain 1: Data Preparation for ML and AI (28%)

Task 1.1: Collect and store data

• Extract data from data sources (for example, Amazon S3, Amazon EBS, Amazon EFS, Amazon RDS, Amazon DynamoDB, Amazon OpenSearch Service).
• Make storage decisions and configure storage services based on cost, performance, data structure, and data compliance.
• Troubleshoot and debug data ingestion and storage issues that involve capacity and scalability.
• Use AWS streaming data sources to ingest data (for example, by using Amazon Kinesis, Apache Flink, Apache Kafka).
• Ingest from and write by using appropriate data formats (for example, Apache Parquet, JSON, CSV, ORC) based on data access patterns.
• Merge data from multiple sources (for example, by using programming techniques, AWS Glue, Apache Spark).
• Configure scalable vector databases for AI applications (for example, OpenSearch Service, Amazon RDS with pgvector, Amazon S3) based on specifications.
• Ingest and store diverse data types (for example, text, images, audio) for AI and ML applications.
• Ingest data into SageMaker Feature Store.

Task 1.2: Perform data transformation, feature engineering, and pre-processing

• Transform data by using AWS tools (for example, AWS Glue, AWS Glue DataBrew, Spark on Amazon EMR, SageMaker Data Wrangler).
• Create and manage features by using AWS tools (for example, SageMaker Feature Store).
• Transform streaming data (for example, by using AWS Lambda, Spark).
• Perform feature engineering (for example, scaling, standardization, feature splitting, binning, log transformation, normalization).
• Configure and use embedding models to transform text and image data into numerical representations.
• Apply advanced text pre-processing techniques (for example, tokenization, domain-specific augmentation).
• Prepare documents for Retrieval Augmented Generation (RAG) applications (for example, chunking strategies, metadata extraction).
• Mask, redact, and anonymize data.
• Prepare data for FM fine-tuning, continuous pre-training, and model distillation.

Task 1.3: Validate data quality and manage bias

• Validate data quality (for example, by using DataBrew, AWS Glue Data Quality).
• Label and annotate data.
• Identify and mitigate sources of bias in data by using AWS tools and techniques (for example, dataset splitting, shuffling, augmentation).
• Optimize multimodal data distributions by applying bias metrics across numeric, text, and image assets.
• Resolve class imbalance in numeric, text, and image datasets.
• Validate AI training data integrity (for example, prompt-response pair validation, content safety screening).
• Clean data (for example, by detecting outliers, imputing missing data, deduplication).

Domain 2: ML Model and Foundation Model (FM) Development (24%)

Task 2.1: Choose appropriate modeling approaches for ML and AI solutions

• Evaluate and select appropriate FMs from Amazon Bedrock based on task requirements and performance criteria.
• Identify fine-tuning strategies for pre-trained FMs to meet business needs.
• Compare and select appropriate ML models, generative AI (GenAI) models, algorithms, and solution templates to meet business needs (for example, interpretability, domain-specific performance, latency).
• Evaluate tradeoffs between custom solutions, managed services, pre-trained models, and FMs to meet business needs.
• Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements.
• Assess tradeoffs between ML model performance, training time, and cost.
• Assess tradeoffs between AI model performance, latency, and cost.
• Apply AWS AI services to solve specific business problems (for example, Amazon Textract, Amazon Rekognition, Amazon Comprehend, Amazon Transcribe).

Task 2.2: Train, fine-tune, and customize models for ML and AI solutions

• Apply Amazon SageMaker AI built-in algorithms and common ML libraries.
• Configure SageMaker AI script mode with supported frameworks for simplicity and performance.
• Implement hyperparameter optimization (for example, SageMaker AI automatic model tuning [AMT]).
• Implement training time reduction techniques (for example, early stopping, distributed training).
• Prevent model overfitting, underfitting, and catastrophic forgetting.
• Combine multiple ML models to improve performance or reduce cost.
• Adjust fundamental hyperparameters (for example, epoch, steps, batch size).
• Apply customization techniques for AI solutions (for example, task-specific prompt engineering, fine-tuning).
• Optimize retrieval components and embedding models.

Task 2.3: Analyze and evaluate the performance of ML and AI systems

• Perform reproducible experiments (for example, by using MLflow on SageMaker AI, Amazon Bedrock evaluations, Amazon Bedrock Prompt Management).
• Create model performance baselines and implement drift detection.
• Compare the performance of shadow variants to production variants.
• Explain model outputs.
• Debug model convergence issues.
• Apply comprehensive model evaluation techniques for traditional ML and GenAI models.
• Implement integrated human evaluation frameworks (for example, human-in-the-loop workflows, text generation quality assessment).
• Apply natural language processing (NLP) evaluation metrics (for example, bilingual evaluation understudy [BLEU], Recall-Oriented Understudy for Gisting Evaluation [ROUGE], BERTScore, semantic similarity).
• Perform AI evaluation (for example, model output assessment, content quality validation, bias detection, LLM-as-a-judge frameworks).
• Configure RAG system monitoring, including retrieval accuracy assessment.

Domain 3: Deployment and Orchestration of ML and AI Workflows (24%)

Task 3.1: Manage deployment infrastructure for ML and AI model types

• Select appropriate compute environments and deployment targets.
• Select deployment orchestrators and multi-model or multi-container deployment strategies.
• Select model inference strategies (for example, real-time and batch processing).
• Evaluate and select appropriate foundation model (FM) deployment options.
• Deploy models that were built outside of AWS into AWS environments (for example, Amazon SageMaker AI, Amazon Bedrock Custom Model Import).
• Deploy and configure agents for specific tasks, integration with other services and tools, and agent communication protocols.
• Configure FM deployment, model hosting, and resource allocation.
• Apply Retrieval Augmented Generation (RAG) system configurations (for example, retrieval strategies, reranking).

Task 3.2: Provision and configure resources for ML and AI workloads based on existing architecture and requirements

• Optimize resource provisioning between on-demand and provisioned resources for performance and cost efficiency.
• Automate compute resource provisioning with integrated communication between stacks and orchestration services.
• Build and maintain containers for ML and AI workloads.
• Configure SageMaker AI endpoints within VPC network environments.
• Deploy and host models programmatically (for example, by using the SageMaker AI SDK for Python, AWS CLI, Boto3).
• Select specific metrics for auto scaling implementations.
• Create and manage Amazon Bedrock knowledge bases with vector database configurations, document indexing, and retrieval optimization.
• Implement retrieval pipelines to meet business needs.
• Implement agent state management systems.
• Implement AI-specific resource scaling for GPU workloads.
• Deploy agentic workflow infrastructure.

Task 3.3: Implement automated orchestration and continuous integration and continuous delivery (CI/CD) pipelines for MLOps and AI workloads

• Implement automated deployment strategies and rollback actions.
• Configure and troubleshoot AWS CodeBuild, AWS CodeCommit, AWS CodeDeploy, AWS CodePipeline, and AWS CodeConnections.
• Configure training and inference jobs.
• Configure automated testing strategies within CI/CD pipelines for traditional ML and AI workloads.
• Build and integrate mechanisms to re-train models.
• Manage model versions for repeatability and audits (for example, SageMaker Model Registry, MLflow on SageMaker AI).
• Manage prompts (for example, Amazon Bedrock Prompt Management).
• Implement automated agent deployment pipelines and agent version management.
• Implement AI model testing frameworks, including prompt testing.
• Configure FM deployment automation with fine-tuned model versioning.
• Configure AI-specific pipeline orchestration for RAG system updates and knowledge base refresh cycles.

Domain 4: Operating, Monitoring, and Securing ML and AI Solutions (24%)

Task 4.1: Monitor ML and AI model inference and performance

• Monitor model performance in production by using Amazon CloudWatch generative AI observability, Amazon Bedrock Model Evaluation, and drift detection pipelines.
• Monitor workflows to detect anomalies or errors in data processing or model inference.
• Detect changes in data distribution that can affect model performance.
• Monitor model performance in production by using A/B testing.
• Monitor and automate the management of agent performance and coordination (for example, coordination failure detection, truncated streaming, tool failures).
• Configure AI-specific performance monitoring for foundation models (FMs), such as Amazon Bedrock evaluations.

Task 4.2: Optimize and manage ML and AI infrastructure costs and performance

• Select inference instance families to optimize performance and cost.
• Configure and use tools to troubleshoot and analyze resources (for example, Amazon CloudWatch, Amazon Bedrock AgentCore Observability, AWS X-Ray).
• Set up dashboards to monitor performance metrics.
• Optimize capacity for cost, performance, and reliability.
• Optimize costs and set cost quotas by using appropriate cost management tools.
• Optimize infrastructure costs by selecting purchasing options.
• Evaluate cost implications of using FMs for inference in production.
• Monitor agent resource consumption patterns.
• Manage FM inference costs with usage optimization.
• Monitor AI-specific cost patterns (for example, token usage optimization, embedding computation costs, vector database storage optimization).

Task 4.3: Secure ML and AI workloads and model endpoints

• Secure continuous integration and continuous delivery (CI/CD) pipelines by checking for code and image vulnerabilities (for example, by using Amazon CodeGuru, Amazon Inspector).
• Configure least privilege access to ML and AI artifacts.
• Configure IAM policies and roles for users and applications in ML and AI systems.
• Configure comprehensive monitoring, auditing, compliance, and logging for ML and AI systems (for example, AWS CloudTrail, AWS Config).
• Troubleshoot and debug security issues in ML and AI systems.
• Create VPCs, subnets, and security groups to securely isolate ML and AI systems.
• Identify and mitigate security risks and vulnerabilities in ML and AI systems.
• Select the appropriate credential type to access FMs (for example, Amazon Bedrock API keys, IAM credentials).
• Implement safeguards and sensitive data protection to meet application requirements and responsible AI policies (for example, by using Amazon Bedrock Guardrails).