Course Catalog  > AI and Machine Learning  > AI
WA3387

Tools for Monitoring AI/ML Training

This Artificial Intelligence and Machine Learning (AI/ML) Tools training course gives attendees an in-depth examination of the tools and techniques used in monitoring AI and ML models, focusing on those used in production. Participants learn how to detect and address model drift over time and monitor for data quality, privacy, and security.

Request Information
Request On-Site or Customized Course Info
Course Details

Duration

2 days

Prerequisites

  • Cloud Architecture Basics
  • Machine Learning Fundamentals
  • Data Science Pipelines

Target Audience

  • Data Science DevOps
  • Data Engineers
  • Data Scientists
  • ML Engineers

Skills Gained

  • Understand the importance and types of AI/ML model monitoring
  • Know how to detect anomalies in model behavior
  • Understand the practical applications of anomaly detection in AI/ML monitoring
Course Outline
  • Introduction to Monitoring AI/ML Models
    • Importance of monitoring AI/ML models
    • Key metrics for monitoring AI/ML models
    • Monitoring for model performance vs. monitoring for application performance
    • Monitoring throughout the Data Science pipeline
  • Monitoring Data Quality
    • Understanding data quality issues in AI/ML applications.
    • Tools and techniques for monitoring data quality.
    • How data quality issues affect model performance and strategies to manage this.
  • Detecting and Addressing Model Drift
    • Understanding model drift
    • Techniques for detecting model drift and data drift
    • Tools for drift detection (e.g., AWS SageMaker Model Monitor, Seldon Alibi-Detect)
    • Strategies for addressing model drift
  • Advanced Topics in AI/ML Monitoring
    • Monitoring complex models (e.g., deep learning models)
    • Monitoring at scale: big data considerations
    • Continuous monitoring and automated anomaly detection
  • Monitoring for AI/ML Security
    • Understanding adversarial attacks on AI/ML models.
    • Importance of security monitoring in AI/ML.
    • Tools for monitoring and mitigating adversarial attacks.
  • Privacy, Fairness, and Compliance Considerations
    • How privacy regulations impact AI/ML monitoring.
    • Tools and best practices for privacy-preserving AI/ML monitoring.
    • Case studies in AI/ML privacy and compliance.
    • Understanding model fairness and bias
    • Tools for fairness and bias monitoring (e.g., Fairlearn, Aequitas)
    • Case studies of fairness and bias monitoring
  • Conclusion