Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models with Docker.
- Exporting models from TensorFlow and PyTorch.
- Considerations for versioning and storage.
Model Serving on Kubernetes
- Overview of inference servers.
- Deploying TensorFlow Serving and TorchServe.
- Setting up model endpoints.
Inference Optimization Techniques
- Batching strategies.
- Handling concurrent requests.
- Tuning for latency and throughput.
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA).
- Vertical Pod Autoscaler (VPA).
- Kubernetes Event-Driven Autoscaling (KEDA).
GPU Provisioning and Resource Management
- Configuring GPU nodes.
- Overview of the NVIDIA device plugin.
- Setting resource requests and limits for ML workloads.
Model Rollout and Release Strategies
- Blue/green deployments.
- Canary rollout patterns.
- A/B testing for model evaluation.
Monitoring and Observability for ML in Production
- Metrics for inference workloads.
- Logging and tracing practices.
- Dashboards and alerting.
Security and Reliability Considerations
- Securing model endpoints.
- Network policies and access control.
- Ensuring high availability.
Summary and Next Steps
Requirements
- A solid understanding of containerized application workflows.
- Hands-on experience with Python-based machine learning models.
- Familiarity with the fundamentals of Kubernetes.
Audience
- ML engineers.
- DevOps engineers.
- Platform engineering teams.
Testimonials (4)
basic understanding of container/kubernetes and how they interact features of the openshift plattform
Eric Scholze - NOW IT GmbH
Course - Introduction to Containers, Kubernetes & OpenShift
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
How trainer deliver knowledge so effectively
Vu Thoai Le - Reply Polska sp. z o. o.
Course - Certified Kubernetes Administrator (CKA) - exam preparation
The knowledge and exchanges with Augustin