Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Performance Concepts and Metrics
- Latency, throughput, power consumption, and resource utilization.
- Distinction between system-level and model-level bottlenecks.
- Profiling techniques for inference versus training workloads.
Profiling on Huawei Ascend
- Utilizing CANN Profiler and MindInsight.
- Kernel and operator diagnostics.
- Offload patterns and memory mapping strategies.
Profiling on Biren GPU
- Performance monitoring features within the Biren SDK.
- Kernel fusion, memory alignment, and execution queues.
- Power and temperature-aware profiling.
Profiling on Cambricon MLU
- BANGPy and Neuware performance tools.
- Kernel-level visibility and log interpretation.
- Integration of the MLU profiler with deployment frameworks.
Graph and Model-Level Optimization
- Graph pruning and quantization strategies.
- Operator fusion and computational graph restructuring.
- Input size standardization and batch tuning.
Memory and Kernel Optimization
- Optimizing memory layout and reuse.
- Efficient buffer management across different chipsets.
- Platform-specific kernel-level tuning techniques.
Cross-Platform Best Practices
- Performance portability: abstraction strategies.
- Establishing shared tuning pipelines for multi-chip environments.
- Case study: Tuning an object detection model across Ascend, Biren, and MLU.
Summary and Next Steps
Requirements
- Experience with AI model training or deployment pipelines.
- Understanding of GPU/MLU computing principles and model optimization techniques.
- Basic familiarity with performance profiling tools and associated metrics.
Target Audience
- Performance engineers.
- Machine learning infrastructure teams.
- AI system architects.
21 Hours
