Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Concepts and Metrics
- Latency, throughput, power consumption, and resource utilization
- Distinguishing system-level versus model-level bottlenecks
- Profiling techniques for inference compared to training
Profiling on Huawei Ascend
- Leveraging CANN Profiler and MindInsight
- Diagnosing kernels and operators
- Analyzing offload patterns and memory mapping
Profiling on Biren GPU
- Utilizing Biren SDK performance monitoring capabilities
- Exploring kernel fusion, memory alignment, and execution queues
- Conducting power and temperature-aware profiling
Profiling on Cambricon MLU
- Using BANGPy and Neuware performance tools
- Gaining kernel-level visibility and interpreting logs
- Integrating the MLU profiler with deployment frameworks
Graph and Model-Level Optimization
- Strategies for graph pruning and quantization
- Operator fusion and computational graph restructuring
- Standardizing input sizes and tuning batch parameters
Memory and Kernel Optimization
- Improving memory layout and reuse
- Efficient buffer management across different chipsets
- Platform-specific kernel-level tuning techniques
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies
- Constructing shared tuning pipelines for multi-chip environments
- Case study: tuning an object detection model across Ascend, Biren, and MLU
Summary and Next Steps
Requirements
- Practical experience with AI model training or deployment pipelines
- Knowledge of GPU/MLU compute principles and model optimization techniques
- Familiarity with performance profiling tools and key metrics
Target Audience
- Performance Engineers
- Machine Learning Infrastructure Teams
- AI System Architects
