EECE695E: Efficient ML Systems (Spring 2024) #
Team #
- Instructor. Jaeho Lee ์ด์ฌํธ
firstname.lastname@postech.ac.kr - TA. (forthcoming)
Location & Time #
- Class. (forthcoming)
- Office Hr. (forthcoming)
What we’ll cover (hopefully) #
- Deep Learning Basics
- Architectures and Counting FLOPs
- Hardware bits
- Making models smaller
- Quantization
- Pruning & Sparsity
- Neural Architecture Search
- How to utilize the experience of other models
- Transfer Learning & Distillation
- Meta-Learning & Test-time Training
- Model Merging & Stitching
- Hyperparameter Transfer
- Prompt Tuning
- Organizing Large-scale Learning
- Parallelism & Pipelining
- Federated Learning
- Data Efficiency
- Data Compression
- Dataset Distillation & Condensation / SeiT
- Tips & Tricks for Large Transformers
- KV cache / FlashAttention / PagedAttention
- Speculative Decoding / Medusa
- Continuous Batching
Schedule (tentative) #
(forthcoming)