EECE695E: Efficient ML Systems (Spring 2025) #
Team #
Location & Time #
- Class. MW 11:00–12:15 @ PIAI 122.
- Office Hr. W 17:00–18:00 @ Terarosa Coffee (+ by appointment).
Schedule (tentative) #
- W1. Warm-Up
- (2/17) Intro & Administrivia
- (2/19) Computations of Deep Learning
- W2. Sparsity
- (2/24) Basic ideas
- (2/26) Advanced: Regularization, Structures, and Systems
- Presentation List out
- W3. Quantization
- (3/3) Data numerics and K-means
- (3/5) Linear quantiztaion and advanced stuffs
- Presentation Registration Due
- W4. NAS & KD
- (3/10) Knowledge Distillation
- (3/12) Neural Architecture Search
- W5. Efficient Training & Tuning
- (3/17) Continual Learning
- (3/19) Meta-Learning
- W6. Efficient Training & Tuning
- (3/24) Merging and Editing
- (3/26) Efficient Fine-Tuning
- W7. (Week off; Instructor at Brussel 🇧🇪)
- (3/31, 4/2)
- Project Proposals Due
- W8. Parallelism
- (4/7, 4/9)
- W9. Data Efficiency +
- (4/14, 4/16)
- W10. LLM Compression +
- (4/21, 4/23)
- W11. Long-Context LLMs +
- (4/28, 4/30)
- W12. Low-Precision Training +
- (5/5, 5/7)
- W13. Test-Time Scaling +
- (5/12, 5/14)
- W14. Efficient Diffusion Model +
- (5/19, 5/21)
- W15. Efficient Neural Rendering +
- (5/26, 5/28)
- W16. Poster Session
- (6/2, 6/4)
Recommended Materials #
- Blog Posts
- LLM inference performance engineering by Databricks
- Videos
- State-space models tutorial by Sasha Rush
- Trends in deep learning hardware by Bill Dally
- Related Courses
- Efficient deep learning systems at HSE university and Yandex
- Machine learning compilation at CMU
- TinyML and efficient deep learning at MIT
- Machine learning hardware and systems at Cornell
- Advances in Foundation Models at Stanford
- Books and surveys
- Algorithms for modern hardware by Sergey Slotin
- Efficient deep learning book by Menghani and Singh
- Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities by Bartoldson et al.