EECE695E: Efficient ML Systems (Spring 2024) #

Team #

• Instructor. Jaeho Lee 이재호
  firstname.lastname@postech.ac.kr
• TA. (forthcoming)

Location & Time #

• Class. (forthcoming)
• Office Hr. (forthcoming)

What we’ll cover (hopefully) #

• Deep Learning Basics
  • Architectures and Counting FLOPs
  • Hardware bits
• Making models smaller
  • Quantization
  • Pruning & Sparsity
  • Neural Architecture Search
• How to utilize the experience of other models
  • Transfer Learning & Distillation
  • Meta-Learning & Test-time Training
  • Model Merging & Stitching
  • Hyperparameter Transfer
  • Prompt Tuning
• Organizing Large-scale Learning
  • Parallelism & Pipelining
  • Federated Learning
• Data Efficiency
  • Data Compression
  • Dataset Distillation & Condensation / SeiT
• Tips & Tricks for Large Transformers
  • KV cache / FlashAttention / PagedAttention
  • Speculative Decoding / Medusa
  • Continuous Batching

Schedule (tentative) #

(forthcoming)