24S - (under construction)

EECE695E: Efficient ML Systems (Spring 2024) #

Team #

Location & Time #

  • Class. (forthcoming)
  • Office Hr. (forthcoming)

What we’ll cover (hopefully) #

  • Deep Learning Basics
    • Architectures and Counting FLOPs
    • Hardware bits
  • Making models smaller
    • Quantization
    • Pruning & Sparsity
    • Neural Architecture Search
  • How to utilize the experience of other models
    • Transfer Learning & Distillation
    • Meta-Learning & Test-time Training
    • Model Merging & Stitching
    • Hyperparameter Transfer
    • Prompt Tuning
  • Organizing Large-scale Learning
    • Parallelism & Pipelining
    • Federated Learning
  • Data Efficiency
    • Data Compression
    • Dataset Distillation & Condensation / SeiT
  • Tips & Tricks for Large Transformers
    • KV cache / FlashAttention / PagedAttention
    • Speculative Decoding / Medusa
    • Continuous Batching

Schedule (tentative) #

(forthcoming)