Skip to content

Deep Learning Roadmap

PyTorch-focused — click any node to open the article or video.

Foundations

Understanding Neural Networks
Loss Functions
Activation Functions
Weight Initialization
Vanishing / Exploding Gradients

Architectures

Feedforward Neural Network
Autoencoder
Convolutional Neural Network
Recurrent Neural Network
Transformer
Siamese Network
Generative Adversarial Network
Residual Connections
Encoder / Decoder

Training

Learning Rate Schedule
Batch Normalization
Batch Size Effects
Multitask Learning
Transfer Learning
Curriculum Learning

Optimizers

Regularization

Early Stopping
Parameter Penalties
Data Augmentation
Adversarial Training

Model Optimization

Distillation
Quantization
Neural Architecture Search

Tools

TensorBoard
Hugging Face Transformers