Home
Shorts

Rmsnorm

9:09

Mistral Spelled Out : RMS Norm : Part 5

Aritra Sen

293 views

7:22

LayerNorm, InstanceNorm, GroupNorm: Batch Normalization alternatives for small batch sizes

Ashra Academy

2.9K views

3:04:11

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm

Umar Jamil

31K views

1:10:55

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU

Umar Jamil

53K views

12:46

Speculative Decoding: When Two LLMs are Faster than One

Efficient NLP

9.4K views

8:33

The KV Cache: Memory Usage in Transformers

Efficient NLP

31K views

11:17

Rotary Positional Embeddings: Combining Absolute and Relative

Efficient NLP

27K views

14:06

RoPE (Rotary positional embeddings) explained: The positional workhorse of modern LLMs

DeepLearning Hero

19K views

31:45

LLaMA Pro: Progressive LLaMA with Block Expansion (Paper Explained)

Yannic Kilcher

34K views

31:22

ALiBi - Train Short, Test Long: Attention with linear biases enables input length extrapolation

Yannic Kilcher

20K views

41:07

LLaMA: Open and Efficient Foundation Language Models (Paper Explained)

Yannic Kilcher

88K views

1:33

Metric 1 RMS error

Udacity

28K views