Beta Phase: Square45 is currently in beta testing. Expect some features or content to be incomplete or missing.

Language Models

Field: Natural Language Processing

A probability distribution over sequences of words.

Sequence of Expressions

Definition

Word Embeddings

Theorem

The Chain Rule

Theorem

Cross-Entropy Loss

Theorem

The softmax function

Theorem

Vanishing Gradients

Theorem

Recurrent Neural Networks (RNNs)

Theorem

Long Short-Term Memory (LSTM)

Theorem

The KL Divergence

Principle

Maximum Likelihood Estimation (MLE)

Principle

Attention Mechanism