Generative Adversarial Network

Diffusion Model

A generative model that learns to reverse a gradual noising process, generating high-quality samples from random noise.

A neural network shortcut that allows the output of one layer to bypass intermediate layers and be added to a later layer's output.

A deep learning architecture designed for processing structured grid data like images, using convolutional filters to detect features.

A generative model that learns a probabilistic latent space representation, enabling generation of new data samples.

A technique that injects information about the position of tokens in a sequence into transformer architectures.

A skip connection that adds a layer's input directly to its output, enabling gradient flow through deep networks and allowing training of architectures with hundreds of layers.

A model compression technique where a smaller student model learns to mimic the behaviour of a larger teacher model.

A technique that normalises layer inputs during training to stabilise and accelerate deep neural network learning.

A memory optimisation that trades computation for memory by recomputing intermediate activations during the backward pass instead of storing them all during the forward pass.

Synthetic Data

Artificially generated data that mimics the statistical properties of real-world data for training and testing.