Diffusion Model

Generative Adversarial Network

A framework where two neural networks compete — a generator creates synthetic data while a discriminator evaluates its authenticity.

A neural network architecture based entirely on attention mechanisms, eliminating recurrence and enabling parallel processing of sequences.

Small trainable modules inserted between frozen transformer layers that enable task-specific adaptation without modifying the original model weights.

A regularisation technique that penalises large model weights during training by adding a fraction of the weight magnitude to the loss function, preventing overfitting.

A subset of machine learning using neural networks with multiple layers to learn hierarchical representations of data.

A sequence modelling architecture based on continuous-time dynamical systems that processes long sequences with linear complexity, offering an alternative to attention-based transformers.

A neural network trained to encode input data into a compressed representation and then decode it back to reconstruct the original.

A form of model parallelism that splits neural network layers across devices and pipelines micro-batches through stages, maximising hardware utilisation during training.

A learned dense vector representation of discrete data (like words or categories) in a continuous vector space.