Model Collapse

Tensor Processing Unit

Google's custom-designed application-specific integrated circuit for accelerating machine learning workloads.

Neural Processing Unit

A specialised processor designed to accelerate neural network computations in edge devices and mobile platforms.

Model Distillation

A technique where a smaller, simpler model is trained to replicate the behaviour of a larger, more complex model.

Model Pruning

The process of removing redundant or less important parameters from a neural network to reduce its size and computational cost.

Neural Architecture Search

An automated technique for designing optimal neural network architectures using search algorithms.

Model Quantisation

The process of reducing the numerical precision of a model's weights and activations from floating-point to lower-bit representations, decreasing memory usage and inference latency.

Sparse Attention

An attention mechanism that selectively computes relationships between a subset of input tokens rather than all pairs, reducing quadratic complexity in transformer models.

Neural Scaling Laws

Empirical relationships describing how AI model performance improves predictably with increases in model size, training data volume, and computational resources.

Speculative Decoding

An inference acceleration technique where a small draft model generates candidate token sequences that are verified in parallel by the larger target model.

A theoretical level of AI that surpasses human cognitive abilities across all domains, including creativity and social intelligence.

The branch of ethics examining moral issues surrounding the development, deployment, and impact of artificial intelligence on society.

The coordination and management of multiple AI agents working together to accomplish complex tasks, routing subtasks between specialised agents based on capability and context.

The field of AI dedicated to representing information about the world in a form that computer systems can use for reasoning.

The principle of ensuring AI systems make equitable decisions without discriminating against any group based on protected attributes.

AI systems designed and trained for a specific task or narrow range of tasks, such as image recognition or language translation.

The ability to describe AI decision-making processes in human-understandable terms, enabling trust and regulatory compliance.

A technique combining information retrieval with text generation, allowing AI to access external knowledge before generating responses.