Neural Processing Unit

Tensor Processing Unit

Google's custom-designed application-specific integrated circuit for accelerating machine learning workloads.

Model Distillation

A technique where a smaller, simpler model is trained to replicate the behaviour of a larger, more complex model.

Model Pruning

The process of removing redundant or less important parameters from a neural network to reduce its size and computational cost.

Neural Architecture Search

An automated technique for designing optimal neural network architectures using search algorithms.

Model Quantisation

The process of reducing the numerical precision of a model's weights and activations from floating-point to lower-bit representations, decreasing memory usage and inference latency.

Sparse Attention

An attention mechanism that selectively computes relationships between a subset of input tokens rather than all pairs, reducing quadratic complexity in transformer models.

Model Collapse

A degradation phenomenon where AI models trained on AI-generated data progressively lose diversity and accuracy, converging toward a narrow distribution of outputs.

Neural Scaling Laws

Empirical relationships describing how AI model performance improves predictably with increases in model size, training data volume, and computational resources.

Speculative Decoding

An inference acceleration technique where a small draft model generates candidate token sequences that are verified in parallel by the larger target model.

An approach to AI that uses human-readable symbols and rules to represent problems and derive solutions through logical reasoning.

The simulation of human intelligence processes by computer systems, including learning, reasoning, and self-correction.

The process of determining cause-and-effect relationships from data, going beyond correlation to establish causation.

A hypothetical form of AI that possesses the ability to understand, learn, and apply knowledge across any intellectual task a human can perform.

The AI capability to make inferences based on everyday knowledge that humans typically take for granted.

Systematic errors in AI outputs that arise from biased training data, flawed assumptions, or prejudicial algorithm design.

The principle of ensuring AI systems make equitable decisions without discriminating against any group based on protected attributes.

AI systems designed and trained for a specific task or narrow range of tasks, such as image recognition or language translation.

Neural Network

A computing system inspired by biological neural networks, consisting of interconnected nodes that process information in layers.