Model Distillation

Tensor Processing Unit

Google's custom-designed application-specific integrated circuit for accelerating machine learning workloads.

Neural Processing Unit

A specialised processor designed to accelerate neural network computations in edge devices and mobile platforms.

Model Pruning

The process of removing redundant or less important parameters from a neural network to reduce its size and computational cost.

Neural Architecture Search

An automated technique for designing optimal neural network architectures using search algorithms.

Model Quantisation

The process of reducing the numerical precision of a model's weights and activations from floating-point to lower-bit representations, decreasing memory usage and inference latency.

Sparse Attention

An attention mechanism that selectively computes relationships between a subset of input tokens rather than all pairs, reducing quadratic complexity in transformer models.

Model Collapse

A degradation phenomenon where AI models trained on AI-generated data progressively lose diversity and accuracy, converging toward a narrow distribution of outputs.

Neural Scaling Laws

Empirical relationships describing how AI model performance improves predictably with increases in model size, training data volume, and computational resources.

Speculative Decoding

An inference acceleration technique where a small draft model generates candidate token sequences that are verified in parallel by the larger target model.

The capability of AI agents to invoke external tools, APIs, databases, and software applications to accomplish tasks beyond the model's intrinsic knowledge and abilities.

Automated machine learning that automates the end-to-end process of applying machine learning to real-world problems.

A harmonic mean of precision and recall, providing a single metric that balances both false positives and false negatives.

The movement to make AI tools, knowledge, and resources accessible to non-experts and organisations of all sizes.

Specialised hardware designed to speed up AI computations, including GPUs, TPUs, and custom AI chips.

A semiconductor designed specifically for AI and machine learning computations, optimised for parallel processing and matrix operations.

A technique where a language model is given a small number of examples within the prompt to guide its response pattern.

A metric for evaluating the quality of machine-generated text by comparing it to reference translations or texts.