Overview
Direct Answer
Online learning is a machine learning paradigm in which models are continuously updated incrementally as individual data points or small batches arrive, rather than retraining on the complete dataset at once. This approach enables systems to adapt dynamically to concept drift and non-stationary environments.
How It Works
Models process each incoming observation (or mini-batch) sequentially, updating internal parameters through algorithms such as stochastic gradient descent or adaptive learning rules. The system discards or downweights old data, allowing it to reflect recent patterns whilst maintaining computational efficiency by avoiding full retraining cycles.
Why It Matters
Organisations benefit from reduced memory overhead, lower latency in adapting to changing data distributions, and the ability to process unbounded data streams in real-time. This is critical for applications where retraining on historical data is impractical or where rapid response to emerging patterns directly impacts business decisions.
Common Applications
Practical deployments include recommendation systems that personalise suggestions as user behaviour evolves, fraud detection systems that adjust to new attack patterns, sensor monitoring in IoT networks, and stock price prediction in financial markets. Autonomous vehicle perception systems and web search ranking similarly exploit this capability.
Key Considerations
Trade-offs include potential instability from individual noisy samples, difficulty in tuning hyperparameters without cross-validation datasets, and the risk of catastrophic forgetting in neural networks. Practitioners must carefully balance learning rates and implement safeguards to prevent degradation on earlier learned concepts.
Cross-References(1)
Referenced By1 term mentions Online Learning
Other entries in the wiki whose definition references Online Learning — useful for understanding how this concept connects across Machine Learning and adjacent domains.
More in Machine Learning
Loss Function
Training TechniquesA mathematical function that measures the difference between predicted outputs and actual target values during model training.
t-SNE
Unsupervised Learningt-Distributed Stochastic Neighbour Embedding — a technique for visualising high-dimensional data in two or three dimensions.
Collaborative Filtering
Unsupervised LearningA recommendation technique that makes predictions based on the collective preferences and behaviour of many users.
A/B Testing
Training TechniquesA controlled experiment comparing two variants to determine which performs better against a defined metric.
Naive Bayes
Supervised LearningA probabilistic classifier based on applying Bayes' theorem with the assumption of independence between features.
SHAP Values
MLOps & ProductionA game-theoretic approach to explaining individual model predictions by computing each feature's marginal contribution, based on Shapley values from cooperative game theory.
Gradient Descent
Training TechniquesAn optimisation algorithm that iteratively adjusts parameters in the direction of steepest descent of the loss function.
Matrix Factorisation
Unsupervised LearningA technique that decomposes a matrix into constituent matrices, widely used in recommendation systems and dimensionality reduction.