Underfitting

Ridge Regression

A regularised regression technique that adds an L2 penalty term to prevent overfitting by constraining coefficient magnitudes.

Elastic Net

A regularisation technique combining L1 and L2 penalties, balancing feature selection and coefficient shrinkage.

Cross-Validation

A resampling technique that partitions data into subsets, training on some and validating on others to assess model generalisation.

Overfitting

When a model learns the training data too well, including noise, resulting in poor performance on unseen data.

Bias-Variance Tradeoff

The balance between a model's ability to minimise bias (error from assumptions) and variance (sensitivity to training data fluctuations).

Regularisation

Techniques that add constraints or penalties to a model to prevent overfitting and improve generalisation to new data.

Gradient Descent

An optimisation algorithm that iteratively adjusts parameters in the direction of steepest descent of the loss function.

Stochastic Gradient Descent

A variant of gradient descent that updates parameters using a randomly selected subset of training data each iteration.

Adam Optimiser

An adaptive learning rate optimisation algorithm combining momentum and RMSProp for efficient deep learning training.

Learning Rate

A hyperparameter that controls how much model parameters are adjusted with respect to the loss gradient during training.

Loss Function

A mathematical function that measures the difference between predicted outputs and actual target values during model training.

Backpropagation

The algorithm for computing gradients of the loss function with respect to network weights, enabling neural network training.

Techniques that reduce the number of input variables in a dataset while preserving essential information and structure.

Errors or inconsistencies in the annotations of training data that can degrade model performance and lead to unreliable predictions if not properly addressed.

A classification algorithm that models the probability of a binary outcome using a logistic function.

Uniform Manifold Approximation and Projection — a dimensionality reduction technique for visualisation and general non-linear reduction.

An ensemble learning method that constructs multiple decision trees during training and outputs the mode of their predictions.

An online learning algorithm that balances exploration of new options with exploitation of known good options to maximise reward.

The end-to-end automation of the machine learning pipeline including feature engineering, model selection, hyperparameter tuning, and deployment, making ML accessible to non-experts.

The process of converting a trained model into a format that can be stored, transferred, and later reconstructed for inference.