Random Forest

Boosting

An ensemble technique that sequentially trains models, each focusing on correcting the errors of previous models.

Gradient Boosting

An ensemble technique that builds models sequentially, with each new model correcting residual errors of the combined ensemble.

XGBoost

An optimised distributed gradient boosting library designed for speed and performance in machine learning competitions and production.

Decision Tree

A tree-structured model where internal nodes represent feature tests, branches represent outcomes, and leaves represent predictions.

Support Vector Machine

A supervised learning algorithm that finds the optimal hyperplane to separate different classes in high-dimensional space.

K-Nearest Neighbours

A simple algorithm that classifies data points based on the majority class of their k closest neighbours in feature space.

Naive Bayes

A probabilistic classifier based on applying Bayes' theorem with the assumption of independence between features.

Linear Regression

A statistical method modelling the relationship between a dependent variable and one or more independent variables using a linear equation.

Logistic Regression

A classification algorithm that models the probability of a binary outcome using a logistic function.

Polynomial Regression

A form of regression analysis where the relationship between variables is modelled as an nth degree polynomial.

Tabular Deep Learning

The application of deep neural networks to structured tabular datasets, competing with traditional methods like gradient boosting through specialised architectures and regularisation.

A technique that decomposes a matrix into constituent matrices, widely used in recommendation systems and dimensionality reduction.

A learning paradigm where models generate their own supervisory signals from unlabelled data through pretext tasks.

A subset of AI that enables systems to automatically learn and improve from experience without being explicitly programmed.

A machine learning paradigm where agents learn optimal behaviour through trial and error, receiving rewards or penalties.

A situation where the distribution of classes in a dataset is significantly skewed, with some classes vastly outnumbering others.

A variant of gradient descent that updates parameters using a randomly selected subset of training data each iteration.

A regularisation technique combining L1 and L2 penalties, balancing feature selection and coefficient shrinkage.

The process of using domain knowledge to create, select, and transform input variables to improve model performance.