Gradient Boosting

Boosting

An ensemble technique that sequentially trains models, each focusing on correcting the errors of previous models.

Random Forest

An ensemble learning method that constructs multiple decision trees during training and outputs the mode of their predictions.

XGBoost

An optimised distributed gradient boosting library designed for speed and performance in machine learning competitions and production.

Decision Tree

A tree-structured model where internal nodes represent feature tests, branches represent outcomes, and leaves represent predictions.

Support Vector Machine

A supervised learning algorithm that finds the optimal hyperplane to separate different classes in high-dimensional space.

K-Nearest Neighbours

A simple algorithm that classifies data points based on the majority class of their k closest neighbours in feature space.

Naive Bayes

A probabilistic classifier based on applying Bayes' theorem with the assumption of independence between features.

Linear Regression

A statistical method modelling the relationship between a dependent variable and one or more independent variables using a linear equation.

Logistic Regression

A classification algorithm that models the probability of a binary outcome using a logistic function.

Polynomial Regression

A form of regression analysis where the relationship between variables is modelled as an nth degree polynomial.

Tabular Deep Learning

The application of deep neural networks to structured tabular datasets, competing with traditional methods like gradient boosting through specialised architectures and regularisation.

An optimisation algorithm that iteratively adjusts parameters in the direction of steepest descent of the loss function.

A mathematical function that measures the difference between predicted outputs and actual target values during model training.

The algorithm for computing gradients of the loss function with respect to network weights, enabling neural network training.

Machine learning techniques that combine multiple models to produce better predictive performance than any single model, including bagging, boosting, and stacking approaches.

A recommendation approach that suggests items similar to those a user has previously liked, based on item attributes.

Techniques that reduce the number of input variables in a dataset while preserving essential information and structure.

Unsupervised learning technique that groups similar data points together based on inherent patterns without predefined labels.

An adaptive learning rate optimisation algorithm combining momentum and RMSProp for efficient deep learning training.