Machine Learning Cheat Sheet for Beginners (Algorithm Selection Map)

Machine Learning Cheat Sheet for Beginners

Most machine learning failures are not modelling failures. They are the wrong metric, the wrong split, or a model that memorised its training data.

Picking the algorithm is the easy part — there is a small map from data shape to algorithm family, and this sheet contains it. The parts that decide whether your model works are further down.

Updated for 2026. Examples use scikit-learn conventions.

The Three Learning Types

Supervised learning splits again by what the label is:

• Classification — the label is a category. Spam or not. Which of five products.
• Regression — the label is a number. Price, temperature, days until churn.

What people get wrong: framing a regression problem as classification by bucketing the target. Predicting "high, medium, low" price throws away information the model could have used and forces you to defend arbitrary boundaries.

The Algorithm Selection Map

This is the table to actually use. Find your row, start with the first suggestion, and only move right if it is not enough.

The single most useful line in this table: for tabular data of any realistic size, gradient boosting wins most of the time. Neural networks dominate images, audio, and text; on spreadsheets, boosted trees remain the default answer, and beginners routinely reach past them for a deep model that performs worse.

Algorithms — When To Use, When Not To

The interpretability tradeoff is real and it is a business decision. A logistic regression that a loan officer can explain in a sentence may be worth two accuracy points against a boosted model nobody can defend to a regulator.

Metrics — The Section That Decides Whether Your Model Works

The confusion matrix

Everything below is derived from four counts.

                 Predicted +      Predicted −
Actual +    True Positive     False Negative   ← missed it
Actual −    False Positive    True Negative
                 ↑ false alarm

The metrics themselves

Why accuracy is wrong on imbalanced data

This is the most consequential idea on this page.

Suppose 1% of credit card transactions are fraudulent. A model that predicts "not fraud" for every single transaction achieves 99% accuracy and catches zero fraud.

Predict all negative on 1% positives:
  accuracy  = 99%     ← looks excellent
  precision = 0       ← undefined / no positives predicted
  recall    = 0       ← caught nothing

Accuracy counts every correct prediction equally, which hides the fact that all the errors landed on the only class you cared about. On any dataset where the classes are unbalanced — fraud, disease, churn, defects, click-through — accuracy is not just imperfect, it is actively misleading.

Use instead: precision and recall, the F1 score, or the confusion matrix directly.

Precision or recall — which one

They pull against each other. Flagging more items raises recall and lowers precision. The choice depends entirely on which error costs more.

ROC-AUC is threshold-independent and useful for comparing models, but on heavily imbalanced data prefer the precision-recall curve, which does not flatter a model that is good at the easy majority class.

Train, Validation, Test

The test set is not a validation set. Every time you look at test performance and change something, you leak information into the model, and the test score stops being an honest estimate of new data. If you have tuned against it, it is now a validation set and you no longer have a test set.

Cross-validation

from sklearn.model_selection import cross_val_score, StratifiedKFold

scores = cross_val_score(model, X, y, cv=StratifiedKFold(5), scoring='f1')
print(scores.mean(), scores.std())

Cross-validation trains on all folds but one, evaluates on the held-out fold, and rotates until each fold has been held out once. The reported score is the average, and the standard deviation tells you how stable the result is.

Two variants that are not optional:

• Stratified folds for classification, so each fold keeps the original class balance.
• Time-based splits for time series, so the model never trains on data from after the period it is predicting.

What people get wrong: shuffling time series data before splitting. The model learns from the future, scores brilliantly, and fails completely in production.

Overfitting vs Underfitting

The gap between training and validation score is the diagnostic. Check it after every training run, before you look at anything else.

L1 (Lasso) shrinks some coefficients to exactly zero, so it performs feature selection. L2 (Ridge) shrinks all coefficients toward zero without eliminating any. Use L1 when you suspect many features are useless; L2 when they are all mildly informative.

Feature Scaling

The rule underneath: anything that measures distance or uses gradient descent needs scaling; anything that splits on thresholds within one feature does not.

from sklearn.preprocessing import StandardScaler, MinMaxScaler

StandardScaler()   # mean 0, std 1 — the default
MinMaxScaler()     # squeeze into [0, 1] — for bounded inputs

Without scaling, a feature measured in thousands overwhelms a feature measured in decimals purely because of its units, and the model quietly ignores the smaller one.

Fit the scaler on training data only, then apply it to validation and test. Fitting on the full dataset leaks the test set's distribution into training — one of the most common and most invisible mistakes in beginner pipelines.

The Workflow Order

1. Define the problem and the metric first. Before any code, decide what a good model means numerically and what an error costs.
2. Split the data. Before exploration, before scaling, before anything. Hold out the test set immediately.
3. Explore and clean. Missing values, obvious errors, class balance, leakage candidates.
4. Build a dumb baseline. Predict the majority class, or the mean. Every later model is measured against this.
5. Engineer features. Usually worth more than the model choice on tabular data.
6. Train a simple model. Logistic or linear regression. Compare to the baseline.
7. Train a strong model. Gradient boosting for tabular; a pretrained network otherwise.
8. Tune hyperparameters using cross-validation. Never using the test set.
9. Evaluate on the test set once. Report that number, not the best one you saw.
10. Monitor after deployment. Data drifts; a model that was good in March may not be in September.

Step 4 is skipped most often and matters most. Without a baseline you cannot tell whether your gradient boosting model is adding value or just adding complexity.

The Five Mistakes

1. Using accuracy on imbalanced data. A 99% accurate fraud detector that catches no fraud is the canonical example. Check the class balance before you choose a metric, always.

2. Data leakage. Scaling before splitting, using a feature that would not exist at prediction time, or including the target in disguise. The signal is a validation score that looks too good — investigate rather than celebrate.

3. Tuning against the test set. Once you have made decisions using test performance, it is no longer an honest estimate. Keep a genuinely untouched holdout.

4. Reaching for deep learning on tabular data. Gradient boosting beats neural networks on most spreadsheet-shaped problems with far less tuning and far less compute. Deep learning earns its place on images, audio, and text.

5. Shuffling time series before splitting. The model trains on the future, scores brilliantly, and fails in production. Time series need time-ordered splits, no exceptions.

Print This Section

PICK      tabular + labels ......... gradient boosting (XGBoost/LightGBM)
          tabular, tiny / explain .. logistic or linear regression
          images ................... pretrained CNN, fine-tuned
          text ..................... pretrained transformer
          no labels, k known ....... k-means
          no labels, k unknown ..... DBSCAN
          too many features ........ PCA
          find outliers ............ isolation forest

METRICS   balanced classes ......... accuracy is fine
          imbalanced classes ....... precision / recall / F1  (NEVER accuracy)
          false alarms costly ...... optimise precision
          misses costly ............ optimise recall
          regression ............... RMSE (penalises big errors), MAE

FIT       train high + val low ..... overfitting → more data, regularise, simplify
          both low ................. underfitting → richer model, better features

SCALE     needed: kNN, SVM, k-means, PCA, linear w/ GD, neural nets
          not needed: trees, random forest, gradient boosting
          fit the scaler on TRAIN only

ORDER     metric → split → clean → baseline → features → simple → strong
          → CV tune → test ONCE → monitor

Machine learning rewards discipline more than cleverness. Choose the metric before the model, split before you touch the data, and beat a dumb baseline before you reach for anything complicated.

🔗 Next in this collection: Pandas and NumPy Cheat Sheet, or return to The Complete Developer Cheat Sheet Collection.