Every classifier is just a line in the right place
Strip away the vocabulary and almost all of classification is this: given two kinds of thing scattered in space, where do you put the boundary? Draw yours, then watch four algorithms draw theirs — and find out who wins.
7 min · Beginner · Playable
Two kinds of thing, one question
Imagine plotting every email you have ever received: spam in one colour, real mail in the other, positioned by how many links they contain and how urgent the language is. Spam clusters in one corner. Now draw a line. Anything on that side gets filtered. That is a classifier — and everything else is a different opinion about where the line goes.
The axes are features
Two here so you can see it. A real model might use two hundred, which is why nobody draws them.
The line is the model
Training means searching for the boundary that separates the training points best.
The score is on new points
Any boundary can be twisted to fit what it has seen. Only held-out data tells you if it works.
You versus the algorithms
Press “Draw my boundary” and swipe across the chart. Then click each algorithm to see where it drew its own, and how it scored. Points ringed in white are the ones that model got wrong.
Scoreboard
Whatever line you drew. Humans are usually better at this than they expect.
All four models are trained in your browser on the points you can see — real logistic regression by gradient descent, real trees built by Gini impurity. Nothing is precomputed.
In plain words
Most people beat logistic regression on the first try, because you can see the curve and it can only draw a straight line. Then tick “Score on held-out data” — points nobody has seen. That is the number that decides whether a model ships.
Each one has a shape it prefers
Look at the coloured regions rather than the score. The shape of the boundary tells you what the algorithm believes about the world before it sees any data — and that belief is why they disagree.
Logistic regression draws one straight line
It cannot do anything else. That constraint makes it fast, readable, and impossible to overfit badly — which is exactly why it is the baseline every project should start from.
k-NN draws islands
It has no line at all; it just asks the nearest five points. That makes it flexible and slightly paranoid — every outlier gets its own little territory.
A tree draws a staircase
It can only cut horizontally and vertically, one feature at a time. Perfectly readable as a set of rules, and visibly awkward on a diagonal boundary.
A forest blurs the staircases
Twelve trees on twelve different samples, averaged. Each is jagged; together they are smooth. This is why ensembles still win on tabular data.
What this explains
Why there is no best algorithm
Each one assumes a different shape. On this data the forest usually wins; on data with a genuinely straight boundary, logistic regression wins and is a tenth of the cost. Match the assumption to the problem.
Why you always run a baseline
If the complicated model cannot beat a straight line by a margin worth the extra risk, the straight line is the answer. Surprisingly often, it is.
Why k-NN gets slow
It does no training at all, then does all the work at prediction time — comparing against every stored point, forever. Fine for thousands, hopeless for millions.
Why features matter more than models
Add one good feature and every boundary above gets easier to draw. Most real gains come from better inputs, not better algorithms.
Next
Now do it with two hundred dimensions
Module II is this, on real data you cannot draw: feature engineering, validation you can defend, and choosing the model that suits the problem rather than the one in the headline.