Number Class
← Back to Explore
Computer Science Artificial Intelligence & Data Predictive Analytics

Recommendation Engines Are Built to Distrust Their Own Math

Spotify counts a recommendation as a win only if you listen 30 seconds; Netflix invented 76,897 micro-genres to stop your taste from calcifying into one vector

S.J. Nam 8 min read
Recommendation Engines Are Built to Distrust Their Own Math

Somewhere in Netflix's catalog metadata sits a category called Understated Romantic Road Trip Movies. It sits alongside Dark Road Trip Thrillers, Road Trip Art House Movies, and Canadian Independent Road Trip Movies — distinct enough to be separate rows on your homepage, narrow enough that you'd never invent them yourself. By 2014, Netflix had generated 76,897 such micro-genres, or "altgenres," created by its tagging system. That number is the tell. No human editorial team sits around drafting four flavors of road-trip cinema for fun. A machine did this, optimizing something, and the something was never "genre" in any sense a film critic would recognize.

That's the real story hiding under the tidy explainer version of "how Netflix and Spotify recommend things." The tidy version says: they measure similarity between what you like and what else exists, then show you the closest matches. True, as far as it goes. But the more interesting and more disruptive claim — the one the engineering documentation actually supports — is that neither company primarily uses similarity to show you what you'll like. They use it to generate an enormous, mathematically tractable space of "close enough" candidates, and then they spend just as much effort fighting that similarity, injecting noise, randomness, and diversity back into it, because pure resemblance is a trap. Similarity math, at scale, is self-defeating unless you deliberately break it a little. That's the part the "AI recommends your next binge" pieces skip.

Your Taste, Reduced to a Row of Numbers

Start with the mechanics, because they're stranger than the marketing suggests. Neither Netflix nor Spotify actually compares your list of favorite shows to someone else's list of favorite shows the way a person would. Instead, both convert users and titles into vectors — ordered lists of numbers with no inherent meaning — and measure the angle between them. This came out of the Netflix Prize, the contest Netflix launched in 2006 offering $1 million to anyone who could improve upon its existing recommender by at least 0.10 RMSE. The winning approach didn't rely on genre tags or plot descriptions at all. It relied on matrix factorization techniques, particularly Singular Value Decomposition, which decomposes the sparse user-item matrix into three smaller, dense matrices. Each user and each movie ends up as a short string of numbers — "latent factors" — and the power of cosine similarity lies in its ability to handle sparse data effectively, focusing on the angle between preference vectors rather than their magnitude.

Spotify runs the identical trick on sound instead of ratings. After Spotify's 2014 purchase of the music-intelligence firm The Echo Nest, its recommendation stack fused a natural language processing model that turns text about a song into a vector representation that can determine whether two pieces of music are similar with collaborative-filtering vectors built from listening behavior. Erik Bernhardsson, the engineer who built much of Spotify's early recommender, has described the resulting geometry with almost comic precision: after matrix factorization runs, every user and item can be represented as a vector in f-dimensional space, and Spotify used this to put a couple of million tracks in 40-dimensional space and then query for the most similar tracks. Forty numbers. That's what your favorite song reduces to before it ever reaches a search index — no title, no artist, no waveform, just coordinates.

Here's the detail almost no consumer coverage mentions: at that scale, computing exact similarity is too expensive to do for real. Comparing one track to five million others, in forty dimensions, for every user, every day, would be computationally absurd. So Bernhardsson wrote Annoy — Approximate Nearest Neighbors Oh Yeah — a randomized, tree-based algorithm built during a couple of afternoons during Spotify's internal Hack Week, which trades a small amount of accuracy for enormous speed. The recommendation you get isn't even the mathematically closest song. It's a close-enough approximation, chosen because true nearest-neighbor search doesn't scale. Similarity, in production, is already a compromise before anyone starts deliberately corrupting it further.

The Trap Pure Similarity Sets

Now the counterintuitive part. If cosine similarity worked the way the explainer pieces imply — find your taste vector, show you its nearest neighbors — both platforms would converge fast on staleness. Recommend only what's closest to what you already loved, and by definition you never encounter the record that redefines your taste or the show that becomes your new obsession. Engineers at both companies know this, and they've built entire secondary systems whose job is to undermine pure similarity on purpose.

Spotify's system for this has a name: BaRT, for Bandits for Recommendations as Treatments, described in a 2018 paper by Spotify researchers McInerney, Lacker, Hansen, Higley, Bouchard, Gruson, and Mehrotra. Bart learns how items and explanations interact within any given context to predict user satisfaction, and does this in a reinforcement learning setting where it must decide which actions to take next to gather feedback. In practice this means Spotify runs a constant, live experiment against itself: mostly show you what your vectors say you'll like, but periodically gamble on something your vectors say you probably won't — because the only way to learn whether you'd secretly love it is to occasionally ignore the math and just try. One internal detail rarely quoted outside engineering slide decks: Spotify's bandit reward function for the home screen counts a recommendation as a "success" only when a user streams the playlist for at least 30 seconds. Thirty seconds. That threshold — not any notion of aesthetic resemblance — is what half of Spotify's front page is quietly optimizing toward.

Netflix's version of the same instinct is less mathematically explicit but structurally identical. Rather than showing a single ranked list of "most similar" titles, it fragments the homepage into scores of narrow rows — those 76,897 altgenres — precisely so that a user's viewing history doesn't collapse into one dominant taste vector. Journalists who've reported from inside Netflix's personalization team have documented over 2,000 "taste communities," — clusters of subscribers who, say, streamed House of Cards and also watched It's Always Sunny in Philadelphia, an odd pairing no genre label would predict. Netflix's own staff call the resulting user clusters "taste doppelganger" profiles, and the phrase is telling: the system isn't finding what's objectively similar to a title, it's finding who else, statistically, behaves like you, then borrowing their history as a proxy for your future.

The Human Costume on the Machine

It's tempting to read the altgenre names — Emotional Independent Sports Movies, Gritty Chinese Action & Adventure from the 1970s — as evidence that Netflix's system understands culture. NPR's Alexis Madrigal, who spent months tracing where these categories came from, landed on a more honest description: only some of the logic that drives these categories feels human. That's the crux. The tags are human-applied — actual Netflix taggers watch films and mark up dozens of attributes, down to the moral status of characters — but the combination of those tags into a specific row shown to a specific person is pure vector arithmetic, matching your latent-factor coordinates to a community's. The English-language label is a costume. What's underneath is an angle in forty- or hundred-dimensional space that no tagger ever chose and no viewer will ever see.

A skeptic could object that this is just recommendation systems doing what recommendation systems have always done — cold-start problems, diversity injection, and exploration-exploitation trade-offs are textbook material in any machine learning course, not some hidden scandal. That's fair, and it's also beside the point. The point isn't that engineers are hiding something. It's that the popular story — "the algorithm knows what you like because it finds what's similar" — inverts the actual priority. Both companies discovered that naive similarity, taken seriously and applied at scale, produces a worse product: monotony, filter bubbles, staleness. So the real intellectual achievement of the last fifteen years in this field wasn't building better similarity math. It was learning exactly how much to distrust it.

That has an uncomfortable implication worth sitting with. If what streams into 2,000 taste communities and 76,897 microgenres is calibrated not toward resemblance but toward a 30-second engagement threshold and a bandit's gamble, then the "taste" these platforms claim to understand was never quite the target. Engagement was. Similarity was only ever the delivery mechanism — and knowing that changes what it means when Netflix says it knows what you'll love next.

References