Number Class
← Back to Explore
Computer Science Artificial Intelligence & Data Vector Embedding Design

The Equation Behind 'Recommended for You' Has a Popularity Thumb on the Scale

Netflix and Cornell researchers now dispute whether the standard fix—cosine similarity—actually removes the bias, or just hides it better

S.J. Nam 7 min read
The Equation Behind 'Recommended for You' Has a Popularity Thumb on the Scale

Simon Funk didn't work for Netflix. In December 2006, he was a programmer with a blog, tinkering with a spreadsheet-sized problem that was actually enormous: predict how half a million people would rate movies they hadn't seen yet, using only the ratings they had already given.
Netflix had handed contestants 100 million ratings of 17,000 movies from 500,000 users and asked them to fill in the rest.
Out of roughly 8.5 billion possible user-movie pairings, all but a sliver were blank. Funk's fix, posted to his personal site rather than a journal, became
the approach that made matrix factorization widely known during the Netflix Prize challenge, after he shared his findings with the research community.
At the center of it sat one of the plainest operations in mathematics: multiply, multiply, add. The dot product. Sixteen years later, that same operation is quietly deciding what plays next on your phone, and it turns out to be a much less neutral referee than anyone expected.

Multiply, Add, Rank

Strip away the jargon and a dot product is arithmetic a middle schooler could do. Take two lists of numbers — a user's list and a movie's list — multiply the numbers in matching positions, then add up the results. One number falls out the end. Funk's version, later called Funk SVD, represented every user as a short list of latent "taste" numbers and every movie as a matching list of "style" numbers, then
predicted a rating as a global mean plus per-user and per-item biases plus the dot product of those low-rank latent factors
. Nobody had to label the numbers "loves gritty crime dramas" or "has explosions"; gradient descent found factors that made the dot products line up with the ratings people had actually given. It worked well enough to help push Funk into the Netflix Prize's top three, and the trick generalized far beyond movies — it's the same skeleton underneath Spotify's song embeddings, Amazon's "customers also bought," and the two-tower models that narrow a billion-item catalog down to a shortlist before any fancier ranking begins.

The seductive part of the story usually stops there: a simple sum of products, computed at planetary scale, somehow captures taste. But the dot product was never only measuring how well two vectors point in the same direction. It was also measuring how long they are — and length turns out to carry a signal nobody asked it to carry.

The Ruler Nobody Told You Was Rigged

Here is the detail most explainers skip. A dot product between vectors uu and vv equals ∣u∣∣v∣cos⁡θ|u||v|\cos\theta — the angle between them, scaled by both magnitudes. Direction is the "taste match" part. Magnitude is something else entirely, and in real recommendation systems it isn't random noise; it correlates with fame. As one practitioner explanation of similarity metrics puts it,
when user and item data go unnormalized, popular items and optimistic users tend to have long vectors, because they carry high average ratings and long vectors are more likely to produce a high predicted score when multiplied.
Redis's own engineering guidance recommends the dot product specifically
for recommendation systems and collaborative filtering, where magnitude represents importance, like user activity levels
— which is another way of saying: the busier or more popular you are, the harder your vector pushes the score upward, on top of whatever your actual content matches.

This isn't theoretical. A study tracking embeddings inside a large social platform's recommender found that
since most ranking approaches use the dot product of user and content embeddings, content embeddings with a higher norm produce a higher ranked order, meaning they get recommended more
— and the researchers watched that norm inflate as content simply accumulated more views, a feedback loop with no ceiling. A separate spectral analysis of recommendation models found the same fingerprint from a different angle:
item popularity gets memorized in the principal spectrum of the model's predicted score matrix, and a phenomenon called dimension collapse amplifies that spectrum's prominence, intensifying the popularity bias further
. Put plainly: the machinery meant to find your personal match is structurally tilted toward whatever is already a hit, and it gets more tilted the longer it runs. The counterintuitive part isn't that recommenders favor blockbusters — everyone suspects that. It's that the bias isn't a bug introduced by bad training data alone; it's latent in the geometry of the scoring function itself, baked in before a single popularity feature is added.

Netflix's Own Engineers Reopen the Case

The textbook fix is cosine similarity: divide out the magnitudes, keep only the angle, and the popularity thumb comes off the scale — in theory. That's the received wisdom underpinning a decade of "just normalize your embeddings" advice. Then, in 2024, researchers from Netflix itself and Cornell University complicated the tidy story.
A WWW '24 paper by H. Steck and colleagues, "Is Cosine-Similarity of Embeddings Really About Similarity?," challenged the widespread belief that cosine similarity's directional alignment offers a more meaningful measure than a simple dot product.
The core argument: because matrix-factorization models are trained to make dot products match observed ratings, and regularization can rescale the factors in ways that leave those dot products unchanged while shifting the angles between vectors, the cosine similarity computed afterward can be an artifact of arbitrary choices made during training rather than a clean read of "true similarity." In other words, the fix inherits its own hidden knob, one that engineers can turn — deliberately or not — without anyone downstream noticing.

The Referees Are Still Arguing

What makes this genuinely rare territory for a math explainer is that the argument isn't settled. A rebuttal circulating under the title "In Defense of Cosine Similarity: Normalization Eliminates the Gauge Freedom" pushes back directly on Steck's team, arguing that
the pathology they identified is real but narrower than claimed — it stems from a mismatch between the training objective and the evaluation metric, not from any inherent flaw in cosine similarity itself
. Their proposed remedy is to normalize embeddings onto a sphere during training itself, closing the loophole rather than discarding the metric. So the field's two camps agree on the symptom — the numbers can be gamed — and disagree on the cure. That is an unusually candid, in-progress fight for a piece of math most people assume was settled in a 19th-century physics textbook and never touched again.

Whose Thumb Is Actually on the Scale

The lesson isn't that dot products are broken or that engineers should switch metrics and move on. It's that "similarity," as computed by the systems recommending your next show or song, was never a description of you alone. It is a joint function of your taste, the item's content, and — quietly, structurally — how loud that item already is in the room. Choosing dot product over cosine, or cosine over dot product, or neither over a normalized sphere, is not a neutral engineering default; it is a decision about how much extra credit fame gets before taste is even consulted. Recommendation platforms rarely advertise which choice they've made, and as the Netflix-Cornell dispute shows, even the people building these systems are still arguing about whether their supposed fix actually fixes anything. The next time an app insists it "knows what you like," it's worth asking a blunter question: does it know what you like, or does it just know what everyone else already likes, run through an equation old enough to have been taught to Euclid's students?

References