The theory is this. If you assume that music is distributed in a high-dimensional feature space (as they surely do) the distribution of pieces in that space is almost sure to be highly uneven. Some areas (recordings of the Beethoven string quartets) will be densely populated; others (suites for toy piano) will be quite sparse.

If you then use k-nearest neighbors approaches to serve up recommendations for music (Spotify built the best-known library, so we know that they use it), you’ll likely hit music on the periphery of its local clusters far more often than music at the center.

Here’s a simple 2-d analogy. Imagine an alien crashing into a random point on Earth and searching for the nearest human to say “take me to your leader.” The odds are they’ll find someone rural; and it’s basically guaranteed they’ll hit a suburbanite before an urbanite unless they happen to crash into the middle of Central Park. They’re more likely to meet a Russian speaker than a Chinese speaker. And so on.

Spotify isn’t serving up songs randomly, but I wonder how much a similar dynamic comes into play when each person is turned into a vector to predict their next streams.

When I browse around this vector representation of all the books in the Hathi Trust I made, genre outliers just tend to pop out naturally. I love these, because they’re intrinsically interesting; I end up finding– for instance–a book telling the history of England in doggerel just as often as I find “normal” poetry.

For those of who can’t read it:

And trade’s embarrassment redoubles.

If I mistake, ^tis your’s to judge it,

But only overhaul the Budget

Which, for the service of the year,

Will millions, twenty-three appear ;

Thousands^ seven hundred fifty-six,

And hundreds, (as accountants fix,)

Some one or two ; a sum so great

Had ne’er before disturb’d the state;

But I’ll certainly get the wrong idea about what sort of books exist in the library if I assume that the elements that pop out in the less dense areas are more typical. In fact, I’d probably To some degree, Spotify is doing the same thing with music. And instead of cities and rural people, we have dense established genres.

My thinking about this has been heavily influenced by thinking about what I listen to now that I mostly use Spotify rather than my own digitized CDs for music listening. A typical example that Spotify’s recommendation algorithm surfaced for me quite early on is the music of the Austrian theorbist and composer Christina Pluhar, who puts together ensembles which, depending on your taste, are enchanting, insufferable, or inane. Here’s a track from her album of Purcell arrangements.

I like this. I have no idea idea if you do; and I don’t know exactly why I was recommended it. But if you assume this came out of a nearest-neighbor search in some region of a high-dimensional space, it’s easy to imagine why. This is an album that sits at an intersection of a bunch of different styles; recklessly loose early-music bands, non-traditional world music borrowers, Leonard Cohen “Allelulia” completists. Not something for everyone; but something for several different groups that ensures it will float way off in its own region of an embedding space.

What doesn’t Spotify surface? That’s a much harder question to answer. But I know that the only album I’ve recommended to anyone recently probably fits the bill quite nicely; the last couple disks of the Beaux-Arts Trio’s complete Haydn Piano Trios.

Without streaming, this is probably a less obscure disk than the Pluhar, but it’s also pretty damned obscure in its own right. The Haydn trios are far less often played than his string quartets or symphonies because those two genres ended up becoming more prestigious, but Haydn didn’t know that, and the music is equally as good. And while late Haydn has its own deeply appealing weirdness, I find it hard to imagine that there are any existing listeners out there who come to it before mostly exhausting their path through the Beethoven piano sonatas and Mozart quartets first. Pluhar’s music is sitting in a cabin in the Canadian woods waiting for any comers; Haydn’s trios are crammed into several walkups in Ditmars Steinway along with C.P.E. Bach, Boccherini, the rest of the less-trafficked classical canon.

So is this happening everywhere in culture? The degree to which its an algorithmic product isn’t clear, but it sure seems like the streaming services have settled into a bubble of half-hour unclassifiable formats rather than “sitcoms” and “dramas.” Netflix’s “personalized genres” are not the product of an embedding system, but do play naturally alongside one, because they generate an affinity for works that cut across different realms.

The causation here is complicated because, as with many other trends, the technology merely gloms onto a larger cultural trend. It seems quite possible to me that music recommendation services succeed right now because the zeitgeist is aligned in a way where many people are amenable to being served these kinds of hybrid works. If you want pure genre, there are ways of getting it; modern satellite radio stations, like Sirius-XM, will give you all the expert-curated music you want for any microgenre imaginable.

Is this a problem? I don’t think many would see it as such; but it’s worth thinking, nonetheless, about what it does to culture. Anyone who manages to occupy an empty space in the cultural manifolds will be richly rewarded; anyone who tries to stay in a heavily-trafficked space will languish. The idea of cultural areas as fluid, non-differentiable groups flowing into each other will be a self-fulfilling prophecy; anyone insisting that genres are real may seem hopelessly old fashioned. Anyone who navigates cultural spaces through digital means will be over-exposed to hybrid cultural forms, which will only lead them further to think that the different genres were an old-fashioned illusion, brought about by a particular set of constraints around channels, record labels, and the rest. And of course they’ll be right. But if they think there’s anything more natural about an enforced space emphasizing novelty, sparsity, and so forth, they’ll be wrong; and a cultural dynamic around filling in the valleys of a manifold spaces rather than building up the summits may be less rewarding than we hope.