Surprise in Recommendations

There's a principle in user interface design called "The Principle of Least Surprise" (or astonishment), which states that any action with a potentially ambiguous interpretation should result in the the least surprising consequence. Often the results of recommender systems are interpreted similarly: people evaluating the recommendations deem a recommendation to be good if it is expected. Once, long ago, i wrote here about how this doesn't seem to provide a good experience with music recommendations, because if a recommendation is not surprising, it is probably already familiar.

I've since decided that a bigger problem with music recommendations is that even great recommendations can't be appreciated unless you listen to the music (again, if you already know the recommended music, then it's not a compelling recommendation unless it's in the context of something like a personalized radio stream). However, i still think about this problem occasionally. It still seems to me that the interesting recommendations are those that you don't expect, but which still match your taste (if you eliminate the latter restriction, it's easy to make unexpected recommendations).

The main reason why unexpected recommendations are rare is (i think) because most recommendation systems are based on some measure of similarity between either items or users (this is the idea behind so-called collaborative filtering systems and content-based systems obviously seek to find item similarity). Often the set of recommended items will be chosen by comparison with similar items or by comparison with the tastes of similar users. So suppose that you like The Shins and the system discovers that other people who like the Shins often also like the The Decemberists. The latter is a good recommendation by most standards (including my personal subjective standards). But it is not a surprising recommendation.

Surprising in this context does not mean obscure like say Neutral Milk Hotel, or outrageous like say Iron Maiden. For me a surprising recommendation would be something that maybe takes a detour into a different, but adjacent genre. For example, i've been listening to a lot of Richard Thompson recently, and i'd be surprised but pleased if there were a connection from The Shins to Thompson via the intersection of indie rock with alt. country and alt. country with folk. It's unlikely that many systems would make that connection because the two artists don't have a significant shared fan base (though i'm sure it's larger than just me).

A mathematical model of surprise was developed a while back, but it treats surprise more in the sense of jumping-out-of-the-bushes rather than one-of-these-things-is-not-like-the-other. The idea of surprise that i have in mind is more like Ted Dunning's use of the word in his paper Accurate Methods for the Statistics of Surprise and Coincidence, in which surprise is more of a rare but significant co-occurence.

The reason why even the latter approach doesn't often produce the sorts of surprises that i want is that the information isn't in the data. For example, the connection that i claim between The Shins and Richard Thompson above is subjective. In the general population of music listeners there simply isn't enough data to establish a connection between the two artists. It seems like what is needed is something that can infer or enhance these unusual connections from a single person's listening habits. Or, perhaps, this connection could be found with content-based recommendation systems or some future refinement thereof.

I suppose that if you imagined a giant network that connects every musical artist with closest neighbors and so on until every pair of artists (A,B) are connected by some path, then you could over time weight the connections based on an individual user so that certain paths become "shorter". It might even be possible to structure this as a Bayesian network, where information about a user's artist preferences is used as evidence to connect previously unrelated artists for that individual. But, honestly, i'm not sure how you'd scale this across millions of users.