5 Models for Engineering Personalized Digital Experiences (Part 1 – Amazon and Netflix)

This article is the beginning of a three-part series (you can also find part 2 and part 3) on algorithms and predictive models for creating personalized digital experiences.

Majority of websites are generic, one-size-fits-all, and unvarying. And email isn’t much better with manual input, static cadences, and fixed content.

Marketers and audiences deserve better, though.

As big data, artificial intelligence (AI), machine learning, and deep learning become larger influences and more attainable, now is the time for digital marketers to seize upon the potential these technologies offer.

Brands can now build custom on-site and email experiences for individuals that are automated and personalized by incorporating predictive modeling, journey mapping, and algorithms.

The ability to predict what users would be most interested in, calculate what they’re are most likely to interact with, and build a user experience that feels truly crafted for the individual. The resulting experience based on the data signals they provide through search, web activity, and site interaction can lead to increased engagement, use, positive sentiment, and sales (or conversions).

While site personalization is a benefit, automation is another. The ability to have dynamic and personalized content feed email messages and newsletters and send automatically (either based on a set cadence and/or behavioral triggers) provides marketers the opportunity to improve the efficacy of messages and content.

With so much potential, it is no surprise that all of my work has shifted to creating these types of website and digital marketing programs within the past few months.

If you’re beginning to invest in this area or are curious as to how all of this comes together in real life, here are the top models from my research — many of which in the companies’ own words — and that I’ve used as a base for similar predictive and personalized content systems. 

Amazon

Amazon has one of the most popular and longest running personalized digital systems, which is what offers up the recommendations users are used to seeing.

To accomplish these personalized recommendations that come from a massive database of users, items, and a constant influx of anonymous users, Amazon utilizes a process called “item to item collaborative filtering.” 

A snapshot of my Amazon homepage recommendations.

A snapshot of my Amazon homepage recommendations.

Amazon engineers Greg Linden, Brent Smith, and Jeremy York explain this process in greater detail, stating, “To determine the most-similar match for a given item, the algorithm builds a similar-items table by finding items that customers tend to purchase together. We could build a product-to-product matrix by iterating through all item pairs and computing a similarity metric for each pair. However, many product pairs have no common customers, and thus the approach is inefficient in terms of processing time and memory usage. The following iterative algorithm provides a better approach by calculating the similarity between a single product and all related products: 

 

A snippet of Amazon's algorithm.

A snippet of Amazon's algorithm.

It’s possible to compute the similarity between two items in various ways, but a common method is to use the cosine measure we described earlier, in which each vector corresponds to an item rather than a customer, and the vector’s M dimensions correspond to customers who have purchased that item. This offline computation of the similar-items table is extremely time intensive, with O(N2 M) as worst case. In practice, however, it’s closer to O(NM), as most customers have very few purchases. Sampling customers who purchase best-selling titles reduces runtime even further, with little reduction in quality. Given a similar-items table, the algorithm finds items similar to each of the user’s purchases and ratings, aggregates those items, and then recommends the most popular or correlated items. This computation is very quick, depending only on the number of items the user purchased or rated.”

Essentially, for every item X, Amazon creates a network of related items S(X). Whenever someone buys or looks at an item, Amazon then recommends the accumulated items from item X’s related network.

Rather than matching users to other similar users, this method matches items and forms them into the recommendation lists users see on Amazon.

Netfilx

 Another brand with a well-known personalized homepage based on recommendations is Netflix.

 Netflix doesn’t have one algorithm for its homepage, but instead runs tests on multiple types to find the most efficacious

Carlos Gomez-Uribe and Neil Hunt from Netflix have expanded on this notion by writing, “Historically, the Netflix recommendation problem has been thought of as equivalent to the problem of predicting the number of stars that a person would rate a video after watching it, on a scale from 1 to 5… Now, our recommender system consists of a variety of algorithms that collectively define the Netflix experience, most of which come together on the Netflix homepage.”

Some of the models and features that set the Netflix homepage include Personalized Video Ranker (PVR), the Top-N Video Ranker, Trending Now, Continue Watching, and Video-Video Similarity.

Mostly though, Netflix populates these sections through a method similar to Amazon’s item-to-item collaborative filtering, which uses restricted Boltzman machine (RBM) and a form of Matrix Factorization for collaborative filtering.

This combination creates a prediction system that compares — and makes recommendations — based on similar user behaviors.

Netflix's recommendation engine.

Netflix's recommendation engine.

Netflix’s Chris Alvino and Justin Basilico further explain the homepage construction in saying, “Once we have a set of possible video groups to consider for a page, we can begin to assemble the homepage from them. To do this, we start by finding candidate groupings that are likely relevant for a member based on the information we know about them. This also involves coming up with the evidence (or explanations) to support the presentation of a row, for example the movies that the member has previously watched in a genre. Next, we filter each group to handle concerns like maturity rating or to remove some previously watched videos. After filtering, we rank the videos in each group according to a row-appropriate ranking algorithm, which produces an ordering of videos such that the most relevant videos for the member in a group are at the front of the row. From this set of row candidates we can then apply a row selection algorithm to assemble the full page. As the page is assembled, we do additional filtering like deduplication to remove repeat videos and format rows to the appropriate size for the device.“ 

With such a system, there’s also the problem of over-personalization. To combat this issue and presentation bias, Netflix surfaces variants.

As shown, the algorithms will point toward related shows and genres, but every now and then, they’ll offer something a little different to gauge a user’s interest.

Again, Gomez-Uribe and Hunt note, “A problem in this area is finding clusters of members that respond similarly to different recommendations; another is finding effective ways to introduce randomness into the recommendations and learn better models.”

It is also important to note that Netflix’s personalization engine relies heavily on an extensive tagging structure with roughly 80,000 subgenre tags — something similar to Spotify’s system that I’ll examine in the next post.

Lastly, if you’re interested, Yehuda Koren also offers a more detailed description (than I could ever give) of collaborative filtering models used by Amazon and Netflix — and stay tuned for more.

Thank you

Thanks for reading! Feel free to connect with me on Twitter and/or LinkedIn.

You can also join my newsletter to receive the latest posts, digital marketing news, a marketing book recommendation, and a custom playlist every month.