🎓 SASRec: The "Transformer" for Your Shopping Cart Welcome! If you've ever wondered how Netflix or Amazon seems to know exactly what you’re going to click next, you’re in the right place. SASRec (Self-Attentive Sequential Recommendation) is the "cool younger sibling" of recommendation models. It takes the Transformer architecture (the same tech behind ChatGPT) and applies it to user behavior.
🤔 Wait, What’s the Big Idea?
Most older recommendation systems look at your history in two ways:
The "Last Click" (Markov Chains): They only care about the very last thing you did. (Too short-sighted!)
The "Whole History" (RNNs): They try to remember everything in order, but often get "confused" by long sequences or random clicks.
The SASRec Breakthrough SASRec uses Self-Attention. Imagine you are looking at your own Amazon history. You bought a laptop, then a bag, then a bag of chips, then a mouse.
A normal model might think the chips are the most important because they are the most recent.
SASRec "looks" at all those items and says: "Wait, the laptop and the bag are related. The chips were just a snack. I should suggest a mousepad."
🛠️ How it Works
-
Embedding Layer We turn every item (like a Movie ID or Product ID) into a mathematical vector. We also add Positional Embeddings because, in sequential recommendation, the order matters. Buying a "Screwdriver" after "Screws" makes more sense than the other way around!
-
The Self-Attention Block. This is where the magic happens. The model calculates three things for every item in your history: Query (
$Q$ ): "What am I looking for"? Key ($K$ ): "What do I represent?" Value ($V$ ): "What information do I carry?" It then uses this formula to decide which past items are worth paying attention to: -
Prediction Finally, the model takes that "weighted summary" of your history and compares it against all possible items to find the best match for your next click.
📚 References for the Nerds Original Paper: Self-Attentive Sequential Recommendation by Kang & McAuley. https://arxiv.org/abs/1808.09781
Jay Alammar’s Blog: If the math is scary, read "The Illustrated Transformer"—it's the gold standard for learning this stuff.