Skip to content

Incremental and temporal similarity computations.

License

Notifications You must be signed in to change notification settings

gdamaskinos/isim

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SwIFT: Scalable Incremental Flexible Temporal recommender

Author: Georgios Damaskinos (georgios.damaskinos@gmail.com)

SwIFT is a recommender introduced in Capturing the Moment: Lightweight Similarity Computations that employs a new similarity metric, namely I-SIM. A secondary application of I-SIM, namely I-TRUST (presented in the same paper), can be found here.

Quick Start

The following steps evaluate SwIFT on the MovieLens100K dataset with a test set of 100 ratings.

  1. Setup
    • Ubuntu 18.04.3 LTS
    • Python 3.7
    • Java 1.8.0_77
    • spark-2.4.5-bin-hadoop2.7
    • apache-cassandra-3.11.6
    • sudo apt-get install bc
    • Export variables:
      • JAVA_HOME
      • SPARK_HOME
      • CASSANDRA_HOME
    • Setup cluster:
    cd utils/
    RUN_TESTS=1 bash deploy_cluster.sh STANDALONE /path/to/spark_localdir/
    
    • Shutdown cluster:
    cd utils/
    bash stop_cluster.sh STANDALONE
    
  2. Download and extract the MovieLens100K dataset
  3. Parse dataset:
    python parsers/movielensParser.py ml-100k/u.data dataset.csv
    bash parsers/splitter.sh dataset.csv ./ 100
    
  4. Deploy SwIFT
    bash local_deploy.sh trainingSet.csv testSet.csv 5 10 0 100 1 9995 /path/to/log
    

Structure

SwIFT consists of two main components:

  • Frontend
    • Accumulates the ratings in microbatches and sends them to the backend
    • Provides recommendations to users and computes CTR and recall
  • Backend
    • Bootstraps (if necessary) the database
    • Incrementally updates the database given the microbatches that the frontend sends
    • Logs latency measurements and computes the RMSE

More detailed information is available in the paper.

About

Incremental and temporal similarity computations.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published