Spotify Stats App - download history, create custom playlists, view stats about your listening habits
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
SpotifyStats.client Updated comments, added README, added Client code to repo Jan 10, 2019
SpotifyStats.server Updated ESLint config Jan 15, 2019
.gitignore Initial commit Jan 7, 2019
README.md Added API server application Jan 10, 2019

README.md

About Spotify Stats

Spotify Stats is an application written in JavaScript / Node / Angular and uses MySQL as database for downloading Spotify listening history, create custom playlists, view stats about your listening habits and much more.

Spotify Stats Server

Spotify at current time does not offer an API to download user history so we have to go around it. Theres an API endpoint that allows anyone with a Spotify account to access most recently played tracks. In return we get a list of up to 50 most recent playbacks.

To get started you must follow https://developer.spotify.com/documentation/general/guides/authorization-guide/ to get keys and access tokens which are needed to make API calls.

What you need are the following

SPOTIFY CLIENT ID
SPOTIFY CLIENT SECRET
SPOTIFY REFRESH TOKEN

Spotify Stats Server uses MySQL as database storage, so you will need to set up a MySQL installation and use database schema provided in /api/db/mysql-schema.sql file.

Once you have all the required authorization data for Spotify API and database, they must be placed inside .env file in the root of SpotifyStats.server folder. It's an environment file used by Node when executing server scripts. It has the following format:

MYSQL_HOST = mysql_host_name
MYSQL_USERNAME = mysql_username
MYSQL_PASSWORD = mysql_password
MYSQL_DATABASE = spotify_database_name
SPOTIFY_CLIENT_ID = spotify_client_id
SPOTIFY_CLIENT_SECRET = spotify_client_secret
SPOTIFY_REFRESH_TOKEN = spotify_refresh_token

Server application offers three functions:

  1. Generate the access_token using our credentials (placed insdie .env file) to download our listening history and dumping the data in a new .json file every day. To make sure that the access token doesn’t expire, new one is generated with every call to the API. (get-data.js file)

  2. Parse downloaded data and save it to a database (save-data.js file) To keep track history up to date I have a CRON job set up to ping Spotify API every three hours.

0 */3 * * * (cd /path_to/SpotifyStats.server/api/; /path_to/node get-data.js)

Files are saved in YYYY-MM-DD.json format and every Spotify response is appended to current day file. Because we are pinging the API every three hours, at the end of the day we have files that contain 8 arrays of the last 50 tracks. Unless you listen to Spotify non-stop all day every day, there is going to be lots of redundancy because of duplicate records. This is taken care of in 'save-data.js' file with removeDuplicateTracks function:

const removeDuplicateTracks = (arr) => {
  return arr.filter((item, pos, array) => {
    return array.map(track => track.track_played_at).indexOf(item.track_played_at) === pos;
  });
};

This functions takes each downloaded chunk and returns the deduplicated list of tracks containing only unique items according to the 'played_at' field, which is the timestamp of each song played. Also, MySQL table has an UNIQUE attribute set on that field.

To save day worth of listening history i also have a CRON job which is launched at 1 AM every day. This script takes last day's history file, parses it and saves it to a database.

0 1 * * * (cd /path_to/SpotifyStats.server/api/; /path_to/node save-data.js)
  1. Serve results from database to client application. Backend server uses express web framework as a web API for the client. Just run node server.js (defaults to port 8080)


## Spotify Stats Client WORK IN PROGRESS