Skip to content

yonathan06/cassandra-GLAM-tools

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
app
 
 
 
 
 
 
 
 
 
 

GLAM Wiki Dashboard

The purpose of this project is to Support GLAMs* in monitoring and evaluating their cooperation with Wikimedia projects. Starting from a Wikimedia Commons category this tool collects data about usage, views, contributors and topology of the files inside.

** GLAM - galleries, libraries, archives and museums

Structure

The project is split into two packages - app & services.

  • app - includes an express.js server that functions both as an API and as a front-end server (using mostly handlebars as render engine)
  • services - includes python & node.js scripts for ETL (extract, transform & load) and recommendations (not currently active). The ETL includes a new GLAM loader (for when adding a new GLAM to the system), and a daily cron job to updated the analytical data of yesterday for all of the GLAMs in the system.

Installation - app

Enter the /app folder

cd app

Install Node.js project dependencies:

npm install

Add a production config file inside the config folder: ./config/config.production.json With the same structure as in ./config/config.sample.json

export ENV:

export ENV="production"

Run the local server:

pm2 start server.js

You can see the logs by running th following command:

pm2 logs

Installation - services

Enter services folder

cd services

Install Python dependencies:

pip3 install -r requirements.txt
npm install

export ENV:

export ENV="production"

Add a production config file inside the config folder: ./config/config.production.json With the same structure as in ./config/config.sample.json

Run initdaily.sh - script that run dail.py.
Run it with the following commands to do it with a daily cron job

The daily script runs every day at 4:00 AM

Run daily cronjob

pm2 start daily.py --cron '0 4 * * *' --interpreter python3 --no-autorestart -- -e production

Save the process

pm2 save

A Logs folder will be created, every day at 4:00 AM a new file will be added with the date of the day, where you can always see the logs of the daily.

Run new glam listener

pm2 start new_glam_listener.py --interpreter python3 -- -e production

You can see the logs by running th following command:

pm2 logs

Starting local postgres instance using docker compose

docker-compose up -d

About

A usage analysis tool for GLAM institutes to follow free contents contributed to Wikimedia projects

Resources

License

Stars

Watchers

Forks

Releases

No releases published