RED-I Uniq - A small version of the RED-I UniqueIds project

Designed to process RED-I data in individual files, the ability to run a single person is operational, to run a group of people is not yet, but will be configurable

Contents of the Repository

• The Makefile:
  • Pre RED-I Tasks
    • make - will show all available make targets
    • make emr - Get the EMR data (based on a RED-I settings.ini file)
    • make filter - Filter 'raw.txt' based on a configured rule list
    • make count - Count lines in 'raw.txt' and compare to per participant files
    • make uniq - Create the unique file based from a raw file
  • During RED-I Tasks
    • make lastlog - Get the last log for a particular subject (ID=000-0000)
  • Post RED-I Tasks
    • make watch - Watch the state of the current run for a specific site
    • make cprep - Copy reports files from complete runs
    • make merge - Merge the reports files into JSON for email report
    • make email - Send the reports to target recipient
• All scripts:
  • Shell Scripts:
    • collect_reports.sh - Used by 'make cprep', this script goes through all the user_data participant files and looks for the report file that is set in the configuration (uniq.ini). These reports are generated by RED-I if the run actually completes
    • count_lines.sh - This is used by 'make count', this script goes through the participant files in user_data and compares the total line count with the raw files line count.
    • cron.sh - This is the script that is setup in a cron entry, which triggers get_emr.sh, filter_raw.sh, get_unique.sh, run_redi.sh, and collect_reports.sh (based on configuration in uniq.ini)
    • filter_raw.sh - Used by 'make filter', this script goes through the raw file and runs the python script filter_raw.py, using a rules.json file to clean the raw file.
    • get_config.sh - Used to be the traffic cop, this will collect the path to your uniq.ini file.
    • get_emr.sh - Used by 'make emr', this script uses a RED-I settings.ini file (emr_* variables) and collets a fresh copy of the EMR data for the configured site.
    • get_last_log.sh - Used by 'make lastlog', this script collects the latest log path from a particular participant (specified by ID=000-0000).
    • get_unique.sh - Used by 'make uniq', this script is the script that started the whole idea. It parses the raw.txt data and collects all subjects based on a column given in uniq.ini.
    • merge_reports.sh - Used by 'make merge', this script collects all the designated output XML files from RED-I and merges them into a single report JSON file
    • run_redi.sh - This is the script that actually triggers the RED-I process to run
    • send_email.sh - Used by 'make email', this script collects the output from 'make merge' and sends a customized email to the configured recipients
    • watch_run.sh - Used by 'make watch', this script looks at the latest RED-I run, based on configuration from uniq.ini for a specific site.
  • Python Scripts:
    • init.py - Um duh!
    • filter_raw.py - Used in the execution of 'make filter', which runs filter_raw.sh.
    • get_unique.py - Used in the execution of 'make uniq', which runs get_unique.sh.
    • merge_reports.py - Used in the execution of 'make merge', which runs merge_reports.sh
    • send_email.py - Used in the execution of 'make email', which runs send_email.sh
    • myrules.py - Yes, the only one not named something_somethingelse. This script is the 'decision.py' of this process, and sets the available rule types for the filter process.

Danger

Only tested with redi version 0.15.4