John Eskew edited this page Oct 3, 2017 · 152 revisions

This wiki contains a centralized listing of shared tools for working with the edX platform, hosted both in this repository (edx/edx-tools) and in other public repositories.

List of Tools for Working with the edX Platform

Tools are divided into three categories, based on how ready they are for public consumption, how well they are documented, and how much setup they may require for a new user.

  • GREEN (easy to use) -- plug-and-play. Well documented and tested.
  • YELLOW (medium difficulty of use) -- requires some technical skills to set up. Some documentation and testing.
  • RED (advanced) -- requires advanced technical skills or domain knowledge to use. Not much documentation or not fully tested; use at your own risk.

GREEN -- Plug-and-Play

  1. Dump Databases - https://github.com/sefk/dump-dbs
  • Description: Nightly database dumps of production databases. Nicely driven by a YML configuration file to control things like dump type (mysql or mongo), compression, sanitizing via sed commands. Intended to be run from a nightly cron job, see the README
  • Link: dump-dbs
  • Repository: sefk/dump-dbs
  • Author: Sef Kloninger
  • Date Contributed: 27-October-2013
  1. RuQL, a Ruby-embedded question language - https://rubygems.org/gems/ruql
  • Description: Simple DSL embedded in Ruby for creating multiple-choice and other short-answer questions. The questions can be rendered to "EdXML", HTML 5, JSON (eg for use with MOOCrp), AutoQCM (for generating machine-scorable Scantron-style exam sheets), and more. Requires Ruby >=1.8.6 and Rubygems.
  • Link: ruql on RubyGems
  • Repository: armandofox/ruql
  • Author: Armando Fox
  • Date Contributed: 13-January-2014
  1. Presentation to edXML - https://github.com/edx/pres2edxml
  1. Clean Studio XML - https://github.com/pmitros/edxml-tools/blob/master/clean_studio_xml.py
  • Description: Converts ugly Studio edXML/OLX to pretty edXML/OLX. New XML imports just fine, but replaces Studio hashes with nice names, pretty-prints JSON, and merges many little XML files into a more manageable number of files. Note that url_names should not be changed mid-run (everything else it does is okay).
  • Link: https://github.com/pmitros/edxml-tools/blob/master/clean_studio_xml.py
  • Repository: pmitros
  • Author: http://mitros.org/p
  • Date Contributed: 10 Oct 2014
  1. Make course RSS - https://github.com/pmitros/edxml-tools/blob/master/make_course_rss.py
  1. Course Content Reader - https://github.com/zhenzhai/edX_CourseContentReader
  • Description: Generate a detail README.md file to describe the course content exported from edX.
  • Link: makeDoc.py
  • Repository: zhenzhai
  • Author: Zhen Zhai
  • Date Contributed: Sep 7, 2017
  1. Test Result Summarizer
  • Description: Summarizes the results of an edx-platform Jenkins test run.
  • Link: summarize_test_results.py
  • Repository: edx-tools
  • Author: @doctoryes
  • Date Contributed: Oct 3, 2017

#. Name of Tool - http://reposito.ry/path/to/script.py

  • Description: Short description of script
  • Link: script.py
  • Repository: Repository or hosting site
  • Author: Author's name, github handle, and/or website
  • Date Contributed: Date script was contributed or last updated

YELLOW -- Some Setup Required

  1. Course Parser - course.py - https://github.com/edx/edx-tools/blob/master/analytics/course_events/course.py
  • Description: A tool for interpreting course data from MongoDB
  • Link: course.py
  • Repository: edx/edx-tools
  • Author: David Glance (dglance)
  • Date Contributed: Aug 02, 2013
  1. Branch Drift - drift.py - https://github.com/edx/edx-tools/blob/master/branch-drift/drift.py
  • Description: Within a repo, measure how different two branches are.
  • Link: drift.py
  • Repository: edx/edx-tools
  • Author: Sef Kloninger
  • Date Contributed: 11-July-2013
  1. Transcoding helper utilities - https://github.com/edx/edx-tools/tree/master/transcoding
  • Description: Some simple tools to help drive Amazon's elastic transcoder to get video assets in the right format. Mostly just thin wrappers around boto.
  • listbucket.py - dump bucket contents
  • start-transcodes.py - given a set of videos, kick off a slew of transcoding jobs with the same settings
  • set-s3-metadata.py - to download correctly you need to set the content disposition for videos, otherwise they will be played in-browser. Given a set of videos, set the disposition for all
Unfortunately these have a lot of hardcoding in them, to use you'll need to edit the source. * Link: transcoding * Repository: edx/edx-tools * Author: Sef Kloninger * Date Contributed: 8-June-2013
  1. clean_cs_users.py - https://github.com/edx/edx-tools/blob/master/forums/clean_cs_users.py
  • Description: Administrative utility to remove invalid / unreachable user data from the comments service MongoDB instance
  • Link: clean_cs_users.py
  • Repository: edx/edx-tools
  • Author: Jim Abramson
  • Date Contributed: March 21, 2014
  1. edx_data_scripts - https://github.com/dcoetzee/edx_data_scripts
  • Description: Scripts to anonymize and import edX data dumps into MySQL/mongodb
  • Link: edx_data_scripts
  • Repository: dcoetzee/edx_data_scripts
  • Author: Derrick Coetzee
  • Date Contributed: April 2013
  1. MongoDB conversion scripts - https://github.com/McGillX/edx-data-research
  1. Course Export Broken Link Checker - https://github.com/McGillX/utilities/blob/master/scripts/broken_link_detector.py
  • Description: A script to check for dead hyperlinks and images within your course. Extract your *.tar.gz file and run the script specifying the path to your extracted folder.
  • Link: McGillX/broken_link_detector.py
  • Repository: McGillX
  • Author: McGillX, added by Pete Vungoc
  • Date Contributed: August 27, 2014
  1. Zeemaps Image Downloader - https://github.com/McGillX/utilities/blob/master/scripts/zeemaps_downloader.py
  • Description: Download all the images submitted by your students in a map on Zeemaps. Uses the *.csv file generated from Zeemaps to download all images on your computer.
  • Link: McGillX/zeemaps_downloader.py
  • Repository: McGillX
  • Author: McGillX, added by Pete Vungoc
  • Date Contributed: August 27, 2014
  1. Reporting Scripts- https://github.com/McGillX/edx-data-research
  1. OpenEdX Course Test - https://github.com/mitodl/openedx-course-test
  1. moocRP: Learning Analytics Platform - https://github.com/kk415kk/moocRP
  • Description: A Learning Analytics Platform from UC Berkeley, hosted from the Computational Approaches to Human Learning (CAHL) Research (cahlr) page.
  • Link: moocRP
  • Repository: moocRP
  • Author: Kevin Kao and Zachary Pardos
  • Date Contributed: August 6, 2015
  1. MOOCczar: A modular system for managing MOOC research data - https://github.com/UQ-UQx/MOOCczar
  • Description: MOOCczar: A Data Analytics Platform from the University of Queensland for managing MOOC research data. Includes documentation and a Course Report Generator.
  • Link: MOOCczar
  • Repository: https://github.com/UQ-UQx
  • Author: John Zornig and Aneesha Bakharia
  • Date Contributed: April 23, 2016
  1. ANALYSE: A learning analytics tool - https://github.com/analyseuc3m/ANALYSE-v1
  1. Data upload to a relational db-https://github.com/McGillX/C_sharp-code
  • Description: C# app to read the tracking logs (and the mongo discussion logs), parse the relevant lines, and upload to a relational db (mysql).
  • Link: Documentation
  • Repository: McGillX
  • Authors: McGillX, added by Melanie Lyman-Abramovitch
  • Date Contributed: January 12, 2017
  1. Name of Tool - http://reposito.ry/path/to/script.py
  • Description: Short description of script
  • Link: script.py
  • Repository: Repository or hosting site
  • Author: Author's name, github handle, and/or website
  • Date Contributed: Date script was contributed or last updated

RED -- Advanced

  1. Parse and Store edX Tracking Logs as MoocDB - https://github.com/paepcke/json_to_relation
  • Description: Set of python scripts to load edX tracking logs into a mysql database using the MOOCdb schema. Currently under active development so likely to change.
  • Link: json_to_relation
  • Repository: paepche/json_to_relation
  • Author: Andreas Paepcke
  • Date Contributed: ...
  1. edx-bundle - http://github.com/saasbook/edx-bundle
  • Description: Modify an edX course so videos can be streamed from local storage (vs. the Internet)
  • Link: edx-bundle
  • Repository: sassbook/edx-bundle
  • Author: Armando Fox
  • Date Contributed: May 20, 2014
  1. Copy and add your tool (RED) here - http://reposito.ry/path/to/script.py
  • Description: ...
  • Link: script.py
  • Repository: ...
  • Author: ...
  • Date Contributed: ...
  1. Name of Tool - http://reposito.ry/path/to/script.py
  • Description: Short description of script
  • Link: script.py
  • Repository: Repository or hosting site
  • Author: Author's name, github handle, and/or website
  • Date Contributed: Date script was contributed or last updated

Contributions

These wiki pages are public, and we welcome contributions from the broader community.

If you have created a tool that you would like to share, please feel free to publish it here. For more information, see How to Contribute.

You can share tools hosted in external repositories just by editing this page and adding it to the appropriate section (GREEN/YELLOW/RED).

If you have any questions, please contact Olga Stroilova.


See Also

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.