tuulos edited this page Sep 12, 2010 · 14 revisions
Clone this wiki locally

Main components of the Disco core

As of Disco 0.3, migration to the architecture below is still work in progress. Especially master and event server still need refactoring.

  • Master disco_server.erl:
    • controls available resources (nodes)
    • maintains the global state of running workers
    • handles fault-tolerance
  • Scheduler fair_scheduler*.erl
    • controls priority of all jobs and tasks
    • controls data locality and load balancing
  • Event (log) server event_server.erl
    • controls global events and statistics
    • controls logs for archived jobs
  • Job coordinator job_coordinator.erl
    • controls that all tasks belonging to a job are executed
    • handles events related to a job
  • Worker supervisor disco_worker.erl
    • controls a single running task
  • Web API web_server.erl, disco_web.erl, mochiweb/*.erl
    • Maps requests from the Web API to internal requests
  • DDFS ddfs/*.erl
    • Storage subsystem