Scale 4.1.0

@dfaller dfaller released this Dec 5, 2016 · 187 commits to master since this release

Deprecated

  • The v3 version of the REST API is now deprecated. We recommend that you migrate all use of the Scale REST API to use the new v4 REST API. There are only a few changes between the v3 and v4 versions:
    1. The old log API /job-executions/{id}/logs/ is NOT available in v4 (returns 404).
    2. For the job details API, v4 does not include the deprecated "input_files" and "products" fields while v3 still does.
    3. For the recipe details API, v4 does not include the deprecated "input_files" field while v3 still does.

New Features

  • Scale now performs clean up of any Docker containers and volumes it creates. The Scale Cleanup system job has been removed. As Scale job executions finish, the scheduler tracks them and periodically submits clean up tasks that delete the containers and volumes associated with the finished executions. Also when a node is first registered with the scheduler, it goes into a state where a clean up task removes all non-running containers and dangling volumes before the node begins running Scale jobs. Issue 219

Enhancements/Updates

  • Batches can now be run on recipes based on input file data time (Issue 580)
  • Scale webserver logs are now captured (Issue 584)
  • Improved display and unit conversion of metric page total (Issue 563)
  • Improved table sorting on failure rates page (Issue 567)
  • Inactive jobs can now be filtered out of the REST API (Issue 604)
  • The ingest table can now be filtered by Strike (Issue 323)
  • Added link to job details on Strike detail page (Issue 571)
  • Added meta-data labels to the Scale Docker image (Issue 630)
  • Improved Scale's handling of lost Mesos tasks and ability to reconcile long-running tasks (Issue 629)

Bug Fixes

  • Fixed DC/OS health check for the Scale webserver (Issue 568)
  • Fixed REST API to work within the context of the DC/OS /service URL (Issue 569)
  • Fixed S3 Strike monitor to periodically reload configuration (Issue 617)
  • Fixed issue in batch creator job (Issue 603)
  • Fixed job type filter on failure rates page (Issue 566)
  • Fixed bug with querying ElasticSearch for job execution logs (Issue 633)
  • Fixed bug in handling empty S3 credential fields (Issue 615)
  • Fixed deployment bug where sometimes the Scale webserver or logstash apps would not destroy/re-create correctly (Issue 594)

Database Migrations

  • A new table called recipe_file is created to support Issue 580. It will be populated with the input files for each existing recipe, possibly taking a while to complete based upon how large your recipe table is.
  • Fields related to the old Scale clean up job are removed from the job_exe and job_type tables.
  • The maximum character length for fields within the task_update table are increased to 250.

Downloads