v0.11.0-RC4

Fixed Issues

• #2267 - Redirect to /ui/ doesn't work if Marathon is proxied
• #2254 - Exception after reregistration "A metric named ... already exists"

Changelog

Changelog from Marathon 0.11.0-RC3 to 0.11.0-RC4: v0.11.0-RC3...v0.11.0-RC4

Please see the release notes of v0.11.0-RC3 for a complete list of the changes in v0.11.0.

v0.11.0-RC3

Changes from 0.10.0 to 0.11.0-RC3

Breaking Changes

• Java 8 or higher is needed to run Marathon, since Java 6 and 7 support has reached end of life.
• --revive_offers_for_new_apps is now the default. If you want to avoid resetting filters if new tasks need to be started, you can disable this by --disable_revive_offers_for_new_apps.

Overview

Marathon uses Java 8

Java 8 has been out for more than a year now. The support for older versions of Java has reached end of life. We switched completely to the latest stable JVM release. We compile, test and run Marathon against Java 8.

Note for the Marathon Docker image: the base image of Marathon is now the standard java:8-jdk docker image.

Faster reconciliation and links to task Mesos sandboxes

Marathon now saves the slaveID of newly launched tasks. That leads to faster reconciliation for lost tasks and allows the UI to link to the mesos UI.

If the auto-discovered mesos UI is not reachable from wherever you access the Marathon UI from, you can use the --mesos_leader_ui_url configuration to change the base URL of these links.

New versioning information in API and UI

We now have versionInfo with lastConfigChangeAt and lastScalingAt in the apps JSON of our API. lastConfigChangeAt is the timestamp of the last change to the application that was not just a restart or a scaling operation. lastScalingAt is the time stamp of the last scaling or restart operation.

Additional task statistics in the /v2/apps and /v2/apps/{app id} endpoints (Optional)

If you pass embed=apps.taskStats/embed=app.taskStats as a query parameter, you get additional taskStats embedded in you app JSON.

Task statistics are provided for the following groups of tasks. If no tasks for the group exist, no statistics are offered:

• withLatestConfig contains statistics about all tasks that run with the same config as the latest app version.
• startedAfterLastScaling contains statistics about all tasks that were started after the last scaling or restart operation.
• withOutdatedConfig contains statistics about all tasks that were started before the last config change which was not simply a restart or scaling operation.
• totalSummary contains statistics about all tasks.

Example JSON:

{
  // ...
  "taskStats": {
    {
      "startedAfterLastScaling" : {
        "stats" : {
          "counts" : { // equivalent to tasksStaged, tasksRunning, tasksHealthy, tasksUnhealthy
            "staged" : 1,
            "running" : 100,
            "healthy" : 90,
            "unhealthy" : 4
          },
          // "lifeTime" is only included if there are running tasks.
          "lifeTime" : {
            // Measured from `"startedAt"` (timestamp of the Mesos TASK_RUNNING status update) of each running task
            // until now.
            "averageSeconds" : 20.0,
            "medianSeconds" : 10.0
          }
        }
      },
      "withLatestConfig" : {
        "stats" : { /* ... same structure as above ... */ }
      },
      "withOutdatedConfig" : {
        "stats" : { /* ... same structure as above ... */ }
      },
      "totalSummary" : {
        "stats" : { /* ... same structure as above ... */ }
      }
    }
  }
  // ...
}

The calculation of these statistics is currently performed for every request and expensive if you have very many tasks.

Specific "embed" parameters for app related information

In this release, we introduce "embed" parameters for all GET requests in the app end-point that specify which information to embed. Right now, we deliver all information by default that we delivered before but we encourage you to specify embed parameters for the information that you need.

In the future, we will only deliver information you explicitly requested by default, even though we will allow some compatibility configuration option. This improves performance by not returning information that you might not need.

Smarter resource offer handling

When Marathon needed to launch tasks for multiple apps, it would first launch all needed tasks for the first one and then proceed to the next app. One big deployment could block all others. In this new version, Marathon will spread the offers among all apps that currently need to launch tasks.

When Marathon detects new tasks that it has to launch, it will now explicitly request new offers with the reviveOffers Mesos scheduler API call. This should result in more speedy task launches. If for any reason you dislike this behavior, you can disable it with --disable_revive_offers_for_new_apps (not recommended).

The order in which mesos receives reviveOffers and declineOffer calls is not guaranteed. Therefore, as long as we still need offers to launch tasks, we repeat the reviveOffers call for --revive_offers_repetitions times so that our last reviveOffers will be received after all relevant declineOffer calls with high probability.

• --revive_offers_repetitions (Optional. Default: 3): Repeat every reviveOffer request this many times, delayed by the --min_revive_offers_interval.

When Marathon has no current use for an offer, it will decline the offer for a configurable period. A short duration might lead to resource starvation for other frameworks if you run many frameworks in your cluster. You should only need to reduce it if you use --disable_revive_offers_for_new_apps.

• --decline_offer_duration (Default: 120 seconds) The duration (milliseconds) for which to decline offers by default

New task launching tuning parameters

To prevent overloading Mesos itself, you can now restrict how many tasks Marathon launches per time interval. By default, we allow 1000 unconfirmed task launches every 30 seconds. In addition, Marathon considers TASK_RUNNING status updates from Mesos as launch confirmations and allows launching one more task for every confirmed launch.

• --launch_token_refresh_interval (Optional. Default: 30000): The interval (ms) in which to refresh the launch tokens to --launch_token_count.
• --launch_tokens (Optional. Default: 1000): Launch tokens per interval.

As a result of this, the --max_tasks_per_offer_cycle options is deprecated and has no effect anymore.

To prevent overloading Marathon and maintain speedy offer processing, there is also a timeout for matching each incoming resource offer, i.e. finding suitable tasks to launch for incoming offers.

• --offer_matching_timeout (Optional. Default: 1000): Offer matching timeout (ms). Stop trying to match additional tasks for this offer after this time. All already matched tasks are launched.

All launched tasks are stored before launching them. There is also a new timeout for this:

• --save_tasks_to_launch_timeout (Optional. Default: 3000): Timeout (ms) after matching an offer for saving all matched tasks that we are about to launch. When reaching the timeout, only the tasks that we could save within the timeout are also launched. All other task launches are temporarily rejected and retried later.

If the mesos master fails over or in other unusual circumstances, a launch task request might get lost. You can configure how long Marathon waits for the first TASK_STAGING update.

• --task_launch_confirm_timeout (Optional. Default: 10000): Time, in milliseconds, to wait for a task to enter the TASK_STAGING state before killing it.

No pseudo-deterministic assignment of host ports anymore

If you specify non-zero "ports" in your app JSON, they are used as service ports. The old code contained logic that would assign these ports as host ports if available in the processed offer. That misled people into thinking that these ports corresponded to host ports. The new code always randomizes host ports assignment without "requirePorts" or explicit "hostPort" configuration.

Last task failures are persisted

Marathon has exposed the "lastTaskFailure" via the API for a while but this information was not persisted across restarts or on fail over. Now it is.

Logging command line parameters on startup

Marathon will now log all command line parameters on startup. Example:

2015-08-02 11:48:08,047] INFO Starting Marathon 0.11.0-SNAPSHOT with --zk zk://master.mesos:2181/marathon --master master.mesos:5050

Improved logging of deployment plans

Example:

DeploymentPlan 2015-09-02T07:56:12.105Z
step 1:
  * Start(App(app1, cmd="cmd")), instances=0)
step 2:
  * Scale(App(app1, cmd="cmd")), instances=2)

Improved logging of offer rejections

Example for missing port:

[2015-08-04 20:11:20,510] INFO Offer [20150804-173037-16777343-5050-4340-O110]. Cannot find range with host port 8080 for app [/product/frontend]
[2015-08-04 20:11:29,422] INFO Offer [20150804-173037-16777343-5050-4340-O110]. Insufficient resources for [/product/frontend] (need cpus=1.0, mem=64.0, disk=1.0, ports=([8080] required + 1 dynamic), available in offer:
...

Example for missing basic resources:

[2015-08-04 20:17:27,986] INFO Offer [20150804-173037-16777343-5050-4340-O110]. Not all basic resources satisfied: cpu NOT SATISFIED (30.0 > 8.0), disk SATISFIED (0.0 <= 0.0), mem SATISFIED (16.0 <= 15360.0)
[2015-08-04 20:17:27,989] INFO Offer [20150804-173037-16777343-5050-4340-O110]. Insufficient for [/test] (need cpus=30.0, mem=16.0, disk=0.0, ports=(1 dynamic), available in offer:
...

Immediately store changed apps

In the past, Marathon would update the data for the group endpoint when it accepted a new deployment request like a change an app configuration. Marathon would only update the data in the app endpoint when it started the related deployment step. This could cause confusion for our users and thus we will now update both immediately when accepting the deployment.

No automatic reset of the app backoff when detecting a running task

When encountering task failures, Marathon uses the backoff duration to throttle launching task...

v0.10.1

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• #2029 Killing task not possible

Changelog

Changelog from Marathon 0.10.0 to 0.10.1: v0.10.0...v0.10.1

Please see the changelog for v0.10.0 for further information on the 0.10 release.

v0.10.1-RC2

This is a release candidate for a bug fix release. Thanks for your feedback!

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• #2029 Killing task not possible

v0.10.1-RC1

This is a release candidate for a bug fix release. Thanks for your feedback!

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• #1984 Creation of state path in Zookeeper fails when performed concurrently

v0.9.3-RC1

This is the release candidate for a bug fix release. Thanks for your feedback!

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• Creation of state path in Zookeeper fails when performed concurrently #1984

Changelog

Changelog from Marathon 0.9.2 to 0.9.3-RC1: v0.9.2...v0.9.3-RC1

v0.9.2

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• #1924 - Launch tasks that require no disk resource even if no disk resource is offered
• #1931 - Flags for reviveOffers and the duration for which to reject offers (to avoid resource starvation when running many frameworks)
• #1927 - Improve logging of unsatisfied resource requirements
• Fix showing Marathon logo in UI

Changelog

Changelog from Marathon 0.9.1 to 0.9.2: v0.9.1...v0.9.2

v0.10.0

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended
Mesos version for this release.

Notes

This will be the last release that supports Java 6, since Java 6 and Java 7 have reached End of Life.
Starting with 0.11.0 we will rely on Java 8.

Overview

This version corresponds to 0.10.0-RC5.

Administratively zeroing the task launch delay

It is now possible to reset the task launch delay via the REST endpoint /v2/queue/{app_id}/delay.
The UI displays two additional possible statuses: "Delayed" and "Waiting".
The "Delayed" status also displays a tooltip showing the remaining time until the next launch attempt.
The App page now allows the user to reset the task launch delay for a "Delayed" app, thus forcing a new immediate launch attempt.

The FrameworkId is invalidated, if the scheduler reports an error

If the scheduler reports an error, the framework id is removed from the backing persistence store.
This is important because on certain kinds of framework errors (such as exceeding the framework failover timeout),
the scheduler may never re-register with the saved FrameworkID until the leading Mesos master process is killed.

Maintain constraints while scaling down

Constraints in Marathon is a powerful mechanism.
While the constraints are maintained when scaling up, previous versions of Marathon did not do so while scaling down.
This version picks instances to kill based on defined constraints and will not violate those.

Custom prefix for automatically created environment variables

It is now possible to specify a custom prefix via the --env_vars_prefix
command line flag. This prefix will be added to the name of task's environment
variables created automatically by Marathon (e.g., PORT, PORTS)

Note: This prefix will not be added to variables that are already prefixed,
such as MESOS_TASK_ID and MARATHON_APP_ID

Handle requests with missing Accept and Content-Type headers correctly

If an HTTP request is made without setting appropriate request headers, the Accept and Content-Type header
is set automatically to application/json.

Option to restrict the number of concurrent HTTP requests

With this version of Marathon it is possible to restrict the number of concurrent HTTP requests,
that are handled concurrently in the service layer.
This enables the Marathon service to apply back pressure when receiving too many requests
and helps prevent Denial of Service attacks.
If the limit of concurrent requests is reached, a HTTP 503 Service Temporarily Unavailable is returned,
which means that this operation can be retried.
You can turn on this feature by setting the parameter --http_max_concurrent_requests.

Serialize all concurrent change requests

In former versions of Marathon it was possible that multiple concurrent changes of the same application definition
would lead to an inconsistent state of that application. All changes of groups or application definitions now
get serialized, so the resulting state is always consistent.

Enhance the reported metrics

The metrics endpoint (/metrics) gives valuable insight into the system behavior.
We now have meters for all service objects as well as the number applications and groups in the system.

Decouple API (un)marshalling types from internal models

With this change we can control API changes and are free to change internal models.
This will change nothing in the current API, but will help a lot while we create the v3 API.

Infrastructure

Added integration tests for all API endpoints

We now have an integration test suite, that covers all endpoints with almost all possible parameter combinations.
This test suite runs with every commit on every pull request.
This test suite is our safety net to ensure compatibility while introducing new functionality in future versions.

Code coverage reporting using coveralls

See our github page https://github.com/mesosphere/marathon or go directly to
https://coveralls.io/github/mesosphere/marathon to see our code coverage reporting.
With every Pull Request we try to make sure, this coverage will be increased.

Improved the release process of Marathon itself

Several changes are made to the sbt build file, but also to the infrastructure that is performing the release.
We now have a more reliable way of pushing distributions.

Fixed Issues

• #1710 - Too many simultaneous requests will kill Marathon
• #1709 - Concurrent changes to the same AppDefinition will lead to inconsistent state
• #1669 - Inconsistent content-type and response from REST API
• #1660 - The GUI doesn't allow creating apps with 0 instances, but it is possible through the v2 API
• #1647 - Keep the rest-api return format consistent when request headers without "Accept: application/json"
• #1397 - Kill tasks with scaling does not update group
• #1654 - Marathon 0.8.1 - API - /v2/tasks
• #555 - Constraints not satisfied when scaling down
• #650 - deployment_success/failed should contain deployment info
• #1316 - Invalidate framework ID if registration triggers an error
• #1254 - Add flag to define Marathon PORT prefix
• #1827 - Apps with multiple service ports fail to update
• #1829 - Marathon forgets value for acceptedResourceRoles when app is updated (including scaling)
• #1647 - Keep the rest-api return format consistent when request headers without "Accept: application/json"
• #1853 - App can't be deleted because it "does not exist", although /apps still returns it
• #1924 - Launch tasks that require no disk resource
• #1927 - Improve logging of unsatisfied resource requirements
• #1931 - Flag for reviveOffers and the duration for which to reject

Changelog

Changelog from Marathon 0.9.0 to 0.10.0: v0.9.0...v0.10.0

Commits	Contributor
47	Matthias Veit
29	Peter Kolloch
18	Gastón Kleiman
14	Gaston Kleiman
5	Matthias Eichstedt
3	Chris Darko
3	Cody Maloney
3	Connor Doyle
2	Lukas Loesche
2	Lukas Lösche
2	Pierluigi Cau
1	rfast
1	Arthur Barr
1	Christoph Neuroth
1	Dana Renfrow
1	Felix Gertz
1	Gastón Kleiman
1	Konstantin Baumann
1	Philip Norman
1	aameek
1	janisz
1	Artem Harutyunyan

generated by git shortlog -s -n v0.9.0..v0.10.0

v0.9.1

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended Mesos version for this release.

Fixed Issues

• #1829 - Marathon forgets value for acceptedResourceRoles when app is updated (including scaling).

Changelog

Changelog from Marathon 0.9.0 to 0.9.1: v0.9.0...v0.9.1

v0.10.0-RC4

Recommended Mesos version is 0.22.1

We tested this release against Mesos version 0.22.1. Thus, this is the recommended
Mesos version for this release.

Notes

For a complete list of the changes in v0.10.0 look here.

Fixed Issues

• #1647 - Keep the rest-api return format consistent when request headers without "Accept: application/json"
• #1853 - App can't be deleted because it "does not exist", although /apps still returns it

Changelog

Changelog from Marathon 0.10.0-RC3 to 0.10.0-RC4: v0.10.0-RC3...v0.10.0-RC4