forked from apache/mesos
/
CHANGELOG
5680 lines (5155 loc) · 346 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release Notes - Mesos - Version 1.5.0 (WIP)
-------------------------------------------
This release contains the following new features:
* [MESOS-1739] - Agents now support the `--reconfiguration_policy`
flag which allows them to recover the agent ID and running tasks
after configuration changes. See docs/agent-recovery.md for more
details.
* [MESOS-7973] - Non-leading replica is now allowed to catch-up
missing log positions in the replicated log. This opens the door
for implementing hot standby (by offloading some reading from a
leader to standbys) and fast failover time (by keeping in-memory
storage represented by the log “hot”).
* Several improvements and fixes to the enforcement of quota
guarantees have been made: MESOS-4527, MESOS-7099, MESOS-8293,
MESOS-8339, MESOS-8352.
* [MESOS-7302] - Support launching standalone containers on the
agent using MesosContainerizer without a master or framework
running.
* [MESOS-7289, MESOS-7235] - **Experimental* support for Container
Storage Interface (CSI) to simplify storage management in Mesos,
and allow 3rdparty vendors to plugin into Mesos very easily.
* [MESOS-4945] Agents now can automatically garbage collect unused
Docker image layers used by MesosContainerizer.
Deprecations/Removals:
* Agent flag `--executor_secret_key` has been deprecated. Operators
should use `--jwt_secret_key` instead.
Additional API Changes:
* [MESOS-7660] `Filter::refuse_seconds` is now capped to 31536000
seconds (365 days).
* [MESOS-7941] Built-in executors will now send a TASK_STARTING
status update when a task is starting.
* [MESOS-8165] Master will now send TASK_GONE status for unknown
tasks of PARTITION_AWARE frameworks belonging to registered agents
during explicit reconciliation.
* [MESOS-7550] The fields `Resource.disk.source.path.root` and
`Resource.disk.source.mount.root` can now be set to relative paths
to an agent's work directory.
* [MESOS-8040] Return nested/standalone containers in `GET_CONTAINERS`
API call.
* [MESOS-7973] A new `catchup` method has been added to the
`Log.Reader` interface (including Java binding).
Unresolved Critical Issues:
Feature Graduations:
* v1 Operator API is now considered stable.
All Experimental Features:
Release Notes - Mesos - Version 1.4.2 (WIP)
-------------------------------------------
* This is a bug fix release.
** Bug
* [MESOS-7975] - The command/default/docker executor can incorrectly send a TASK_FINISHED update even when the task is killed.
* [MESOS-8159] - ns::clone uses an async signal unsafe stack.
* [MESOS-8237] - Strip (Offer|Resource).allocation_info for non-MULTI_ROLE schedulers.
* [MESOS-8297] - Built-in driver-based executors ignore kill task if the task has not been launched.
* [MESOS-8356] - Persistent volume ownership is set to root despite of sandbox owner (frameworkInfo.user) when docker executor is used.
Release Notes - Mesos - Version 1.4.1
-------------------------------------------
* This is a bug fix release.
** Bug
* [MESOS-7873] - Expose `ExecutorInfo.ContainerInfo.NetworkInfo` in Mesos `state` endpoint.
* [MESOS-7921] - ProcessManager::resume sometimes crashes accessing EventQueue.
* [MESOS-7964] - Heavy-duty GC makes the agent unresponsive.
* [MESOS-7968] - Handle `/proc/self/ns/pid_for_children` when parsing available namespace.
* [MESOS-7969] - Handle cgroups v2 hierarchy when parsing /proc/self/cgroups.
* [MESOS-7980] - Stout fails to compile with libc >= 2.26.
* [MESOS-8051] - Killing TASK_GROUP fail to kill some tasks.
* [MESOS-8080] - The default executor does not propagate missing task exit status correctly.
* [MESOS-8090] - Mesos 1.4.0 crashes with 1.3.x agent with oversubscription
* [MESOS-8135] - Masters can lose track of tasks' executor IDs.
* [MESOS-8169] - Incorrect master validation forces executor IDs to be globally unique.
Release Notes - Mesos - Version 1.4.0
-------------------------------------------
This release contains the following new features:
* [MESOS-5116] - The `disk/xfs` isolator now supports the
`--enforce_container_disk_quota` flag to efficiently measure disk
usage without enforcing usage constraints.
* [MESOS-6223] - Agents are now allowed to recover the agent ID
after a host reboot. See docs/upgrades.md for details.
* [MESOS-6375] - **Experimental** Support for hierarchical resource
allocation roles. Hierarchical roles allows delegation of resource
allocation policies (i.e. fair sharing and quota) further down the
hierarchy. For example, the "engineering" organization gets a 75%
share of the resources, but it's up to the operators within the
"engineering" organization to figure out how to fairly share between
the "engineering/backend" team and the "engineering/frontend" team.
The same delegation applies for quota. NOTE: There are known issues
related to hierarchical roles (e.g. hierarchical quota allocation
is not implemented and quota will be over-allocated if used with
hierarchical roles, see: MESOS-7402) and thus it is not recommended
for production usage at this time.
* [MESOS-7418, MESOS-7088] - File-based secrets are now supported for Mesos
and Universal containerizer. Image-pull secrets are supported for Docker
registry credentials.
* [MESOS-7477] - Linux ambient capabilites are now supported, so
frameworks can run tasks that use ambient capabilites to grant
limited additional privileged to tasks.
* [MESOS-7476, MESOS-7671] - Support for frameworks and operators
specifying Linux bounding capabilities in order to limit the
maximum privileges that a task may acquire.
Deprecations/Removals:
* [MESOS-7671] - LinuxInfo.capabilities is deprecated in favor
of LinuxInfo.effective_capabilities.
* [MESOS-7477] - The agent `--allowed_capabilities` flag is
deprecated in favor of `--effective_capabilities`
Unresolved Critical Issues:
* [MESOS-7643] - The order of isolators provided in '--isolation' flag is not preserved and instead sorted alphabetically
* [MESOS-7402] - Quota is over-allocated when used with hierarchical roles.
Additional API Changes:
* [MESOS-7755] The interpretation of the optional resource argument
passed in `Allocator::updateSlave` was changed from the total
amount of oversubscribed resources on the agent to the new total
resources (both revocable and non-revocable) on the agent. Custom
allocator implementation should be changed to interpretation of the
passed value as a total before updating.
Feature Graduations:
* [MESOS-2533] - Support HTTP checks in Mesos.
* [MESOS-3567] - Support TCP checks in Mesos.
All Resolved Issues:
** Bug
* [MESOS-1987] - Add support for SemVer build and prerelease labels to stout.
* [MESOS-4210] - Investigate increasing protobuf protocol message size limit.
* [MESOS-4331] - git commit-msg hook completely breaks fixup commits.
* [MESOS-4467] - Implement `sleep` in Windows
* [MESOS-4983] - Segfault in ProcessTest.Spawn with GCC 6
* [MESOS-4992] - sandbox uri does not work outisde mesos http server
* [MESOS-5187] - The filesystem/linux isolator does not set the permissions of the host_path.
* [MESOS-5903] - `GTEST_IS_THREADSAFE` guards prevent many tests from being run on Windows.
* [MESOS-5937] - `flags::parse` assumes the filesystem is rooted at '/'
* [MESOS-5938] - `net::links` is not implemented on Windows.
* [MESOS-6115] - Source tree contains compiled protobuf source
* [MESOS-6539] - Compile warning in GMock: "binding dereferenced null pointer to reference"
* [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
* [MESOS-6814] - Make sure compilation configuration is propagated correctly to third party dependencies
* [MESOS-6817] - Audit the use of UNICODE-related code paths
* [MESOS-6916] - Improve health checks validation.
* [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up
* [MESOS-6961] - Executors don't use glog for logging.
* [MESOS-7017] - HTTP API responses can crash the master.
* [MESOS-7115] - Agent should prefer LOG(FATAL) over EXIT().
* [MESOS-7173] - CMake does not define `GIT_SHA` etc. in build.cpp
* [MESOS-7186] - Metrics about used/allocated shared resources are incorrect accounted.
* [MESOS-7193] - Use of `GTEST_IS_THREADSAFE` in asserts is problematic.
* [MESOS-7252] - Need to fix resource check in long-lived framework
* [MESOS-7268] - CNI isolator should mount network related /etc/* files in readonly mode
* [MESOS-7351] - CMake < 3.8.0 cannot find VS2017 tools
* [MESOS-7373] - Remove thread_local workaround on OSX
* [MESOS-7374] - Running DOCKER images in Mesos Container Runtime without `linux/filesystem` isolation enabled renders host unusable
* [MESOS-7378] - Build failure with glibc 2.12.
* [MESOS-7381] - Flaky tests in NestedMesosContainerizerTest
* [MESOS-7389] - Mesos 1.2.0 crashes with pre-1.0 Mesos agents.
* [MESOS-7403] - Resources::apply(const Offer::Operation&) should fail when a shared persistent volume can't be removed
* [MESOS-7441] - RegisterSlaveValidationTest.DropInvalidRegistration is flaky
* [MESOS-7457] - HierarchicalAllocatorTest.NestedRoleQuota is flaky
* [MESOS-7458] - webui display of framework resources is confusing
* [MESOS-7459] - Fix the duration.hpp warning
* [MESOS-7462] - Flaky test HierarchicalAllocatorTest.NestedRoleDRF
* [MESOS-7464] - Recent Docker versions cannot be parsed by stout.
* [MESOS-7468] - Could not copy the sandbox path on WebUI
* [MESOS-7471] - Provisioner recover should not always assume 'rootfses' dir exists.
* [MESOS-7476] - Restrict capabilities to only the bounding set.
* [MESOS-7484] - VersionTest.ParseInvalid aborts on Windows.
* [MESOS-7496] - The /debug:fastlink linker option is not being respected
* [MESOS-7498] - Remove need to set environment variable `PreferredToolArchitecture`
* [MESOS-7502] - Build error on Windows when using "int" for a file descriptor
* [MESOS-7507] - Add a metric for the network size of replicas for the registry.
* [MESOS-7515] - MasterAllocatorTest/0.ResourcesUnused is flaky
* [MESOS-7524] - Basic fetcher success metrics
* [MESOS-7545] - Volume secret isolator breaks Windows build
* [MESOS-7552] - MasterAllocatorTest/0.FrameworkExited is flaky
* [MESOS-7569] - Allow "old" executors with half-open connections to be preserved during agent upgrade / restart.
* [MESOS-7581] - Specifying an unbundled dependency can cause build to pick up wrong Boost version
* [MESOS-7584] - ASF Jenkins build errors out on missing 'python-six' dependency
* [MESOS-7597] - libprocess build is broken
* [MESOS-7618] - CMake files incompatible with multi-configuration generators
* [MESOS-7627] - Mesos slave stucks
* [MESOS-7638] - The command `false` does not exist on Windows
* [MESOS-7640] - Docker containerizer fails to set sandbox logs ownership correctly.
* [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
* [MESOS-7655] - Reservation Refinement: Update the resources logic.
* [MESOS-7662] - Documentation regarding TASK_LOST is misleading
* [MESOS-7666] - Update the agent to use the new resource format
* [MESOS-7667] - Update the master to use the new resource format.
* [MESOS-7669] - Update the test utilities to produce the resources in the new format
* [MESOS-7671] - Let frameworks specify the task bounding capabilities.
* [MESOS-7674] - Update the generic Protobuf to JSON facility to not output deprecated fields
* [MESOS-7679] - V1 Operator API update for reservation refinement.
* [MESOS-7689] - Libprocess can crash on malformed request paths for libprocess messages.
* [MESOS-7690] - The agent can crash when an unknown executor tries to register.
* [MESOS-7700] - Prevent reserve/create operations with refined reservations on non-capable agents.
* [MESOS-7703] - Mesos fails to exec a custom executor when no shell is used
* [MESOS-7711] - Master updates registry for reregistering agents even when they haven't been unreachable
* [MESOS-7714] - Fix agent downgrade for reservation refinement
* [MESOS-7716] - Mesos 1.2.0 agent crashes Mesos 1.4.0 master
* [MESOS-7725] - PersistentVolumeEndpointsTest.ReserveAndSlaveRemoval test is flaky
* [MESOS-7728] - Java HTTP adapter crashes JVM when leading master disconnects.
* [MESOS-7735] - The master crashes when state endpoint is hit during a task authorization.
* [MESOS-7744] - Mesos Agent Sends TASK_KILL status update to Master, and still launches task
* [MESOS-7751] - Mesos failed to build on Windows due to error C2039: 'parse': is not a member of 'mesos::internal::protobuf'
* [MESOS-7753] - `log.LearnedMessage` could be rejected due to being sent from '@0.0.0.0:0'
* [MESOS-7758] - Stout doesn't build standalone.
* [MESOS-7761] - Website ruby deps do not bundle on macOS
* [MESOS-7765] - MasterTest.KillUnknownTask is failling due to a bug in `net::IPv4::ANY()`
* [MESOS-7769] - libprocess initializes to bind to random port if --ip is not specified
* [MESOS-7770] - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.
* [MESOS-7772] - Copy-n-paste error in slave/main.cpp
* [MESOS-7775] - Eliminate extra process abort in a subprocess watchdog
* [MESOS-7777] - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13
* [MESOS-7778] - Hide per-platform subprocess headers.
* [MESOS-7783] - Framework might not receive status update when a just launched task is killed immediately
* [MESOS-7794] - Mesos failed with error c2102 when build in conformance mode (/permissive-)
* [MESOS-7796] - LIBPROCESS_IP isn't passed on to the fetcher
* [MESOS-7797] - Hard-coded forward slash breaks windows docker container task in DC/OS
* [MESOS-7805] - mesos-execute has incorrect example TaskInfo in help string
* [MESOS-7817] - CreateProcess wrapper's error message is bad
* [MESOS-7821] - Resource refinement does downgrade task.executor.resources in LAUNCH_GROUP handler.
* [MESOS-7830] - Sandbox_path volume does not have ownership set correctly.
* [MESOS-7831] - Resource refinement is not applied to tasks in completed_frameworks.
* [MESOS-7849] - The rlimits and linux/capabilities isolators should support nested containers
* [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer
* [MESOS-7863] - Agent may drop pending kill task status updates.
* [MESOS-7865] - Agent may process a kill task and still launch the task.
* [MESOS-7869] - Build fails with `--disable-zlib` or `--with-zlib=DIR`
* [MESOS-7871] - Agent fails assertion during request to '/state'
* [MESOS-7872] - Scheduler hang when registration fails.
* [MESOS-7888] - Track fetcher task success and failures
* [MESOS-7909] - Ordering dependency between 'linux/capabilities' and 'docker/runtime' isolator.
* [MESOS-7912] - Master WebUI not working in Chrome.
* [MESOS-7921] - process::EventQueue sometimes crashes
* [MESOS-7922] - Fix communication between old masters and new agents.
* [MESOS-7926] - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
* [MESOS-7934] - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.
** Documentation
* [MESOS-7246] - Add documentation for AGENT_ADDED/AGENT_REMOVED events.
* [MESOS-7349] - Document Mesos "check" feature.
* [MESOS-7501] - Change legacy --with-network-isolator to --with-port-mapping-isolator
** Epic
* [MESOS-6975] - Prevent pre-1.0 agents from registering with 1.3+ master.
* [MESOS-7088] - Support private registry credential per container.
* [MESOS-7623] - Automatically publish website through CI
** Improvement
* [MESOS-5116] - Investigate supporting accounting only mode in XFS isolator
* [MESOS-5417] - define WSTRINGIFY behaviour on Windows
* [MESOS-6053] - Combine test helpers into one single binary.
* [MESOS-6223] - Allow agents to re-register post a host reboot
* [MESOS-6535] - The default executor should support kill policies
* [MESOS-6549] - Asynchronous dir removal in agent GC
* [MESOS-6782] - Inherit Environment from parent container when launching DEBUG container.
* [MESOS-6905] - Task status updates caused by task health update do not set appropriate reason.
* [MESOS-6976] - Disallow (re-)registration attempts by old agents.
* [MESOS-6977] - Cleanup tech debt in master for old agents
* [MESOS-6978] - Update webui to remove orphan tasks
* [MESOS-7006] - Launch docker containers with --cpus instead of cpu-shares
* [MESOS-7015] - Frameworks should be able to (re)register in suppressed state
* [MESOS-7092] - Health checker duplicates a lot of checker's functionality.
* [MESOS-7228] - Upgrade Mesos to build with proto3.
* [MESOS-7327] - Add a test with multiple tasks and checks for the default executor.
* [MESOS-7343] - Add a ReviewBot for testing patches on Windows
* [MESOS-7355] - Set MESOS_SANDBOX in debug containers.
* [MESOS-7364] - Upgrade vendored GMock / GTest
* [MESOS-7401] - Optionally reject messages when UPIDs does not match IP.
* [MESOS-7418] - Add support for file-based secrets
* [MESOS-7429] - Allow isolators to inject task-specific environment variables.
* [MESOS-7451] - Expose MOUNT volumes of an agent in master's v0 HTTP API
* [MESOS-7477] - Support ambient capabilities.
* [MESOS-7540] - Add an agent flag for executor re-registration timeout.
* [MESOS-7542] - Add executor reconnection retry logic to the agent
* [MESOS-7572] - Attach latest symlink when executor is registered.
* [MESOS-7585] - Added 'mesos config show' command to the new Mesos CLI.
* [MESOS-7608] - Protobuf definitions for domains
* [MESOS-7609] - Protobuf definitions for region-aware framework capability
* [MESOS-7610] - Support domains in master and agent
* [MESOS-7611] - Prevent master from joining mixed-region cluster
* [MESOS-7612] - Prevent agent with misconfigured domain from registering
* [MESOS-7614] - Only offer resources on remote agents to region-aware frameworks
* [MESOS-7630] - Add simple filtering to unversioned operator API
* [MESOS-7644] - Add DomainInfo to offers
* [MESOS-7782] - Add fetcher cache size metrics.
* [MESOS-7792] - Add support for ECDH ciphers
* [MESOS-7808] - Bundling gRPC into 3rdparty
* [MESOS-7809] - Building gRPC with Autotools
* [MESOS-7810] - gRPC support in libprocess
* [MESOS-7814] - Improve the test frameworks.
* [MESOS-7862] - Get rid of timestamp and date in generated javadoc files
* [MESOS-7870] - Refactor libssl and libcrypto checks for building gRPC
* [MESOS-7881] - Building gRPC with CMake
** Task
* [MESOS-6101] - Add Framwork events to master's operator API
* [MESOS-6162] - Add support for cgroups blkio subsystem blkio statistics.
* [MESOS-6441] - Display reservations in the agent page in the webui.
* [MESOS-7149] - Support reservations for role subtrees
* [MESOS-7283] - Add ability to initialize a test cluster for Mesos CLI unit-test infrastructure
* [MESOS-7304] - Fetcher should not depend on SlaveID.
* [MESOS-7315] - Design doc for resource provider and storage integration.
* [MESOS-7414] - Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
* [MESOS-7415] - Add authorization to master's operator maintenance API in v0 and v1
* [MESOS-7416] - Filter results of `/master/slaves` and the v1 call GET_AGENTS
* [MESOS-7417] - Design doc for file-based secrets.
* [MESOS-7433] - Set working directory in DEBUG containers.
* [MESOS-7449] - Refactor containerizers to not depend on TaskInfo or ExecutorInfo
* [MESOS-7488] - Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
* [MESOS-7505] - Enable hierarchical roles
* [MESOS-7560] - Add 'type' and 'name' to ResourceProviderInfo.
* [MESOS-7571] - Add `--resource_provider_config_dir` flag to the agent.
* [MESOS-7576] - Add master flag `--filter-gpu-resources={true|false}`
* [MESOS-7582] - Add Config class to manage the Mesos CLI config file.
* [MESOS-7591] - Update master to use resource provider IDs instead of agent ID in allocator calls.
* [MESOS-7593] - Update offer handling in the master to consider local resource providers
* [MESOS-7624] - Move website from svn to git
* [MESOS-7625] - Create script to automate publishing website
* [MESOS-7626] - Create a CI job to publish the website
* [MESOS-7631] - DefautlExecutor needs to inform tasks about IP addresses
* [MESOS-7632] - Add `HIERARCHICAL_ROLE` agent capability
* [MESOS-7633] - Prevent hierarchical roles from being allocated resources from non-HIERARCHICAL_ROLE agents.
* [MESOS-7665] - V0 Operator API update for reservation refinement.
* [MESOS-7668] - Update authorization to handle reservation refinement.
* [MESOS-7696] - Update resource provider design in the master
* [MESOS-7709] - Add --default_container_dns flag to the agent.
* [MESOS-7713] - Optimize number of copies made in dispatch/defer mechanism
* [MESOS-7755] - Update allocator to support updating agent total resources
* [MESOS-7757] - Update master to handle updates to agent total resources
* [MESOS-7767] - Make `net::IP` fields protected to allow for inheritance
* [MESOS-7780] - Add `SUBSCRIBE` call handling to the resource provider manager
* [MESOS-7806] - Add copy assignment operator to `net::IP::Network`
* [MESOS-7853] - Support shared PID namespace.
* [MESOS-7879] - The kill nested container call should provide ability to specify a signal.
Release Notes - Mesos - Version 1.3.2 (WIP)
-------------------------------------------
* This is a bug fix release.
All Issues:
** Bug
* [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
* [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
* [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
* [MESOS-7674] - Update the generic Protobuf to JSON facility to not output deprecated fields.
* [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
* [MESOS-7863] - Agent may drop pending kill task status updates.
* [MESOS-7865] - Agent may process a kill task and still launch the task.
* [MESOS-7872] - Scheduler hang when registration fails.
* [MESOS-7909] - Ordering dependency between 'linux/capabilities' and 'docker/runtime' isolator.
* [MESOS-7912] - Master WebUI not working in Chrome.
* [MESOS-7926] - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
* [MESOS-7934] - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.
* [MESOS-8135] - Masters can lose track of tasks' executor IDs.
* [MESOS-8237] - Strip (Offer|Resource).allocation_info for non-MULTI_ROLE schedulers.
* [MESOS-8356] - Persistent volume ownership is set to root despite of sandbox owner (frameworkInfo.user) when docker executor is used.
Release Notes - Mesos - Version 1.3.1
-------------------------------------
* This is a bug fix release.
All Issues:
** Bug
* [MESOS-5187] - The filesystem/linux isolator does not set the permissions of the host_path.
* [MESOS-7252] - Need to fix resource check in long-lived framework.
* [MESOS-7429] - Allow isolators to inject task-specific environment variables.
* [MESOS-7540] - Add an agent flag for executor re-registration timeout.
* [MESOS-7546] - WAIT_NESTED_CONTAINER sometimes returns 404.
* [MESOS-7569] - Allow "old" executors with half-open connections to be preserved during agent upgrade / restart.
* [MESOS-7581] - Fix interference of external Boost installations when using some unbundled dependencies.
* [MESOS-7689] - Libprocess can crash on malformed request paths for libprocess messages.
* [MESOS-7690] - The agent can crash when an unknown executor tries to register.
* [MESOS-7692] - Default environment variables defined in Docker image are not available in Mesos containerizer.
* [MESOS-7703] - Mesos fails to exec a custom executor when no shell is used.
* [MESOS-7728] - Java HTTP adapter crashes JVM when leading master disconnects.
* [MESOS-7770] - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.
* [MESOS-7777] - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13.
* [MESOS-7796] - LIBPROCESS_IP isn't passed on to the fetcher.
* [MESOS-7830] - Sandbox_path volume does not have ownership set correctly.
Release Notes - Mesos - Version 1.3.0
-------------------------------------
This release contains the following new features:
* [MESOS-1763] - Support for frameworks to receive resources for multiple
roles. This allows "multi-user" frameworks to leverage the role-based
resource allocation in mesos. Prior to this support, one had to run
multiple instances of a single-user framework to achieve multi-user
resource allocation, or implement multi-user resource allocation in
the framework.
* [MESOS-6365] - Authentication and authorization support for HTTP executors.
A new `--authenticate_http_executors` agent flag enables required
authentication on the HTTP executor API. A new `--executor_secret_key` flag
sets a key file to be used when generating and authenticating default tokens
that are passed to HTTP executors. Note that enabling these flags after
upgrade is disruptive to HTTP executors that were launched before the
upgrade; see 'docs/authentication.md' for more information on these flags
and the recommended upgrade procedure. Implicit authorization rules have
been added which allow an authenticated executor to make executor API calls
as that executor and make operator API calls which affect that executor's
container. See 'docs/authorization.md' for more information on these
implicit authorization rules.
* [MESOS-6627] - Support for frameworks to modify the role(s) they are
subscribed to. This is essential to supporting "multi-user" frameworks
(see MESOS-1763) in that roles are expected to come and go over time
(e.g. new employees join, new teams are formed, employees leave, teams
are disbanded, etc).
**NOTE**: In Mesos 1.3.0, the master will no longer allow 0.x agents to
register. Interoperability between 1.1+ masters and 0.x agents has never
been supported; however, it was not explicitly disallowed, either.
Starting with this release of Mesos, registration attempts by 0.x Mesos
agents will be ignored.
Deprecations/Removals:
* [MESOS-7259] - Remove deprecated ACLs `SetQuota` and `RemoveQuota`.
This change is only applicable to the local authorizer since internally
these acls were being translated to the `UPDATE_QUOTA` action.
* [MESOS-7320] - Remove deprecated ACL `ShutdownFramework`.
This change is only applicable to the local authorizer since internally
these acls were being translated to the `TEARDOWN_FRAMEWORK` action.
Unresolved Critical Issues:
* [MESOS-1625] - Extra trailing CRLF being sent after the HTTP body in libprocess.
* [MESOS-1718] - Command executor can overcommit the agent.
* [MESOS-2554] - Slave flaps when using --slave_subsystems that are not used for isolation.
* [MESOS-2774] - SIGSEGV received during process::MessageEncoder::encode().
* [MESOS-2842] - Update FrameworkInfo.principal on framework re-registration.
* [MESOS-3533] - Unable to find and run URIs files.
* [MESOS-3747] - HTTP Scheduler API no longer allows FrameworkInfo.user to be empty string.
* [MESOS-3794] - Master should not store arbitrarily sized data in ExecutorInfo.
* [MESOS-4259] - mesos HA can't delete the the redundant container on failure slave node.
* [MESOS-4297] - Executor does not shutdown when framework teardown.
* [MESOS-4642] - Mesos Agent Json API can dump binary data from log files out as invalid JSON.
* [MESOS-4996] - 'containerizer->update' will always fail after killing a docker container.
* [MESOS-5352] - Docker volume isolator cleanup can be blocked by first cleanup failure.
* [MESOS-5396] - After failover, master does not remove agents with same UPID.
* [MESOS-5849] - Agent sandboxes on Windows surpass the 260 character path length limit.
* [MESOS-5859] - Some tasks are always in staged state.
* [MESOS-5989] - Libevent SSL Socket downgrade code accesses uninitialized memory / assumes single peek is sufficient.
* [MESOS-5995] - Protobuf JSON deserialisation does not accept numbers formated as strings.
* [MESOS-6356] - ASF CI has interleaved logging.
* [MESOS-6615] - Running mesos-slave in the docker that leave many zombie process.
* [MESOS-6623] - Re-enable tests impacted by request streaming support.
* [MESOS-6632] - ContainerLogger might leak FD if container launch fails.
* [MESOS-6780] - ContentType/AgentAPIStreamingTest.AttachContainerInput test fails reliably.
* [MESOS-6784] - IOSwitchboardTest.KillSwitchboardContainerDestroyed is flaky.
* [MESOS-6804] - Running 'tty' inside a debug container that has a tty reports "Not a tty".
* [MESOS-6843] - Fetcher should not assume stdout/stderr in the sandbox.
* [MESOS-6913] - AgentAPIStreamingTest.AttachInputToNestedContainerSession fails on Mac OS.
* [MESOS-6974] - DefaultExecutorTest.CommitSuicideOnTaskFailure test is flaky.
* [MESOS-6986] - `abort` in `DRFSorter::add`.
* [MESOS-7017] - HTTP API responses can crash the master.
* [MESOS-7082] - ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.KillTask/0 is flaky.
* [MESOS-7099] - Quota can be exceeded due to coarse-grained offer technique.
* [MESOS-7215] - Race condition on re-registration of non-partition-aware frameworks.
* [MESOS-7298] - Fetcher caches files with world-readable permissions.
* [MESOS-7362] - GPU support can't work when run spark.
* [MESOS-7374] - Running DOCKER images in Mesos Container Runtime without `linux/filesystem` isolation enabled renders host unusable.
* [MESOS-7381] - Flaky tests in NestedMesosContainerizerTest.
* [MESOS-7386] - Executor not cleaning up existing running docker containers if external logrotate/logger processes die/killed.
Feature Graduations:
* [MESOS-2449] - Support group of tasks (Pod) constructs and API in Mesos.
* [MESOS-4641] - Support Container Network Interface (CNI).
* [MESOS-6419] - Teardown unregistered frameworks.
All Experimental Features:
* [MESOS-2533] - Support HTTP checks in Mesos.
* [MESOS-3094] - Mesos on Windows.
* [MESOS-3421] - Support sharing of resources across task instances.
* [MESOS-3567] - Support TCP checks in Mesos.
* [MESOS-4312] - Porting Mesos on Power (ppc64le).
* [MESOS-4355] - Implement isolator for Docker volume.
* [MESOS-4791] - Operator API v1.
* [MESOS-4828] - XFS disk quota isolator.
* [MESOS-5275] - Add capabilities support for mesos containerizer.
* [MESOS-5344] - Partition-aware Mesos frameworks.
* [MESOS-5788] - Added JAVA API adapter for seamless transition to new scheduler API.
* [MESOS-5931] - Support auto backend in Mesos Containerizer.
* [MESOS-6014] - Added port mapping CNI plugin.
* [MESOS-6077] - Added a default (task group) executor.
* [MESOS-6402] - rlimit support for Mesos containerizer.
* [MESOS-6460] - Container Attach/Exec.
* [MESOS-6758] - Support docker registry that requires basic auth.
* [MESOS-6906] - Introduce a general non-interpreting task check.
All Resolved Issues:
** Bug
* [MESOS-1987] - Add support for SemVer build and prerelease labels to stout.
* [MESOS-4245] - Add `dist` target to CMake solution.
* [MESOS-4263] - Report volume usage through ResourceStatistics.
* [MESOS-5028] - Copy provisioner cannot replace directory with symlink.
* [MESOS-5172] - Registry puller cannot fetch blobs correctly from http Redirect 3xx urls.
* [MESOS-5288] - Update leveldb patch file to suport s390x.
* [MESOS-5880] - Semantics of `environment` differ across Windows and POSIX.
* [MESOS-6134] - Port CFS quota support to Docker Containerizer using command executor.
* [MESOS-6138] - Add 'syntax=proto2' to all .proto files in Mesos.
* [MESOS-6327] - Large docker images causes container launch failures: Too many levels of symbolic links.
* [MESOS-6560] - The default stout stringify always copies its argument.
* [MESOS-6606] - Reject optimized builds with libcxx before 3.9.
* [MESOS-6720] - Check that `PreferredToolArchitecture` is set to `x64` on Windows before building.
* [MESOS-6730] - Reserve operation should validate reserved resource role against resource allocationInfo role.
* [MESOS-6731] - Create a test filter for stout tests that use `symlink` on Windows, as they will fail if not run as admin.
* [MESOS-6732] - XFS disk isolator should check whether quotas are enabled.
* [MESOS-6742] - Adding support for s390x architecture.
* [MESOS-6815] - Enable glog stack traces when we call things like `ABORT` on Windows.
* [MESOS-6858] - network/cni isolator generates incomplete resolv.conf.
* [MESOS-6868] - Transition Windows away from `os::killtree`.
* [MESOS-6892] - Reconsider process creation primitives on Windows.
* [MESOS-6907] - FutureTest.After3 is flaky.
* [MESOS-6951] - Docker containerizer: mangled environment when env value contains LF byte.
* [MESOS-6953] - A compromised mesos-master node can execute code as root on agents.
* [MESOS-6976] - Disallow (re-)registration attempts by old agents.
* [MESOS-6982] - PerfTest.Version fails on recent Arch Linux.
* [MESOS-7022] - Update framework authorization to support multiple roles.
* [MESOS-7029] - FaultToleranceTest.FrameworkReregister is flaky.
* [MESOS-7035] - Add test for framework upgrading to MULTI_ROLE with tasks running.
* [MESOS-7049] - CgroupsAnyHierarchyWithPerfEventTest.ROOT_CGROUPS_PERF_PerfTest is broken on Fedora 25.
* [MESOS-7097] - Framework credentials can be used to register as an agent.
* [MESOS-7133] - mesos-fetcher fails with openssl-related output.
* [MESOS-7135] - Outstanding offers to a dropped framework role should be rescinded.
* [MESOS-7146] - OSX broken due to wrong configuration of LevelDB after update.
* [MESOS-7158] - Add `role` to task/executor to indicate allocation role of their resources.
* [MESOS-7165] - Agents should be able to upgrade to be MULTI_ROLE capable.
* [MESOS-7172] - CMake does not incrementally recompile.
* [MESOS-7182] - Couple of MULTI_ROLE related tests are flaky.
* [MESOS-7197] - Requesting tiny amount of CPU crashes master.
* [MESOS-7208] - Persistent volume ownership is set to root when task is running with non-root user.
* [MESOS-7210] - HTTP health check doesn't work when mesos runs with --docker_mesos_image.
* [MESOS-7225] - Tasks launched via the default executor cannot access disk resource volumes.
* [MESOS-7236] - Base64 encoding/decoding (via stout) behaves differently on Windows.
* [MESOS-7237] - Enabling cgroups_limit_swap can lead to "invalid argument" error.
* [MESOS-7248] - RemoveNestedContainer returns unsupported.
* [MESOS-7255] - New mesos-style.py linter behavior breaks commiting when virtualenv is not installed.
* [MESOS-7259] - Remove deprecated ACLs `SetQuota` and `RemoveQuota`.
* [MESOS-7261] - maintenance.html is missing during packaging.
* [MESOS-7263] - User supplied task environment variables cause warnings in sandbox stdout.
* [MESOS-7264] - Possibly duplicate environment variables should not leak values to the sandbox.
* [MESOS-7265] - Containerizer startup may cause sensitive data to leak into sandbox logs.
* [MESOS-7270] - Java V1 Framwork Test failed on macOS.
* [MESOS-7272] - Unified containerizer does not support docker registry version < 2.3.
* [MESOS-7280] - Unified containerizer provisions docker image error with COPY backend.
* [MESOS-7281] - Backwards incompatible UpdateFrameworkMessage handling.
* [MESOS-7287] - Fix post-reviews.py to find `rbt.cmd` on Windows.
* [MESOS-7300] - Mesos failed to build on Windows due to error C2440: 'return': cannot convert from 'Error' to 'bool'.
* [MESOS-7311] - CopyFetcherPluginTest.FetchExistingFile.
* [MESOS-7316] - Upgrading Mesos to 1.2.0 results in some information missing from the `/flags` endpoint.
* [MESOS-7323] - Framework role tracking in allocator results in framework treated as active incorrectly.
* [MESOS-7340] - Log HTTP accesses to the /files endpoint.
* [MESOS-7346] - Agent crashes if the task name is too long.
* [MESOS-7348] - Network isolator crashes agent on startup when network interface cannot be found.
* [MESOS-7350] - Failed to pull image from Nexus Registry due to signature missing.
* [MESOS-7363] - Improver master robustness against duplicate UPIDs.
* [MESOS-7365] - Compile error with recent glibc.
* [MESOS-7372] - Improve agent re-registration robustness.
* [MESOS-7378] - Build failure with glibc 2.12.
* [MESOS-7389] - Mesos 1.2.0 crashes with pre-1.0 Mesos agents.
* [MESOS-7400] - The mesos master crashes due to an incorrect invariant check in the decoder.
* [MESOS-7427] - Registry puller cannot fetch manifests from Amazon ECR: 405 Unsupported.
* [MESOS-7430] - Per-role Suppress call implementation is broken.
* [MESOS-7431] - Registry puller cannot fetch manifests from Google GCR: 403 Forbidden.
* [MESOS-7453] - glyphicons-halflings-regular.woff2 is missing in WebUI.
* [MESOS-7456] - Compilation error on recent glibc in cgroups device subsystem.
* [MESOS-7464] - Recent Docker versions cannot be parsed by stout.
* [MESOS-7471] - Provisioner recover should not always assume 'rootfses' dir exists.
* [MESOS-7478] - Pre-1.2.x master does not work with 1.2.x agent.
* [MESOS-7484] - VersionTest.ParseInvalid aborts on Windows.
* [MESOS-7521] - Major performance regression in DRF sorter.
* [MESOS-7538] - Don't validate re-registrations that are going to be dropped.
** Documentation
* [MESOS-7005] - Add executor authentication documentation.
* [MESOS-7324] - Update documentation to reflect the addition of multi-role framework support.
** Epic
* [MESOS-1763] - Add support for frameworks to receive resources for multiple roles.
* [MESOS-6365] - Executor authentication.
* [MESOS-6627] - Allow frameworks to modify the role(s) they are subscribed to.
** Improvement
* [MESOS-970] - Upgrade bundled leveldb to 1.19.
* [MESOS-5186] - mesos.interface: Allow using protobuf 3.x.
* [MESOS-5992] - Complete the list of API Calls on the Operator HTTP API Doc.
* [MESOS-6280] - Task group executor should support command health checks.
* [MESOS-6304] - Add authentication support to the default executor.
* [MESOS-6523] - Agent cgroup assignment should precede agent initialization.
* [MESOS-6906] - Introduce a general non-interpreting task check.
* [MESOS-7021] - Consistent symlink behavior for os::stat accessors.
* [MESOS-7074] - port_mapping isolator: do not depend on /sys/class/net/<ifname>/speed.
* [MESOS-7101] - ExamplesTest.PersistentVolumeFramework failed on ASF CI.
* [MESOS-7120] - Add an Agent API call to cleanup nested container artifacts.
* [MESOS-7226] - Introduce precompiled headers (on Windows).
* [MESOS-7249] - Default executor does not support general checks.
* [MESOS-7256] - Replace Boost Type Traits leftovers with STL.
* [MESOS-7274] - Health checker does not support pause / resume.
* [MESOS-7275] - General checker does not support TCP checks.
* [MESOS-7276] - General checker does not support pause / resume.
* [MESOS-7277] - General checker does not support command checks via agent.
* [MESOS-7376] - Reduce copying of the Registry to improve Registrar performance.
* [MESOS-7387] - ZK master contender and detector don't respect zk_session_timeout option.
** Task
* [MESOS-3139] - Incorporate CMake into standard documentation.
* [MESOS-5418] - Test case: Escape containerizer command line on Windows.
* [MESOS-6022] - unit-test for port-mapper CNI plugin.
* [MESOS-6032] - Add infrastructure for unit tests in the new python-based CLI.
* [MESOS-6123] - Implement GET_AGENT call in v1 agent API.
* [MESOS-6447] - Display role weight / role quota information in the webui.
* [MESOS-6636] - Validate that tasks / executors / reservations / volumes do not mix Resource.allocation_info.roles.
* [MESOS-6637] - Validate that schedulers cannot perform operations on offers with different allocation roles.
* [MESOS-6657] - Update the webui to reflect that frameworks have multiple roles.
* [MESOS-6691] - Enable SSL in Mesos builds.
* [MESOS-6762] - Update release notes for multi-role changes.
* [MESOS-6791] - Allow to specific the device whitelist entries in cgroup devices subsystem.
* [MESOS-6808] - Refactor Docker::run to only take docker cli parameters.
* [MESOS-6855] - Add `role` section to response of /state endpoint.
* [MESOS-6886] - Add authorization tests for debug API handlers.
* [MESOS-6940] - Do not send offers to MULTI_ROLE schedulers if agent does not have MULTI_ROLE capability.
* [MESOS-6967] - Ensure offer operations can be applied for MULTI_ROLE and non-MULTI_ROLE frameworks.
* [MESOS-6992] - Remove validation against "/" characters in roles to support hierarchical roles.
* [MESOS-6995] - Update the webui to reflect hierarchical roles.
* [MESOS-6996] - Add a 'Secret' protobuf message.
* [MESOS-6997] - Add the SecretGenerator module interface.
* [MESOS-6998] - Add authentication support to agent's '/v1/executor' endpoint.
* [MESOS-6999] - Add agent support for generating and passing executor secrets.
* [MESOS-7000] - Implement a JWT SecretGenerator.
* [MESOS-7001] - Implement a JWT authenticator.
* [MESOS-7003] - Introduce a 'Principal' type.
* [MESOS-7004] - Enable multiple HTTP authenticator modules.
* [MESOS-7009] - Add a 'secret' field to the 'Environment' message.
* [MESOS-7011] - Add an '--executor_secret_key' flag to the agent.
* [MESOS-7013] - Update the authorizer interface for executor authentication.
* [MESOS-7014] - Add implicit executor authorization to local authorizer.
* [MESOS-7024] - Update the allocator to handle hierarchical roles.
* [MESOS-7026] - Update authorization / authorization-filtering to handle hierarchical roles.
* [MESOS-7037] - Prevent setting quota on nested roles not contained by parent role quota.
* [MESOS-7038] - Update quota cluster capacity heuristic for hierarchical roles.
* [MESOS-7039] - Prevent quota removal that violates parent role-child role quota containment.
* [MESOS-7047] - Update agent for hierarchical roles.
* [MESOS-7048] - Remove adjustment code within Resources::apply.
* [MESOS-7061] - Re-persist tasks/executors with allocation info during agent recovery.
* [MESOS-7063] - Add a test for a MULTI_ROLE master re-registering an old agent.
* [MESOS-7269] - Migrate setting in config.py to a TOML file.
* [MESOS-7282] - Create a table abstraction for the Mesos CLI.
* [MESOS-7320] - Remove deprecated ACL `ShutdownFramework`.
* [MESOS-7336] - Add resource provider API protobuf.
* [MESOS-7339] - Add authorization to agent executor API.
* [MESOS-7377] - Add authentication to the checker and health checker libraries.
* [MESOS-7391] - Add deprecation warning for Visual Studio 14 2015.
* [MESOS-7395] - Benchmark performance of hierarchical roles.
* [MESOS-7439] - Bump the default timeout value for docker volume driver unmount operation.
Release Notes - Mesos - Version 1.2.3
-------------------------------------------
* This is a bug fix release.
All Issues:
** Bug
* [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
* [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
* [MESOS-7365] - Compile error with recent glibc.
* [MESOS-7378] - Build failure with glibc 2.12.
* [MESOS-7627] - Mesos slave stucks.
* [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
* [MESOS-7744] - Mesos Agent Sends TASK_KILL status update to Master, and still launches task.
* [MESOS-7783] - Framework might not receive status update when a just launched task is killed immediately.
* [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
* [MESOS-7863] - Agent may drop pending kill task status updates.
* [MESOS-7865] - Agent may process a kill task and still launch the task.
* [MESOS-7872] - Scheduler hang when registration fails.
* [MESOS-7909] - Ordering dependency between 'linux/capabilities' and 'docker/runtime' isolator.
* [MESOS-7926] - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
* [MESOS-7934] - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.
* [MESOS-7968] - Handle `/proc/self/ns/pid_for_children` when parsing available namespace.
* [MESOS-7969] - Handle cgroups v2 hierarchy when parsing /proc/self/cgroups.
* [MESOS-7975] - The command/default/docker executor can incorrectly send a TASK_FINISHED update even when the task is killed.
* [MESOS-7980] - Stout fails to compile with libc >= 2.26.
* [MESOS-8051] - Killing TASK_GROUP fail to kill some tasks.
* [MESOS-8080] - The default executor does not propagate missing task exit status correctly.
* [MESOS-8135] - Masters can lose track of tasks' executor IDs.
Release Notes - Mesos - Version 1.2.2
-------------------------------------
* This is a bug fix release.
All Issues:
** Bug
* [MESOS-5187] - The filesystem/linux isolator does not set the permissions of the host_path.
* [MESOS-7252] - Need to fix resource check in long-lived framework.
* [MESOS-7546] - WAIT_NESTED_CONTAINER sometimes returns 404.
* [MESOS-7569] - Allow "old" executors with half-open connections to be preserved during agent upgrade / restart.
* [MESOS-7581] - Fix interference of external Boost installations when using some unbundled dependencies.
* [MESOS-7689] - Libprocess can crash on malformed request paths for libprocess messages.
* [MESOS-7690] - The agent can crash when an unknown executor tries to register.
* [MESOS-7703] - Mesos fails to exec a custom executor when no shell is used.
* [MESOS-7728] - Java HTTP adapter crashes JVM when leading master disconnects.
* [MESOS-7770] - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.
* [MESOS-7777] - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13.
* [MESOS-7796] - LIBPROCESS_IP isn't passed on to the fetcher.
* [MESOS-7830] - Sandbox_path volume does not have ownership set correctly.
** Improvement
* [MESOS-7540] - Add an agent flag for executor re-registration timeout.
Release Notes - Mesos - Version 1.2.1
-------------------------------------
* This is a bug fix release.
**NOTE**: In Mesos 1.2.1, the master will no longer allow 0.x agents to
register. Interoperability between 1.1+ masters and 0.x agents has never
been supported; however, it was not explicitly disallowed, either.
Starting with this release of Mesos, registration attempts by 0.x Mesos
agents will be ignored.
All Issues:
** Bug
* [MESOS-1987] - Add support for SemVer build and prerelease labels to stout.
* [MESOS-5028] - Copy provisioner cannot replace directory with symlink.
* [MESOS-5172] - Registry puller cannot fetch blobs correctly from http Redirect 3xx urls.
* [MESOS-6327] - Large docker images causes container launch failures: Too many levels of symbolic links.
* [MESOS-6951] - Docker containerizer: mangled environment when env value contains LF byte.
* [MESOS-6976] - Disallow (re-)registration attempts by old agents.
* [MESOS-7133] - mesos-fetcher fails with openssl-related output.
* [MESOS-7197] - Requesting tiny amount of CPU crashes master.
* [MESOS-7208] - Persistent volume ownership is set to root when task is running with non-root user.
* [MESOS-7210] - HTTP health check doesn't work when mesos runs with --docker_mesos_image.
* [MESOS-7232] - Add support to auto-load /dev/nvidia-uvm in the GPU isolator.
* [MESOS-7237] - Enabling cgroups_limit_swap can lead to "invalid argument" error.
* [MESOS-7261] - maintenance.html is missing during packaging.
* [MESOS-7263] - User supplied task environment variables cause warnings in sandbox stdout.
* [MESOS-7264] - Possibly duplicate environment variables should not leak values to the sandbox.
* [MESOS-7265] - Containerizer startup may cause sensitive data to leak into sandbox logs.
* [MESOS-7272] - Unified containerizer does not support docker registry version < 2.3.
* [MESOS-7280] - Unified containerizer provisions docker image error with COPY backend.
* [MESOS-7316] - Upgrading Mesos to 1.2.0 results in some information missing from the `/flags` endpoint.
* [MESOS-7346] - Agent crashes if the task name is too long.
* [MESOS-7350] - Failed to pull image from Nexus Registry due to signature missing.
* [MESOS-7366] - Agent sandbox gc could accidentally delete the entire persistent volume content.
* [MESOS-7368] - Documentation of framework role(s) in proto definition is confusing.
* [MESOS-7383] - Docker executor logs possibly sensitive parameters.
* [MESOS-7389] - Mesos 1.2.0 crashes with pre-1.0 Mesos agents.
* [MESOS-7400] - The mesos master crashes due to an incorrect invariant check in the decoder.
* [MESOS-7427] - Registry puller cannot fetch manifests from Amazon ECR: 405 Unsupported.
* [MESOS-7429] - Allow isolators to inject task-specific environment variables.
* [MESOS-7453] - glyphicons-halflings-regular.woff2 is missing in WebUI.
* [MESOS-7464] - Recent Docker versions cannot be parsed by stout.
* [MESOS-7471] - Provisioner recover should not always assume 'rootfses' dir exists.
* [MESOS-7478] - Pre-1.2.x master does not work with 1.2.x agent.
* [MESOS-7484] - VersionTest.ParseInvalid aborts on Windows.
Release Notes - Mesos - Version 1.2.0
-------------------------------------------
This release contains the following new features:
* [MESOS-5931] - **Experimental** Support auto backend in Mesos Containerizer,
prefering overlayfs then aufs. Please note that the bind backend needs to be
specified explicitly through the agent flag '--image_provisioner_backend'
since it requires the sandbox already existed.
* [MESOS-6402] - **Experimental** Add rlimit support to Mesos containerizer.
The isolator adds support for setting POSIX resource limits (rlimits) for
containers launched using the Mesos containerizer. POSIX rlimits can be used
to control the resources a process can consume. See `docs/posix_rlimits.md`
for details.
* [MESOS-6419] - **Experimental** Teardown unregistered frameworks. The master
now treats recovered frameworks very similarly to frameworks that are registered
but currently disconnected. For example, recovered frameworks will be reported
via the normal "frameworks" key when querying HTTP endpoints. This means there
is no longer a concept of "orphan tasks": if the master knows about a task, the
task will be running under a framework. Similarly, "teardown" operations on
recovered frameworks will now work correctly.
* [MESOS-6460] - **Experimental** Container Attach and Exec. This feature adds
new Agent APIs for attaching a remote client to the stdin, stdout, and stderr
of a running Mesos task, as well as an API for launching new processes inside
the same container as a running Mesos task and attaching to its stdin, stdout,
and stderr. At a high level, these APIs mimic functionality similar to docker
attach and docker exec. The primary motivation for such functionality is to
enable users to debug their running Mesos tasks.
* [MESOS-6758] - **Experimental** Support 'Basic' auth docker private registry
on Mesos Containerizer. Until now, the mesos containerizer always assumed
Bearer auth, but we now also support basic auth for private registries. Please
note that the AWS ECS uses Basic authorization but it does not work yet due to
the redirect issue MESOS-5172.
Deprecations:
* [MESOS-6650] - Remove slavePreLaunchDockerEnvironmentDecorator and slavePreLaunchDockerHook.
Additional API Changes:
* [MESOS-3601] - Formalize all headers and metadata for HTTP API Event Stream
* [MESOS-6286] - If an agent restarts but fails to complete recovery
within `agent_reregister_timeout`, the master will now mark the
agent as unreachable. This mainly changes behavior in two
situations: (a) the master will now be more robust if agent recovery
hangs indefinitely (e.g., due to a container being in a bad state),
and (b) if agent recovery takes a very long time (e.g., because the
agent's work directory contains a large number of completed tasks),
the master might now mark an agent unreachable that would previously
have been able to eventually recover successfully.
* [MESOS-6419] - When a framework re-registers after master failover,
it is only allowed to change certain fields in its FrameworkInfo.
For example, changing "failover_timeout" is allowed, but changing
"role" is not. In previous Mesos releases, the same restrictions on
changes to FrameworkInfo were only enforced after framework
failover, not master failover.
* [MESOS-6670] - Authz for Agent v1 operator API
* [MESOS-6675] - Changed the allocator API to support adding inactive
frameworks. Custom allocator implementations will need to be updated.
* [MESOS-6865] - Remove the constraint of being only able to launch
2-level nested containers on Agent API.
Unresolved Critical Issues:
* [MESOS-1625] - Extra trailing CRLF being sent after the HTTP body in libprocess
* [MESOS-1718] - Command executor can overcommit the agent.
* [MESOS-2554] - Slave flaps when using --slave_subsystems that are not used for isolation.
* [MESOS-2774] - SIGSEGV received during process::MessageEncoder::encode()
* [MESOS-2842] - Update FrameworkInfo.principal on framework re-registration
* [MESOS-3533] - Unable to find and run URIs files
* [MESOS-3747] - HTTP Scheduler API no longer allows FrameworkInfo.user to be empty string
* [MESOS-3794] - Master should not store arbitrarily sized data in ExecutorInfo.
* [MESOS-4259] - mesos HA can't delete the the redundant container on failure slave node.
* [MESOS-4297] - Executor does not shutdown when framework teardown.
* [MESOS-4642] - Mesos Agent Json API can dump binary data from log files out as invalid JSON.
* [MESOS-4996] - 'containerizer->update' will always fail after killing a docker container.
* [MESOS-5352] - Docker volume isolator cleanup can be blocked by first cleanup failure.
* [MESOS-5396] - After failover, master does not remove agents with same UPID.
* [MESOS-5849] - Agent sandboxes on Windows surpass the 260 character path length limit
* [MESOS-5859] - Some tasks are always in staged state.
* [MESOS-5989] - Libevent SSL Socket downgrade code accesses uninitialized memory / assumes single peek is sufficient.
* [MESOS-6327] - Large docker images causes container launch failures: Too many levels of symbolic links.
* [MESOS-6356] - ASF CI has interleaved logging.
* [MESOS-6615] - Running mesos-slave in the docker that leave many zombie process
* [MESOS-6623] - Re-enable tests impacted by request streaming support
* [MESOS-6632] - ContainerLogger might leak FD if container launch fails.
* [MESOS-6780] - ContentType/AgentAPIStreamingTest.AttachContainerInput test fails reliably
* [MESOS-6784] - IOSwitchboardTest.KillSwitchboardContainerDestroyed is flaky
* [MESOS-6804] - Running 'tty' inside a debug container that has a tty reports "Not a tty"
* [MESOS-6815] - Enable glog stack traces when we call things like `ABORT` on Windows
* [MESOS-6843] - Fetcher should not assume stdout/stderr in the sandbox.
* [MESOS-6913] - AgentAPIStreamingTest.AttachInputToNestedContainerSession fails on Mac OS.
* [MESOS-6974] - DefaultExecutorTest.CommitSuicideOnTaskFailure test is flaky.
* [MESOS-6986] - abort in DRFSorter::add
* [MESOS-7017] - HTTP API responses can crash the master.
* [MESOS-7050] - IOSwitchboard FDs leaked when containerizer launch fails -- leads to deadlock
* [MESOS-7099] - Quota can be exceeded due to coarse-grained offer technique.
Feature Graduations:
* None
All Experimental Features:
* [MESOS-2449] - Support group of tasks (Pod) constructs and API in Mesos.
* [MESOS-2533] - Support HTTP checks in Mesos.
* [MESOS-3094] - Mesos on Windows.
* [MESOS-3421] - Support sharing of resources across task instances.
* [MESOS-3567] - Support TCP checks in Mesos.
* [MESOS-4312] - Porting Mesos on Power (ppc64le).
* [MESOS-4355] - Implement isolator for Docker volume.
* [MESOS-4641] - Support Container Network Interface (CNI).
* [MESOS-4791] - Operator API v1.
* [MESOS-4828] - XFS disk quota isolator.
* [MESOS-5275] - Add capabilities support for mesos containerizer.
* [MESOS-5344] - Partition-aware Mesos frameworks.
* [MESOS-5788] - Added JAVA API adapter for seamless transition to new scheduler API.
* [MESOS-5931] - **NEW** Support auto backend in Mesos Containerizer.
* [MESOS-6014] - Added port mapping CNI plugin.
* [MESOS-6077] - Added a default (task group) executor.
* [MESOS-6402] - **NEW** rlimit support for Mesos containerizer
* [MESOS-6419] - **NEW** Teardown unregistered frameworks
* [MESOS-6460] - **NEW** Container Attach/Exec
* [MESOS-6758] - **NEW** Support docker registry that requires basic auth.
All Issues:
** Bug
* [MESOS-1802] - HealthCheckTest.HealthStatusChange is flaky on jenkins.
* [MESOS-2537] - AC_ARG_ENABLED checks are broken
* [MESOS-2723] - The mesos-execute tool does not support zk:// master URLs
* [MESOS-3335] - FlagsBase copy-ctor leads to dangling pointer.
* [MESOS-3932] - Silence Boost compiler warnings with CMake
* [MESOS-4601] - Don't dump stack trace on failure to bind()
* [MESOS-4695] - SlaveTest.StateEndpoint is flaky
* [MESOS-4973] - Duplicates in 'unregistered_frameworks' in /state
* [MESOS-4975] - mesos::internal::master::Slave::tasks can grow unboundedly
* [MESOS-5218] - Fetcher should not chown the entire sandbox.
* [MESOS-5303] - Add capabilities support for mesos execute cli.
* [MESOS-5662] - Call parent class `SetUpTestCase` function in our test fixtures.
* [MESOS-5821] - Clean up the thousands of compiler warnings on MSVC
* [MESOS-5835] - Audit `PATCH_CMD`; make sure all patches are being applied on Windows.
* [MESOS-5856] - Logrotate ContainerLogger module does not rotate logs when run as root with `--switch_user`.
* [MESOS-5879] - cgroups/net_cls isolator causing agent recovery issues
* [MESOS-5963] - HealthChecker should not decide when to kill tasks and when to stop performing health checks.
* [MESOS-6001] - Aufs backend cannot support the image with numerous layers.
* [MESOS-6002] - The whiteout file cannot be removed correctly using aufs backend.
* [MESOS-6010] - Docker registry puller shows decode error "No response decoded".
* [MESOS-6119] - TCP health checks are not portable.
* [MESOS-6142] - Frameworks may RESERVE for an arbitrary role.
* [MESOS-6206] - Change reconciliation to return results for in-progress removals and reregistrations
* [MESOS-6286] - Master does not remove an agent if it is responsive but not registered
* [MESOS-6288] - The default executor should maintain launcher_dir.
* [MESOS-6293] - HealthCheckTest.HealthyTaskViaHTTPWithoutType fails on some distros.
* [MESOS-6316] - CREATE of shared volumes should not be allowed by frameworks not opted in to the capability.
* [MESOS-6320] - Implement clang-tidy check to catch incorrect flags hierarchies
* [MESOS-6349] - JSON Generation breaks if other locale than C is used.
* [MESOS-6360] - The handling of whiteout files in provisioner is not correct.
* [MESOS-6380] - mesos-local failed to start without sudo
* [MESOS-6388] - Report new PARTITION_AWARE task statuses in HTTP endpoints
* [MESOS-6389] - Update webui for PARTITION_AWARE changes
* [MESOS-6409] - mesos-ps - Invalid header value
* [MESOS-6414] - cgroups isolator cleanup failed when the hierarchy is cleanup by docker daemon
* [MESOS-6419] - The 'master/teardown' endpoint should support tearing down 'unregistered_frameworks'.
* [MESOS-6420] - Mesos Agent leaking sockets when port mapping network isolator is ON
* [MESOS-6432] - Roles with quota assigned can "game" the system to receive excessive resources.
* [MESOS-6444] - Ensure single copy of shared count of total resources in role sorter.
* [MESOS-6446] - WebUI redirect doesn't work with stats from /metric/snapshot
* [MESOS-6448] - Show the leading master hostname in the webUI.
* [MESOS-6452] - Compile error in strerror.h on OSX
* [MESOS-6455] - DefaultExecutorTests fail when running on hosts without docker.
* [MESOS-6459] - PosixRLimitsIsolatorTest.TaskExceedingLimit fails on OS X
* [MESOS-6461] - Duplicate framework ids in /master/frameworks endpoint 'unregistered_frameworks'.
* [MESOS-6478] - "filesystem/linux" isolator leaks (phantom) mounts in `mount` output
* [MESOS-6483] - Check failure when a 1.1 master marking a 0.28 agent as unreachable
* [MESOS-6484] - Memory leak in `Future<T>::after()`
* [MESOS-6501] - Add a test for duplicate framework ids in "unregistered_frameworks"
* [MESOS-6504] - Use 'geteuid()' for the root privileges check.
* [MESOS-6508] - monitor/statistics error in webui when launch mesos via mesos-local
* [MESOS-6516] - Parallel test running does not respect GTEST_FILTER
* [MESOS-6519] - MasterTest.OrphanTasksMultipleAgents
* [MESOS-6520] - Make errno an explicit argument for ErrnoError.
* [MESOS-6526] - `mesos-containerizer launch --environment` exposes executor env vars in `ps`.
* [MESOS-6527] - Memory leak in the libprocess request decoder.
* [MESOS-6544] - MasterMaintenanceTest.InverseOffersFilters is flaky.
* [MESOS-6545] - TestContainerizer is not thread-safe.
* [MESOS-6566] - The Docker executor should not leak task env variables in the Docker command cmd line.
* [MESOS-6569] - MesosContainerizer/DefaultExecutorTest.KillTask/0 failing on ASF CI
* [MESOS-6576] - DefaultExecutorTest.KillTaskGroupOnTaskFailure sometimes fails in CI
* [MESOS-6588] - LinuxRootfs misses required files
* [MESOS-6597] - Include v1 Operator API protos in generated JAR and python packages.
* [MESOS-6598] - Broken Link Framework Development Page
* [MESOS-6602] - Shutdown completed frameworks when unreachable agent re-registers
* [MESOS-6604] - Uninitialized member ObjectApprover::weight_info.
* [MESOS-6606] - Reject optimized builds with libcxx before 3.9
* [MESOS-6618] - Some tests use hardcoded port numbers.
* [MESOS-6619] - Improve task management for unreachable tasks
* [MESOS-6621] - SSL downgrade path will CHECK-fail when using both temporary and persistent sockets
* [MESOS-6624] - Master WebUI does not work on Firefox 45
* [MESOS-6625] - Expose container id in ContainerStatus in DockerContainerizer.
* [MESOS-6640] - mesos-local doesn't hande --work_dir correctly.
* [MESOS-6646] - StreamingRequestDecoder incompletely initializes its http_parser_settings
* [MESOS-6647] - Cyclic header dependency between libprocess' defer.hpp and executor.hpp
* [MESOS-6652] - Perf version not correctly parsed on Fedora 24 (and probably others)
* [MESOS-6653] - Overlayfs backend may fail to mount the rootfs if both container image and image volume are specified.
* [MESOS-6654] - Duplicate image layer ids may make the backend failed to mount rootfs.
* [MESOS-6658] - Mesos tests generated with cmake build fail to unload libraries properly
* [MESOS-6665] - io::redirect might cause stack overflow.
* [MESOS-6666] - HttpServeTest.Discard failed on OSX sierra
* [MESOS-6672] - Class DynamicLibrary's default copy constructor can lead to inconsistent state
* [MESOS-6676] - Always re-link with scheduler during re-registration.
* [MESOS-6677] - Error in Windows agent's Flags::runtime_dir CLI
* [MESOS-6684] - Update addFramework/removeFramework to handle multi-role frameworks