Skip to content

@smarterclayton smarterclayton released this Jun 8, 2017 · 11796 commits to master since this release

This is a feature release of OpenShift Origin.

Backwards Compatibility

  • The experimental command oc import docker-compose has been removed #13795
  • The status.unavailableReplicas field on deployment configs no longer accepts negative numbers #14046
  • The extended certificate validation feature in the router is now much stricter #13897
    • In order to ensure that buggy, malicious, or invalid certificates cannot crash a router the extended certificate feature now decodes and then re-encodes certificates from routes. Only a known set of allowed PEM blocks and certificate types will be accepted, including the common RSA and ECSDA variants of both public and private keys.
    • If you are upgrading your cluster and have not disabled extended certification (on by default) you should start a test router instance and verify that all routes successfully load before completing your rollout.
    • Routes that fail extended validation are taken out of rotation and have a status field message set indicating they are not accepted.


Roadmap for the v3.6 release

v3.6.0-alpha.2 (2017-06-07) Full Changelog


  • Add fields to builds that display the status of build results #13307
    • A series of stages and steps are part of build status and populated by the builder.
  • Two fields are added to build config spec to control how many successful and failed builds are preserved on the cluster #13788
  • The spec.tls.destinationCACertificate field on a route is now optional - routers that don't allow defaulting will reject this route

Component updates

  • Updated to Kubernetes v1.6.1-1-g5115d708d7 + patches
    • 39732: Fix issue #34242: Attach/detach should recover from a crash #14119
    • 40423: Support for v1/v2/autoprobe openstack cinder blockstorage #14005
    • 41498: cinder: Add support for the KVM virtio-scsi driver #14005
    • 41634: Handle error event type #13939
    • 41939: Add an AEAD encrypting transformer for storing secrets encrypted at rest. #14243
    • 42033: fix TODO: find and add active pods for dswp #14119
    • 42672: use separate scheme to serve the kube-aggregator #13974
    • 42801: add local option to APIService #13974
    • 42886: allow fallthrough handling from go-restful routes #13974
    • 42900: rewire aggregation handling chain to be normal #13974
    • 42911: combine kube-apiserver and kube-aggregator #13974
    • 43076: allow combining API servers #13974
    • 43141: Create controller to auto register TPRs with the aggregator #13974
    • 43144: start informers as a post-start-hook #13974
    • 43149: break kube-apiserver start into stages #13974
    • 43170: Add ability to customize fed namespace for e2e #14106
    • 43226: don't start controllers against unhealthy master #13974
    • 43289: Attach/detach controller: fix potential race in constructor #14119
    • 43301: add APIService conditions #14285
    • 43375: Set permission for volume subPaths #13895
    • 43377: only log stacks on server errors #14173
    • 43383: proxy to IP instead of name, but still use host verification #13974
    • 43396: iSCSI CHAP support #14112
    • 43575: util/iptables: check for and use new iptables-restore 'wait' argument #14186
    • 43922: prevent corrupted spdy stream after hijacking connection #13669
    • 43945: Remove 'beta' from default storage class annotation #14427
    • 44066: Improve federation e2e test setup #14106
    • 44068: Use Docker API Version instead of docker version (fixup) #14335
    • 44068: Use Docker API Version instead of docker version #14158
    • 44072: Cleanup e2e framework for federation #14106
    • 44073: Optionally retrieve fed e2e cluster config from secrets #14106
    • 44082: use AvailabilityZone instead of Availability #14005
    • 44221: validateClusterInfo: use clientcmdapi.NewCluster() #13653
    • 44295: Azure disk: dealing with missing disk probe #14072
    • 44406: CRI: Stop following container log when container exited. #14380
    • 44439: controller: fix saturation check in Deployments #13890
    • 44452: Implement LRU for AWS device allocator #14119
    • 44462: 44489: fix selfLink for cluster-scoped resources #14001
    • 44566: WaitForCacheSync before running attachdetach controller #14119
    • 44570: Explicit namespace from kubeconfig should override in-cluster config #13653
    • 44625: Retry secret reference addition on conflict #14033
    • 44639: Set fed apiserver to bind to 8443 instead of 443 #14107
    • 44730: Check for terminating Pod prior to launching successor in StatefulSet #13653
    • 44760: Fix issue #44757: Flaky Test_AttachDetachControllerRecovery #14119
    • 44781: Ensure desired state of world populator runs before volume reconstructor #14144
    • 44798: Cinder: Automatically Generate Zone if Availability in Storage Class is not Configured #14159
    • 44837: Fix Content-Type error of apis #14285
    • 44859: e2e: handle nil ReplicaSet in checkDeploymentRevision #13653
    • 44861: NotRegisteredErr for known kinds not registered in target GV #13653
    • 44895: util/iptables: grab iptables locks if iptables-restore doesn't support --wait #14186
    • 44939: don't HandleError on container start failure #14077
    • 44970: CRI: Fix StopContainer timeout #13938
    • 45100: node-controller: deflake TestUpdateNodeWithMultiplePods #13940
    • 45105: taint-controller-tests: double 'a bit of time' to avoid flakes #13953
    • 45171: Use groupName comment for listers/informers #13982
    • 45235: remove bearer token from headers after we consume it #14007
    • 45238: expose kubelet authentication and authorization builders #14011
    • 45286: When pods are terminated we should detach the volume #14191
    • 45304: increase the QPS for namespace controller #14274
    • 45403: apiserver: injectable default watch cache size #14052
    • 45413: Extend timeouts in timed_workers_test #14225
    • 45427: 45897: GC controller improvements #14358
    • 45496: fix pleg relist time #14282
    • 45505: expose the controller initializers #14033
    • 45515: Ignore openrc group #13964
    • 45601: util/iptables: fix cross-build failures due to syscall.Flock() #14186
    • 45623: Don't attempt to make and chmod subPath if it already exists #14193
    • 45685: fix quota resync #14151
    • 45741: Fix discovery version for autoscaling to be v1 #14255
    • 45747: OwnerReferencesPermissionEnforcement ignores pods/status #14204
    • 45826: prevent pods/status from touching ownerreferences #14204
    • 45835: client-gen: honor groupName overrides in customArgs #14203
    • 45894: Export BaseControllerRefManager #14322
    • 45933: Use informers in scheduler / token controller (part 2, fixing tests) #14412
    • 45933: Use informers in scheduler / token controller #14321
    • 45940: apiserver: no Status in body for http 204 #14237
    • 45977: kuberuntime: report StartedAt regardless of container states #14312
    • 46020: Enable customization of federation image #14239
    • 46037: NS controller: don't stop deleting GVRs on error #14275
    • 46042: ResourceQuota admission control injects registry (federation) #14234
    • 46042: ResourceQuota admission control injects registry #14234
    • 46127: Return MethodNotSupported when accessing unwatcheable resource with ?watch=true #14260
    • 46239: Log out from multiple target portals when using iscsi storage plugin #14457
    • 46246: Fix kubelet event recording #14299
    • 46247: Enable customization of federation etcd image #14239
    • 46299: Fix in-cluster kubectl --namespace override #14307
    • 46305: clear init container status annotations when cleared in status #14331
    • 46315: Fix provisioned GCE PD not being reused if already exists #14329
    • 46323: Use beta annotation for fed etcd pvc storage class #14239
    • 46371: reset resultRun on pod restart #14332
    • 46373: don't queue namespaces for deletion if the namespace isn't deleted #14347
    • 46390: Require DeleteStrategy for all registry.Store #14337
    • 46437: Up namespace controller workers to 5 #14352
    • 46463: AWS: consider instances of all states in DisksAreAttached, not just "running" #14425
    • 46500: Fix standardFinalizers - add missing metav1.FinalizerDeleteDependents (Note: it is in different files from upstream because they moved helpers.go into helper/helpers.go) #14322
    • 46516: kubelet was sending negative allocatable values #14379
    • 46608: fixes kubectl cached discovery on Windows #14399
    • 46614: Add auto_unmount mount option for glusterfs fuse mount. #14443
    • 46628: cleanup kubelet new node status test #14379
    • 46640: Improve validation of active deadline seconds #14424
    • 46751: Pre-generate SNI test certs #14412
    • Fix to avoid REST API calls at log level 2. #13844
    • add OpenShift resources to garbage collector ignore list #13653
    • openapi test, patch in updated package name #13653
    • Set the log level for iptables rule dump to 5 #14359
    • disable apiserver loopback loop in generic context #13653
    • kube-apiserver must not start aggregator #13974
    • Integrate the kube-aggregator to support the service catalog #13974
    • Consume the upstream authorizer #14006
    • Continue to support extensions/v1beta1 version of HorizontalPodAutoscaler #14021
  • Updates to Docker distribution
    • docker/distribution: 1757: Export storage.CreateOptions in top-level package #13653
    • docker/distribution: 1857: Provide stat descriptor for Create method during cross-repo mount #13653
    • docker/distribution: 2008: Honor X-Forwarded-Port and Forwarded headers #13653
    • docker/distribution: 2140: Add 'ca-central-1' region for registry S3 storage driver #13653


Tech preview of cluster federation

Cluster workload federation is nearing beta status in Kubernetes and is now part of the OpenShift distribution. The kubefed binary is built as part of Origin and will help stand up a tech preview cluster.

  • Enable preliminary support for origin federation #14239

Simulate cluster capacity

The cluster capacity command emulates the Kubernetes scheduler for a set of pod workloads and estimates how many pods can be scheduled on a cluster.

  • Add cluster capacity image to OpenShift #14258

Enable Garbage Collection on OpenShift resources

The garbage collection feature in Kubernetes is now stable and enabled in OpenShift. OpenShift controllers like deployment configs, build configs, and templates set owner references on the objects they create, which means deleting a deployment config will now automatically clean up the replication controllers and pods created by the deployment. The web console uses owner information to better organize resources and can now delete resources when they have changed. See the documentation for more on how to leverage garbage collection to manage cleanup.

  • Add DC controllerRef to RC #14322
  • web: Bug 1449908 - Group replica sets by owner reference #1553
  • web: Bug 1449949 - Group pods by owner reference #1538
  • orphan resources by default for SOME resourced under /oapi #14134
  • mark build->buildconfig ownerref as a controller #14250
  • shared GC #14358

Improvements to network egress policy

This release improves egress network policy (handled by an egress router) to make managing traffic leaving namespaces easier via DNS name support on destinations and also targeting multiple destinations.

  • sdn: Support DNS names for egress network policy #13002
  • egress: Allow multiple destinations in egress-router #13837

Use git references like tags and GitHub/GitLab pull request links in builds

Git branches and tags are both represented as "references" inside of a repository. GitHub adds references in Git for each pull request, but to use that reference in an OpenShift build we must first fetch the
information from the remote server. The ref field on a build's Git source specification can now point
to any valid Git reference and OpenShift will attempt to retrieve that ref, allowing pull requests to
be spawned for a specific build.

  • builds: Allow the ref field on builds to point to any Git reference #13893
  • builds: Enable fetch from oc new-app, start-build, source lookup #14025
  • builds: Fix overlap between branch and ref names #14103

Reference OpenShift image streams from Kubernetes resources

OpenShift image streams make it easy to decouple image management from deployment. Image streams can now be used directly from Kubernetes resources like StatefulSets, Jobs, CronJobs, Deployments, or DaemonSets via the new lookupPolicy that has been added to image streams. The oc set image-lookup command allows you to mark an image stream within your project as being a local reference:

$ oc import-image mysql:latest
$ oc set image-lookup mysql

Now you can reference the image mysql:latest from within a Kubernetes controller and the corresponding image stream tag will be used:

$ oc run --image=mysql:latest --restart=OnFailure myjob
$ oc get pods

The pod created by the job will use the image tagged as latest in the image stream mysql. Builds, pods, jobs, replicasets, and replication controllers will all respect these settings, and administrators can configure their image policy to add new resources.

  • Extended test for local name resolution #13210

Trigger updates to Kubernetes deployments and daemon sets on image change

The image trigger controller has been upgraded and now supports updating deployments, stateful sets, daemon sets, and cron jobs whenever an image stream tag is updated. A new alpha annotation can be set on the resource to describe which image stream tag should cause an update.

To update a DaemonSet whenever the image stream tag 'image:latest' in namespace 'namespace1' changes, run:

$ oc set triggers daemonset/monitoring --from-image=namespace1/image:latest -c main

You can remove a trigger by adding the --remove flag. This allows you to run an OpenShift build
or run a scheduled import to keep your applications up to date..

Trigger updates are alpha in 3.6.0.

  • Update Kubernetes resources on image change #13242

Build cleanup policy

Build configs now support two parameters to control how many successful and failed builds are retained. By default, no limit is set and all builds are retained. The fields are spec.failedBuildsHistoryLimit and spec.successfulBuildsHistoryLimit.

  • Cleanup policy for builds #13788

Unification with Kubernetes authorization and core code

The v3.6.0 release integrates a large number of changes the OpenShift team has contributed to
Kubernetes around security, authorization, RBAC, and core code refactoring. While this will continue for several releases, the OpenShift and Kubernetes RBAC resources are being aligned and the primary API going forward for RBAC will be the Kubernetes version. The existing APIs will remain, especially those that expose features not yet supported in Kubernetes like scoped tokens.

Starting in this release, all OpenShift RBAC resources are automatically migrated to Kubernetes RBAC resources. Users should not see any change in behavior while these migrations occur.

Future releases will include migrating from SecurityContextConstraints to PodSecurityPolicy as well naming and policy updates to system managed policies.

  • Use upstream system:masters authorizer #14006
    • Switch to policy watch #14194
  • Use upstream namespace cleanup controller #13587
  • Use bootstrap cluster roles from kube #14026
  • Use upstream x509 request header authenticator #14007
  • Use upstream initialization for most controllers #14126
  • Use upstream remote authentication and authorization #14011
  • Use upstream initialization for the replication controller #14033
  • Synchronize OpenShift RBAC to Kubernetes RBAC #14064
    • Split resource and non-resource rules during conversion #14454

End to end TLS - never generate a certificate again!

The service serving certificate feature makes it easy to generate a valid TLS server certificate for your applications for securely serving HTTPS within a cluster. The router is now enabled to automatically reencrypt traffic to services that use these certificates for routes that specify a blank spec.tls.destinationCACertificate field. When a cluster is configured with a default wildcard certificate, this means that you can deploy applications to OpenShift that are secured end to end without having to generate or manage your own certificates.

See the Prometheus example
for this in action. The service requests generation of a secret prometheus-tls containing a TLS serving certificate for prometheus.NAMESPACE.svc, and the route points to the service with the TLS type Reencrypt, but without a destinationCACertificate. The router will automatically fill in the service-ca.crt file, which is available in every pod.

Sign and verify image signatures

OpenShift has natively supported detached signing certificates on images for several releases, and this release adds a new CLI command to make it easy for an administrator to verify the signatures on images manually or as part of an automated image publish flow. This allows the web UI and CLI to show information to the end user about the state of the signature.

See oc adm verify-image-signature for more.

  • Verify signatures #13585
  • cli: do not require --expected-identity when removing all signatures #14125

Other Features

  • build: Display jenkins url for pipeline build #13979

  • registry: Add prometheus metrics for dockerregistry #12711

    • Force to specify not empty secret for metrics endpoint #13884
  • route: Sanitize certificates from routes in the router #13897

  • storage: iSCSI CHAP support #14112

  • route: Allow controlling via a new permission #13905

  • prometheus: Make the Prometheus example a fully automated secure deployment #13782

  • ha: Enable leader election on endpoints for controllers #14094

  • router: Add defaults and env control of the fin timeouts in the router #14220

  • router: Make HAProxy's log format configurable #13029

  • sdn: Add an OPENSHIFT-ADMIN-OUTPUT-RULES chain for admins to use #14221

  • server: Prepare for API aggregation by supporting the new Kube aggregation endpoint #14285

  • router: Shuffle endpoints for routes #14008

  • web: Update overview to use toast notifications #1654

  • web: Add landing page tour #1508

  • web: Support filtering provisioned services on overview #1444

  • web: Only show service catalog resources when available #1573

  • web: disambiguate kinds on other resources by showing group #1478

  • web: Bug 1447997 - Show warning for unsupported resource versions #1512

  • web: Update the Other Resources Page to only show resources supporting the 'List' verb #1572


  • admission: Support legacy admission configurations without kind fields set #14272
  • authn: Allow service account tokens to be used with WebSocket connections #13978
  • builds: Allow GIT_SSL_NO_VERIFY to be set on build pods via build defaulter #13797
  • builds: Apply build resource defaults to the build pod #13825
  • builds: Better API documentation for image source build behavior #13781
  • builds: Ensure build start time is always set #14131
  • builds: Fix potential hangs if the Docker daemon is overloaded #13817
  • builds: Retry build instantiation on conflict #13910
  • builds: Use credential provider to load image pull secrets #10608
  • cli: Add request-timeout val to oc login restclient #12062
  • cli: Don't show policy rules with attribute restrictions #14034
  • cli: Ensure more oc set commands support --dry-run and --local #14123
  • cli: Fix template objects describer #14207
  • cli: Improve output of oc adm manage-node --list-pods #12528
  • cli: oc status should display services of type ExternalName correctly #14448
  • clients: Fix deployclient imports #14203
  • clients: Generated clients for the images API group #14042
  • cluster: Set docker cgroup driver on kubelet config #13964
  • controllers: Refactor serviceaccount and rest of build controllers to new controller initialization #14293
  • controllers: Rename controller files to be easily distinguishable in logs #13699
  • deploy: Better deployment cancellation message #13813
  • deploy: Clean up log messages from deployment controllers #13762
  • deploy: Don't block triggering deployment when ICT is updated #13886
  • deploy: Suggest cancelling DC instead of RC in oc deploy #14019
  • deploy: rewire deployment controllers initialization to use a controller init func #13996
  • diagnostics: Perform network diagnostic checks if we are able to launch at least 50% of test pods. #13851
  • dns: Add node config option for a resolv.conf to read #14297
  • egress: Add backward compatibility for the old EgressNetworkPolicy "" bug #13822
  • egress: Bug 1445694 Fix locking in syncEgressDNSPolicyRules() #13965
  • examples: Remove references to personal repositories #14178
  • image: Fix prioritizing of semver equal tags #14248
  • image: Set layer size whether it found in cache or not #14166
  • image: Add support for Node.js 6 (official) #13967
  • image: Use docker image reference from ImageStream #13639
  • namespace: Increase concurrency for namespace cleanup #14352
  • newapp: Triggers should not be set when creating new builds from docker images #13807
  • node: Error on connecting to Docker daemon was being silently dropped #14162
  • openid: Use correct base64 scheme to decode id_token #14420
  • project: Bug 1454535 - Use created project name over namespace name in project template #14344
  • proxy: Add locking around userspace map #13847
  • prune: Prefer secure registry connections when pruning from a registry #14114
  • prune: Prune external images by default #13900
  • quota: Controller was not checking for compensation at expected interval, quota was double counted #14151
  • quota: Separate image quota evaluation for admission versus reconciliation #14345
  • router: Add proxy protocol status to reload script output #14256
  • router: Fix panics from routes being out of order #14232
  • router: Increase max request size for HAProxy to be comparable to cloud LBs #13792
  • router: Match subpaths correctly when path contains trailing slash #13867
  • router: Prevent the router from deadlocking itself when calling Commit() #13717
  • router: Reduce log spam from default certificate check #13514
  • router: Simplify router template sections for edge and reencrypt routes #14242
  • router: Support reencrypt routes in F5 #13898
  • router: Syntax error in irule #14223
  • router: adding X-Forwarded-For header to reencrypt route #14142
  • scheduler: Include DefaultTolerationSeconds admission plugin in OpenShift #14118
  • sdn: Ensure multicast rules are cleaned up when net namespace is deleted #14231
  • sdn: Fix NATting of external traffic with ovs-networkpolicy #13877
  • sdn: Fix initialization order to prevent crash on node startup #13766
  • sdn: Fix service IP validation to handle "ClusterIP: None" #13765
  • sdn: Network policy pod watch should ignore pods with HostNetwork set to true #14030
  • sdn: Refactor ClusterNetwork creating/updating/validating #13951
  • sdn: Traffic leaking out of the cluster #13680
  • security: Add projected volume plugin into correct SCCs #14147
  • security: Give docker builders access to optimized image builds #14323
  • security: Prevent new project creation with openshift/kubernetes/kube prefixes #13673
  • security: Remove obsolete pod permissions from the deployment config controller #14288
  • security: Strip proxy credentials when logging proxy env variables #13751
  • security: Use a common function to populate user in subject access review correctly #14304
  • security: Allow most users to view StorageClasses #14209
  • server: Hold startup until etcd has stabilized cluster version #14095
  • server: Start using generated informers #13982
  • server: only log stacks on server errors #14173
  • template: Handle legacy groups better in template processing #13791
  • template: Ignore namespace when processing templates #13725
  • template: Make template service broker namespace(s) configurable #13872
  • template: Template broker checks permissions for users instead of impersonating #14216
  • tests: Add w.close for watch #14271
  • web: Fix error editing build config to push to new tag #1424
  • web: Fix runtime error on browse builds page for non-Git builds #1426
  • web: Hide "Start Build" actions for binary builds #1427
  • web: Always show latest deployment on overview #1429
  • web: Fix overview notifications #1430
  • web: Incorporation of UI animation effects for deployment transitions on overview. #1402
  • web: Set donut alignment based on whether metrics are shown and prevent deployment animation from overlaying alerts within expanded row #1441
  • web: Hide failed and cancelled deployments #1433
  • web: Switch to creating a DeploymentRequest to rollout a new deployment of a DC #1434
  • web: Improve layout of provisioned services on overview #1436
  • web: When maxSurge and maxUnavailable are numbers or not set, put right format into JSON before submitting #1442
  • web: Don't repeat overview "Other Resources" with multiple pipelines #1446
  • web: Improving copy-to-clipboard display #1451
  • web: Switch to code authorization flow #1342
  • web: Support creating templates in a dialog #1492
  • web: Remove some (now incorrect) warnings when deleting resources #1530
  • web: Improve delete dialog message #1544
  • web: Check existing storage quotas and adjust UI appropriately #1217
  • web: Sort deployments on the overview #1556
  • web: Add DC paused message to overview #1545
  • web: Fix details message for paused deployments #1577
  • web: Adding instructions on how to view the API token #1575
  • web: Bug 1451013 - Show Events on PVC detail page #1587
  • web: Don't switch selected pod log container on watch updates #1592
  • web: Remove now incorrect warning when deleting DCs #1619
  • web: Bug 1455105 - Correct quota / HPA donut warnings #1612
  • web: Handle project name being changed by project template when ordering template #1639
  • web: Fix single, unnamed port warning when editing routes #1623
  • web: Hide error notifications when form is resubmitted #1567
  • web: Prompt users when navigating away with unsaved changes #1104
  • web: Prevent cancel from submitting edit dc form #1656

Release SHA256 Checksums

9cc44e7890b39953303ad18f2187a6aac82cd3a6fe570b9432c47df982589075  openshift-origin-client-tools-v3.6.0-alpha.2-3c221d5-linux-64bit.tar.gz
42a56ee6f66e39815874c8e03c1cee373e20ff3f1fd83c5829df043c8988ba0d  openshift-origin-server-v3.6.0-alpha.2-3c221d5-linux-64bit.tar.gz
Assets 7
You can’t perform that action at this time.