fix(provider/aws): Avoid being marked unhealthy after initialization #2245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Avoid being marked unhealthy should there be a temporary blip in connectivity to one or more accounts. This frequently occurs as a result of an AWS rate limit being hit. The `health.amazon.errors` metric can be used to alert on possible errors that would have previously caused an instance to go unhealthy.
This was referenced Jan 3, 2018
dkirillov
added a commit
to lookout/clouddriver
that referenced
this pull request
Jan 10, 2018
* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020) - fixed NPE due to missing credentials in security group description request * feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005) * feat(google): Instrument individual google API calls. (spinnaker#2016) * fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022) * fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023) * fix(aws): Only autocreate app elb security group on Create (spinnaker#2025) - Updated to only auto create <appname>-elb group for new ELBs * feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026) Expectation is that a `subnetType` will still be provided and that `subnetIds` will be a subnet of those valid for `subnetType`. When explicit subnet ids are provided (or inherited), the newly created server group will be tagged with: `SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"` * fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027) * fix(eureka): fix instanceId on eureka caching The instanceId was intended to be the value in dataCenterInfo.metadata but due to some Jackson behaviour the top level instanceId from the discovery registration was written over the dataCenterInfo.metadata during deserialization. This changes to explicitly handle the top level instanceId value in the JsonCreator factory method to prevent the unhandled property behaviour from kicking in. * feat(provider/kubernetes): use kubectl to handle deploy apply (spinnaker#2031) * feat(core): Manifest provider (spinnaker#2032) * fix(provider/kubernetes): fixes ClassCastException on resize operations (spinnaker#2024) In what appears to be the absence of another v1 resize validator, KubernetesResizeServerGroupValidator was being picked up which wasn't correctly declaring its support for v2 only. * feat(provider/kubernetes): v2 manifest provider (spinnaker#2033) * chore(dependencies): Bump spinnaker-dependencies version and pin google (spinnaker#2036) api client lib. * feat(provider/kubernetes): support delete and update strategies opera… (spinnaker#2021) * feat(provider/kubernetes): support delete and update strategies operation for new controllers. * feat(provider/kubernetes): added partition and podManagementPolicy fields. * feat(provider/kubernetes): added deletion parameters. * feat(provider/kubernetes): fixed format problem and cleanup texts. * fix(appengine): remove batch calls, revert client library (spinnaker#2037) * refactor(provider/kubernetes): ignore api versions (spinnaker#2038) * feat(provider/kubernetes): moniker status (spinnaker#2039) * fix(startup): Kubernetes manifest provider autowired ambig bean (spinnaker#2040) * fix(provider/aws): No-op ami tag update when no tags provided (spinnaker#2045) The AWS `createTags` API requires at least one tag to be provided. Rather than error out, let's just skip the update entirely and return success. * fix(provider/kubernetes): collectJob status ClassCastException (spinnaker#2029) * fix(provider/kubernetes): allow to configure uwith service account. (spinnaker#2044) * feat(provider/kubernetes): depend on kubectl where possible (spinnaker#2041) * feat(provider/google): Support UHCs in L7 load balancers. (spinnaker#2030) * fix(provider/gce): Fix GCE destroy backend calls on LB cache misses. (spinnaker#2048) * Closed spinnaker#2072 - Implement GoogleExecutorTraits and wired in spectator. (spinnaker#2049) * fix(amazon): do not copy location-specific policy actions cross-account (spinnaker#2046) * Create README.md (spinnaker#2043) * feat(provider/kubernetes): v2 load balancer provider (spinnaker#2050) * fix(provider/kubernetes): v2 expire logical keys (spinnaker#2051) * feat(provider/kubernetes): instance provider (spinnaker#2053) * fix(javadoc): Load balancer provider typo (spinnaker#2054) * fix(amazon/alb): Fix rule comparison so listeners do not get updated when they are the same (spinnaker#2047) * fix(provider/gce): Fix NPE if no legacy health checks. (spinnaker#2057) * feat(cats): dynomite version bump to stable (spinnaker#2056) * fix(aws): Flagging elb security group autocreation (defaults to off) - made auto creation of <appname>-elb group for new ELBs configurable - defaulting to off * refactor(artifacts): artifact config lives under artifacts (spinnaker#2059) BREAKING CHANGE * fix(provider/google): Cache and propagate GCE numeric instance id. (spinnaker#2058) * fix(gce/defaults): instanceType vs. instance-type (spinnaker#2061) * fix(amazon/alb): Reorder/refactor alb upsert to handle updating port and protocol (spinnaker#2062) * perf(cluster): Optimize target/LARGEST lookups (spinnaker#2064) If `ClusterProvider.supportsMinimalClusters()` is true, we can now sort and filter on the set of unexpanded server groups. Only the actual LARGEST server group will be expanded. This offers significant savings for target/LARGEST lookups on clusters with > 1 server group, particularly those containing _many_ instances. A target/LARGEST lookup commonly occurs when doing a clone operation with `useSourceCapacity` set to `true`. * fix(provider/gce): Update L7 delete for UHCs. (spinnaker#2066) * fix(provider/kubernetes): allow cache data to be serailized/deserialized (spinnaker#2052) * fix(config): SpringApplicationBuilder().showBanner() does not exists in Spring Boot 1.4.1.RELEASE (spinnaker#2018) * feat(cats): Add keyspaces support to dynomite backend (spinnaker#2042) * feat(provider/ecs): ECS Account Credentials (spinnaker#2034) * Added EcsCredentialsInitializer and supporting classes to enable the ECS module to have an Amazon ECS account. * Created NetflixESCredentials. * Typo in class name fixed. * Rename EcsAccountMaker to EcsAccountBuilder, as well as the method. * Test renamed. * EcsAccountBuilderTest didn't refactor properly - fixed. * Naming fixed. * feat(provider/kubernetes): automatically configure caching agents (spinnaker#2067) * feat(provider/kubernetes): v2 register pod handler (spinnaker#2069) * feat(provider/kubernetes): register deployment caching agent (spinnaker#2070) * fix(provider/openstack): Fix type error in LoadBalancerSummary (spinnaker#2071) * fix(provider/kubernetes): Dedup kind map (spinnaker#2072) * feat(provider/kubernetes): v2 statefulset support (spinnaker#2074) * fix(provider/amazon): Only include non-terminating instances (spinnaker#2075) When building a set of partial instances, only include those that are not pending or in-service. * fix(provider/amazon): Include partial instances iff includeDetails = false (spinnaker#2077) No need to look at partial instances when instance relationships have been populated. Loosening the definition of a partial instance to be those that are `InService` (was Inservice + Pending). * fix(cats/redis): Don't hash ttl'd keys (spinnaker#2076) The issue here is that when a key with a ttl is hashed, the hash persists forever, even after the key disappears. The obvious fix would be to associate the hash with a ttl as well, but as far as I can tell, there is no good way to ensure that the hash will always expire when/before the key does without introducing some subtle race. There's not a lot of keys that do have a ttl, so I don't see this having a large performance impact. It feels like the cleanest/most contained way to deal with the problem described above. * chore(gradle): Avoid running tests on master/release branch (spinnaker#2078) * chore(provider/kubernetes): improve v2 cache logging (spinnaker#2079) * fix(cats/dynomite): Don't hash ttl'd keys (spinnaker#2080) * fix(provider/kubernetes): use location over namespace in op (spinnaker#2082) * fix(provider/amazon): Commentary on when includePartialInstances = false (spinnaker#2081) * fix(provider/gce): Distinguish instance id from gceId. (spinnaker#2084) * feat(provider/kubernetes): check controller type for disable/enable operation (spinnaker#2068) * feat(provider/kubernetes): check controller type for disable/enable oepration. * feat(provider/kubernetes): change function name and minor code refactor. * feat(provider/kubernetes): v2 ingress support (spinnaker#2085) * perf(provider/gce): Replace L7 health check and backend service get() with list(). (spinnaker#2073) * fix(core): Handle potentially null `.instances` when target/LARGEST (spinnaker#2086) * feat(provider/kubernetes): allow users to skip creating pull secrets (spinnaker#2088) * feat(provider/ecs): ECS Cache base classes (spinnaker#2065) * Prep classes for the caching agents to come. * Changes based on feedback. * fix(provider/gce):Enable RPS when only RATE or UTILIZATION (spinnaker#2083) * feat(provider/kubernetes): v2 cache namespaces & relationship (spinnaker#2087) * feat(provider/kubernetes): V2 enable minimal clusters (spinnaker#2090) * feat(provider/aws): Enable optional AWS Shield protection on ELB & ALBs (spinnaker#2089) * feat(provider/amazon): block device config for c5 instance types (spinnaker#2092) * fix(provider/gce): Ensure referenced instances exist during L4 delete. (spinnaker#2094) * feat(provider/kubernetes): cache network policies (spinnaker#2095) * feat(provider/kubernetes): v2 server group manager (spinnaker#2093) * feat(provider/amazon): Optionally tag server groups w/ app/stack/details (spinnaker#2096) Set `aws.defaults.addAppStackDetailTags: true` to enable. When enabled, all created server groups will be tagged with: - `spinnaker:application` - `spinnaker:stack` - `spinnaker:details` The tag will only be created if the value is non-null. By default, these tags will be propagated to launched instances. * fix(provider/kubernetes): fix v2 instance lookup (spinnaker#2098) * feat(provider/ecs): ECS Cluster caching classes and tests. (spinnaker#2091) * ECS Cluster caching classes and tests. * Base abstraction. * feat(provider/kuberentes): v2 security group provider (spinnaker#2097) * feat(provider/kubernetes): v2 pod logs (spinnaker#2099) * fix(provider/kubernetes): security group yaml (spinnaker#2100) add yaml field to KubernetesV1SecurityGroup so that the "Show YAML" modal works in deck. * feat(kubernetes/artifacts): Support deploying images dynamically (spinnaker#2102) * fix(provider/kubernetes): docker is not a hard-dependency (spinnaker#2106) * fix(provider/amazon): Do not delete listeners after updating them. Pretty bad idea. (spinnaker#2105) * feat(provider/kubernetes): split out k8s providers (spinnaker#2107) Needed to separate the search endpoints for each provider * feat(provider/kubernetes): v2 search endpoint (spinnaker#2108) * feat(provider/kubernetes): Allow op to override versioning (spinnaker#2110) * feat(provider/kubernetes): Add kubectl to Dockerfile. (spinnaker#2104) * fix(provider/gce): Handle partially-formed L7 LB resources properly. (spinnaker#2111) * fix(provider/kubernetes): fix application key parsing (spinnaker#2109) * perf(provider/gce): s/get/list/ in ILB caching agent. (spinnaker#2112) A few small fixes/guards to the L7 caching agent as well. * feat(provider/kubernetes): remove dependency on api-client (spinnaker#2113) * feat(provider/kubernetes): reenable kube monitoring (spinnaker#2115) * feat(core/search): Add parameter for fetching a batch of servergroups by name (spinnaker#2114) * fix(provider/gce): Fix TCP LB disabled state calculation. (spinnaker#2116) * perf(provider/gce): s/get/list/ in TCP/SSL LB caching agents. (spinnaker#2117) * feat(provider/kubernetes): v2 rollback (undo rollout) (spinnaker#2118) * feat(provider/kubernetes): scale manifest (spinnaker#2119) * feat(provider/kubernetes): k8s only search provider (spinnaker#2121) * fix(docker): Use wget instead of curl (spinnaker#2122) * fix(provider/kubernetes): read namespace as region (spinnaker#2123) * feat(provider/kuberentes): pause rollout (spinnaker#2120) * feat(provider/kuberentes): v2 non-enum k8s kinds (spinnaker#2125) * feat(provider/kubernetes): search result hydrators (spinnaker#2124) * feat(provider/kubernetes): resume rollout (spinnaker#2126) * fix(provider/kubernetes): v2 fix cred validation (spinnaker#2129) * fix(perf): Avoid manually copying bytes between streams (spinnaker#2127) * fix(provider/kubernetes): autowire correct objectmapper (spinnaker#2133) * fix(provider/kubernetes): attach correct op decorators (spinnaker#2130) * fix(provider/kubernetes): switch access modifiers for KubernetesKind constructors (spinnaker#2128) * feat(provider/kubernetes): runJob multi-container (spinnaker#2135) support multiple containers in the run job description. we have to support both `container` and `containers` until we can deprecate the `container` field since the current defs are all using `container`. * feat(entitytags): Support indexing / searching by application (spinnaker#2132) If `entityRef.application` is not explicitly provided, it will be parsed from `entityRef.entityId` using frigga conventions. Searching by application can be accomplished by `/tags?application=my_application` * fix(provider/gce): Tolerate named port values as Doubles. (spinnaker#2137) Pipelines are being persisted with 'port' values of NamedPorts as Doubles. This is an issue. This change tolerates that behavior in deploys since users will have pipelines with Double ports in the wild. * fix(provider/dcos): Use createApp instead of updateApp for deploy (spinnaker#2136) * fix(provider/dcos): Set container type appropriately. * fix(provider/dcos): use createApp instead of updateApp for deploy * fix(provider/dcos): Remove dcos client proxy exception wrapping * Added tests for Keys. Changed separator. (spinnaker#2101) * fix(provider/kubernetes): fix security group loading by name (spinnaker#2134) * fix(kubectl): --to-revision (spinnaker#2140) * feat(provider/kubernetes): allow OAuth token authentication (spinnaker#2147) * refactor(provider/openstack): Updated openstack4j to 3.1.0 and removed components that are available upstream (spinnaker#2103) * fix(elasticsearch): Force refresh when fetching entity tags from Front50 (spinnaker#2149) `elasticsearch_index_template.json` is the index template that should be applied in elastic search. * fix(provider/aws): Fix updating ALB listeners (spinnaker#2148) * feat(provider/google): Label instances so they can easily be grouped later. (spinnaker#2142) * feat(provider/kubernetes): more robust status (spinnaker#2152) * feat(provider/kubernetes): default to frigga (spinnaker#2156) * feat(provider/kubernetes): v2 configmap support (spinnaker#2157) * feat(provider/kuberentes): send deployed artifact to orca (spinnaker#2158) * feat(provider/kubernetes): v2 support secrets (spinnaker#2159) * feat(provider/kubernetes): register docker image replacers (spinnaker#2160) * feat(provider/kubernetes): v1 pod logs (spinnaker#2162) support pod logs for v1 provider. going to be useful when working on run job logs as well. * feat(provider/kubernetes): v2 support daemonset (spinnaker#2161) * fix(provider/gce): Tolerate failed backend service getHealth() calls. (spinnaker#2153) * fix(provider/gce): Break out labels. (spinnaker#2163) * fix(provider/aws): Should use edda to lookup target groups by name (spinnaker#2165) * ECS Container Instance caching classes and tests. (spinnaker#2143) * fix(provider/aws): Remove invalid `spinnaker:` tags (spinnaker#2168) Remove any `spinnaker:*` tags that are no longer valid (ie. cloning across stacks and `spinnaker:stack` no longer points at the correct stack). This is necessary as `deck` will _now_ supply existing tags when cloning. * feat(provider/kubernetes): runJob logs (spinnaker#2164) add support for capturing logs as part of `collectJob`. logs are added to the job status. i'm not sure how well this scales as log size grows so this may not be the best implementation, but it may work for the short term. since execution details are persisted to volitile storage storing them with the jobs status is best since the reference to the job pod is also stored there. * refactor(cf): Stop building CF provider until maintainer is found (spinnaker#1942) BREAKING CHANGE: Stops support of CF until maintainer is found * bug(provider/openstack) - LoadBalancer security groups are optional. (spinnaker#2154) * chore(dependencies): Bump to 0.123.0 (spinnaker#2169) * feat(provider/kubernetes): load artifacts alongside manifest (spinnaker#2166) * refactor(provider/kubernetes): allow api version subclassing (spinnaker#2167) * chore(jobs): turn down log-level (spinnaker#2171) * fix(web): Do not throw exception when sg not found (spinnaker#2172) * feat(provider/ecs): ECS Task Definition caching classes and tests. (spinnaker#2145) * ECS Task Definition caching classes and tests. * ObjectMapper is now injected. * refactor(entitytags): Support for alerts/notices on any entity type (spinnaker#2175) * feat(cats): Support for selectively enabling agents (spinnaker#2177) This is an assist to local development when you do not want _all_ agents to run by default. ``` redis: agent: enabledPattern: .*reservation.* ``` By default `enabledPattern` is `.*` and all agents will be scheduled. * Use setApiKey when using token from service account (spinnaker#2178) * fix(provider/kubernetes): v1 add docker registries in all cases (spinnaker#2176) * bug(provider/openstack) - Display VIP for LBs that do not have Floating IP instead of null.. (spinnaker#2170) * fix(provider/kubernetes): don't throw on invalid lb op (spinnaker#2184) * fix(provider/aws): Reservation caching agent, now with less RxJava (spinnaker#2180) * feat(provider/kubernetes): replace deployed configmap volumes (spinnaker#2183) * feat(xenial_debians): Add systemd service configuration. (spinnaker#2182) * feat(xenial_debians): Add systemd service configuration. * feat(xenial_builds): Fix typo. * feat(provider/ecs): ECS Service caching classes and tests. (spinnaker#2144) * ECS Service caching classes and tests. * Objectmapper is now injected. * feat(provider/aws): Support expected capacity constraint when resizing (spinnaker#2179) Support a constraint wherein the server groups' current capacity must match expected prior to a resize. There are scenarios wherein a user initiates a resize via the UI at the same time that an autoscaling event is happening behind the scenes. In such a scenario, the resize via UI != autoscaling and some churn will occur. `deck` will be changed to supply the capacity constraint. * fix(provider/aws): Retry createAutoScalingGroup() / updateAutoScalingGroup() (spinnaker#2185) * feat(provider/kubernetes): cache controller revisions (spinnaker#2138) * fix(provider/kubernetes): fix registry init (spinnaker#2188) if using the `namespaces` key for kubernetes accounts, reconfigure registries using those namespaces. * feat(provider/kubernetes): v2 support labels on delete & scale (spinnaker#2190) * Xenial builds (spinnaker#2189) * feat(xenial_debians): Add systemd service configuration. * fix(provider/kubernetes): fix onDemand cache update in controller agent. (spinnaker#2131) * chore(dependencies): update to latest spinnaker-dependencies Also updates gradle to 3.5 * fix(provider/kubernetes): fix registry init 2 (spinnaker#2191) the previous fix caused sever performance issues. this fix will only configure hard coded registries once, only reconfiguring failed registries on the next loop, if any. * fix(google): Retry on all 5xx errors from the platform. (spinnaker#2193) * fix(core): fix bean mismatch due to Spring update removes unnecessary corsfilter bean - clouddriver should not be hit by browser agents * chore(warnings): fix compiler warnings * feat(provider/ecs): ECS Task caching classes and tests. (spinnaker#2146) * ECS Task caching classes and tests. * ObjectMapper is now injected. * fix(cats): fix conditional instantiation of EurekaNodeStatusProvider * fix(build): fix spring-boot plugin usage * refactor(kubernetes): change OAuth token configuration (spinnaker#2196) * fix(provider/aws): s/contraints/constraints (spinnaker#2198) * fix(provider/kubernetes): fix envvar source (spinnaker#2199) the bump in client library broke the API for configMap and secret envvar sources. adds an extra property for optional. defaults to true to preserve backwards compatibility. shoutout major version bumps. * chore(logging): remove extra logging (spinnaker#2200) * feat(provider/gce): Support for RMIG zone selection. (spinnaker#2201) * chore(systemd_logs): Cleanup unneeded logging in systemd config. (spinnaker#2202) * feat(provider/ecs): ECS IAM Role caching classes and tests. (spinnaker#2150) * ECS IAM Role caching classes and tests. * Moved Role and the associated classes into clouddriver-aws. * AWS' default region is being used. Cleanup. * refactor(clouddriver-core) Fixed to use const value (spinnaker#2186) * fix(provider/kubernetes): registry init fix (spinnaker#2205) for real this time. the previous fix caused registry namespaces to get set to an empty list the second time the initialization occurred. moving it down into this method allows us to still use the full list of namespaces (`allNamespaces`) and setup the affected ones to be initialized. This would occur if a new namespace was added, for instance. * fix(amazon/loadBalancers): Be smarter about updating ALB listeners (spinnaker#2203) * fix(provider/dcos) Catch the correct exception for the proxy. (spinnaker#2207) * chore(dependencies): update to 0.128.0 (spinnaker#2208) * fix(core/search): Fix search query which matches non-conforming cache keys (spinnaker#2209) Certain cache keys were failing due to Keys.parse() returning null. Turns out there is no need to parse the cache key and then reconstruct it. * feat(provider/kubernetes): support envFrom (spinnaker#2213) adds support for envFrom for the v1 provider. envFrom can be either a secret or config map reference and will be injected into the pod as environment variables. * fix(provider/gce): Tolerate null instance template lists. (spinnaker#2216) * feat(aws/loadBalancer): Support on demand caching for target groups (spinnaker#2214) * refactor(provider/kubernetes): refactor delete pods operation (spinnaker#2210) * feat(artifacts): sets up artifact module (spinnaker#2218) My plan was to reuse the artifact work in appengine, but it would have been too messy to refactor. It didn't expose any artifact-centric interface or allow easy integration with other artifact sources without a fair bit of extra work, so I rewrote in a separate module. T he appengine GCS integration was left unchanged (with the small change of moving the storage utils into a 'AritifactUtils' class). * chore(dependencies): Bump spinnaker-dependencies version. (spinnaker#2220) Picks up the new compute library versions. * fix(provider/amazon): render IPv6 ingress security group rules (spinnaker#2212) * feat(provider/appengine): allow gcloud version to be configurable (spinnaker#2217) * feat(provider/kubernetes): init containers (spinnaker#2219) adds support for initContainer property of the Pod spec * fix(provider/kubernetes): s/name/manifestName (spinnaker#2222) * fix(provider/kubernetes):Server Group is disable in Daemonset when replic set to 0 (spinnaker#2206) * feat(provider/kubernetes): deploy from artifact (spinnaker#2223) * fix(provider/appengine): fix monikers for app engine server groups (spinnaker#2224) * fix(provider/amazon): correctly propagate ipv6 ingress security group rules on upsert (spinnaker#2225) * fix(provider/aws): block device configs for h1.* and i3.* (spinnaker#2227) * fix(registry): Handle situations where tag creation date cannot be fetched (spinnaker#2228) * fix(provider/amazon): ignore description when upserting security group rules (spinnaker#2229) * bug(provider/openstack) - filter images by region if provided. (spinnaker#2230) * fix(entitytags): Retry support when fetching entity tags from Front50 (spinnaker#2232) * fix(core): Create the 'RetrySupport' bean (spinnaker#2233) * feat(provider/kubernetes): return modified manifests (spinnaker#2234) * fix(dockerRegistry): handle 400 for token refresh (spinnaker#2215) By the spec https://docs.docker.com/registry/spec/api/#base, 401 should be used to indicate an expired token, but it seems 400 is used as well. * feat(provider/kubernetes): ds & ss stability conditions (spinnaker#2238) * fix(appengine): remove batch calls, revert client library (spinnaker#2221) This reverts commit 8db9514. * feat(provider/kubernetes): return deployed manifests (spinnaker#2235) * feat(provider/kubernetes): v2 make rollback dynamic (spinnaker#2237) * fix(provider/kubernetes): disable v1 controller caching (spinnaker#2239) This had a chance of deleting k8s resources from the cache when used with daemonsets & statefulsets, since there is no type information provided with a resource name. * chore(dependencies): bump spinnaker-dependencies to 0.131.0 (spinnaker#2242) - newer aws sdk * feat(artifacts): Download artifacts from GitHub (spinnaker#2231) * feat(provider/gcp): Add ability for default app creds to impersonate … (spinnaker#2240) * feat(provider/gcp): Add ability for default app creds to impersonate a service account * duftler comments * fix(provider/aws): Avoid being marked unhealthy after initialization (spinnaker#2245) Avoid being marked unhealthy should there be a temporary blip in connectivity to one or more accounts. This frequently occurs as a result of an AWS rate limit being hit. The `health.amazon.errors` metric can be used to alert on possible errors that would have previously caused an instance to go unhealthy. * feat(provider/appengine): enable flex deployments (spinnaker#2241) Allow App Engine deployments with a built container image and app.yaml. * fix(startup): make constructor bean optional (spinnaker#2247) * fix(clusters): Return all matching server groups for a provider (spinnaker#2249) * fix(provider/kubernetes): v2 Fix kubectl label selector flag (spinnaker#2252) Currently we set `-l=` when `labelSelectors` is null or is not empty, causing all commands that pass a null value for `labelSelectors` to fail. The `-l=somelabel` flag should be set if `labelSelectors` is not null and is not empty. * fix(provider/aws): handle spotPrice == in rollingpush (spinnaker#2251) * debug(provider/aws): Avoid thread pool when building reservation report (spinnaker#2253) Attempting to track down an issue wherein the reservation report caching agents _stop_ and are never rescheduled after some period of time. Will revisit and remove the thread pool configuration if this change happens to improve the stop/reschedule situation. * debug(provider/aws): Additional logging for `ReservationReportCachingAgent` (spinnaker#2254) * fix(provider/aws): Specify `connectionRequestTimeout` for edda (spinnaker#2255) Establishes an upper bounds on the amount of time to spend waiting on an available connection from the `edda` connection pool. Previously, a caching agent could wait _indefinitely_ for an available connection. The default is now 10000 (milliseconds) and can be overridden with `aws.edda.connectionRequestTimeout`. * fix(provider/amazon): do not try to create reserved tags (spinnaker#2256) * fix(provider/aws): Ensure that `HttpEntity` is closed on edda failure (spinnaker#2258) Also log an error when edda retries have been exhausted. * bug(provider/openstack) - allow for binding to multiple loadbalancers with the same port mappings. (spinnaker#2257) * fix(provider/aws): block device config for missing instance types (spinnaker#2266) This should provide coverage for all currently available instance types. Will look for better ways to manage this moving forward. * fix(provider/kubernetes): allow empty serviceName in statefulset (spinnaker#2261) * chore(provider/kubernetes): log malformed cache entries (spinnaker#2262) * feat(provider/kubernetes): v2 ignore caching flag (spinnaker#2268) * feat(provider/kubernetes): return resource create time (spinnaker#2269) * feat(provider/kubernetes): fail deploy if artifacts don't bind (spinnaker#2271) * feat(authz): requiredGroupMemberships to permissions migration (spinnaker#2272) update AllowedAccountsValidator to use permissions config in addition to requiredGroupMemberships * fix(provider/kubernetes): Replace artifact only if target found (spinnaker#2259) Do not attempt artifact replacement if no target was found. Reading a non-existing path can return an empty ArrayNode object, in which case we should not attempt any replacement. Add checks for bound artifacts in artifact replacement tests. * feat(provider/kubernetes): split out required & optional artifacts (spinnaker#2273) * fix(provider/docker): update bearer_token to access_token (spinnaker#2274) * fix(provider/kubernetes): Fixed statefulset and daemonset volumesource bugs. (spinnaker#2276) feat(provider/kubernetes): split out required & optional artifacts (spinnaker#2273) fix(provider/docker): update bearer_token to access_token (spinnaker#2274) * fix(provider/kubernetes): annotate artifacts with reference (spinnaker#2277) * feat(provider/kubernetes): only redeploy versioned artifacts on changes (spinnaker#2278) * fix(jobs): Fix bottleneck around JobLocalExecutor.startJob (spinnaker#2280) The `startJob` method submits your request to a threadpool, which it immediately starts polling. This isn't useful, and limits the number of jobs that can be started to the number of threads supported by your scheduler. As a result, submitting a lot of jobs at once causes (needlessly) a lot of these jobs to timeout. There is still a `sleep(500)` which doesn't seem to add anything in the function, but didn't feel confident removing since other spots in clouddriver make use of it. * fix(provider/kubernetes): v2 Set default namespace dynamically (spinnaker#2279) Set the default namespace according to the `kubectl` rules defined in https://github.com/kubernetes/kubernetes/blob/bd4d511a40e142ee65edff3b286e57de502aa790/pkg/kubectl/cmd/util/factory_client_access.go#L127 In short, the default namespace is set in this way: - If `/var/run/secrets/kubernetes.io/serviceaccount/namespace` exists, assume we are running in a Kubernetes container, and set the default namespace to the contents of this file. - Set the default namespace to the namespace of current `kubectl` config context, if it exists. `kubectl config view` is used to get this namespace, which takes care of the complicated merges and overrides. - Otherwise, set the default namespace to "default". This fixes the issue of manifests without a namespace specification being deployed in the namespace of clouddriver when Spinnaker has been deployed in Kubernetes. Previously manifests without a namespace were assumed to belong to the "default" namespace, but clouddriver would deploy them in the "spinnaker" namespace, where it is deployed by default by Halyard. Deploying such manifests would cause Spinnaker to get stuck waiting for a deployment to show up in the "default" namespace, since clouddriver would instead have created a deployment in the "spinnaker" namespace. * Added EcsCloudMetricAlarmCaching - Agent/Client/Tests. (spinnaker#2263) * Added TaskHealthCaching - Agent/Client/Tests. (spinnaker#2260) * feat(provider/ecs): ECS Loadbalancer cache client and tests. (spinnaker#2264) * Added EcsLoadbalancerCacheClient with a cache model and test. * Object mapper configuration moved to declaration line. * Clean up. * feat(provider/ecs): ECS Target Group cache client and tests. (spinnaker#2265) * Added EcsTargetGroupCacheClient with a model and tests. * Changes based on feedback. * feat(artifacts): add artifact download endpoint (spinnaker#2282) * refactor(provider/kubernetes): remove obsolete source check (spinnaker#2283) * fix(core): Allow for versioned providers in lb controller (spinnaker#2284) * perf(provider/gce): De-dupe getHealth() calls in LB caching agents. (spinnaker#2281) * Added back missing methods. * Added back missing files, or removed files that didnt get removed during the merge.
dkirillov
added a commit
to lookout/clouddriver
that referenced
this pull request
Feb 7, 2018
* feat(provider/ecs): ECS Cache base classes (spinnaker#2065) * Prep classes for the caching agents to come. * Changes based on feedback. * fix(provider/gce):Enable RPS when only RATE or UTILIZATION (spinnaker#2083) * feat(provider/kubernetes): v2 cache namespaces & relationship (spinnaker#2087) * feat(provider/kubernetes): V2 enable minimal clusters (spinnaker#2090) * feat(provider/aws): Enable optional AWS Shield protection on ELB & ALBs (spinnaker#2089) * feat(provider/amazon): block device config for c5 instance types (spinnaker#2092) * fix(provider/gce): Ensure referenced instances exist during L4 delete. (spinnaker#2094) * feat(provider/kubernetes): cache network policies (spinnaker#2095) * feat(provider/kubernetes): v2 server group manager (spinnaker#2093) * feat(provider/amazon): Optionally tag server groups w/ app/stack/details (spinnaker#2096) Set `aws.defaults.addAppStackDetailTags: true` to enable. When enabled, all created server groups will be tagged with: - `spinnaker:application` - `spinnaker:stack` - `spinnaker:details` The tag will only be created if the value is non-null. By default, these tags will be propagated to launched instances. * fix(provider/kubernetes): fix v2 instance lookup (spinnaker#2098) * feat(provider/ecs): ECS Cluster caching classes and tests. (spinnaker#2091) * ECS Cluster caching classes and tests. * Base abstraction. * feat(provider/kuberentes): v2 security group provider (spinnaker#2097) * feat(provider/kubernetes): v2 pod logs (spinnaker#2099) * fix(provider/kubernetes): security group yaml (spinnaker#2100) add yaml field to KubernetesV1SecurityGroup so that the "Show YAML" modal works in deck. * feat(kubernetes/artifacts): Support deploying images dynamically (spinnaker#2102) * fix(provider/kubernetes): docker is not a hard-dependency (spinnaker#2106) * fix(provider/amazon): Do not delete listeners after updating them. Pretty bad idea. (spinnaker#2105) * feat(provider/kubernetes): split out k8s providers (spinnaker#2107) Needed to separate the search endpoints for each provider * feat(provider/kubernetes): v2 search endpoint (spinnaker#2108) * feat(provider/kubernetes): Allow op to override versioning (spinnaker#2110) * feat(provider/kubernetes): Add kubectl to Dockerfile. (spinnaker#2104) * fix(provider/gce): Handle partially-formed L7 LB resources properly. (spinnaker#2111) * fix(provider/kubernetes): fix application key parsing (spinnaker#2109) * perf(provider/gce): s/get/list/ in ILB caching agent. (spinnaker#2112) A few small fixes/guards to the L7 caching agent as well. * feat(provider/kubernetes): remove dependency on api-client (spinnaker#2113) * feat(provider/kubernetes): reenable kube monitoring (spinnaker#2115) * feat(core/search): Add parameter for fetching a batch of servergroups by name (spinnaker#2114) * fix(provider/gce): Fix TCP LB disabled state calculation. (spinnaker#2116) * perf(provider/gce): s/get/list/ in TCP/SSL LB caching agents. (spinnaker#2117) * feat(provider/kubernetes): v2 rollback (undo rollout) (spinnaker#2118) * feat(provider/kubernetes): scale manifest (spinnaker#2119) * feat(provider/kubernetes): k8s only search provider (spinnaker#2121) * fix(docker): Use wget instead of curl (spinnaker#2122) * fix(provider/kubernetes): read namespace as region (spinnaker#2123) * feat(provider/kuberentes): pause rollout (spinnaker#2120) * feat(provider/kuberentes): v2 non-enum k8s kinds (spinnaker#2125) * feat(provider/kubernetes): search result hydrators (spinnaker#2124) * feat(provider/kubernetes): resume rollout (spinnaker#2126) * fix(provider/kubernetes): v2 fix cred validation (spinnaker#2129) * fix(perf): Avoid manually copying bytes between streams (spinnaker#2127) * fix(provider/kubernetes): autowire correct objectmapper (spinnaker#2133) * fix(provider/kubernetes): attach correct op decorators (spinnaker#2130) * fix(provider/kubernetes): switch access modifiers for KubernetesKind constructors (spinnaker#2128) * feat(provider/kubernetes): runJob multi-container (spinnaker#2135) support multiple containers in the run job description. we have to support both `container` and `containers` until we can deprecate the `container` field since the current defs are all using `container`. * feat(entitytags): Support indexing / searching by application (spinnaker#2132) If `entityRef.application` is not explicitly provided, it will be parsed from `entityRef.entityId` using frigga conventions. Searching by application can be accomplished by `/tags?application=my_application` * fix(provider/gce): Tolerate named port values as Doubles. (spinnaker#2137) Pipelines are being persisted with 'port' values of NamedPorts as Doubles. This is an issue. This change tolerates that behavior in deploys since users will have pipelines with Double ports in the wild. * fix(provider/dcos): Use createApp instead of updateApp for deploy (spinnaker#2136) * fix(provider/dcos): Set container type appropriately. * fix(provider/dcos): use createApp instead of updateApp for deploy * fix(provider/dcos): Remove dcos client proxy exception wrapping * Added tests for Keys. Changed separator. (spinnaker#2101) * fix(provider/kubernetes): fix security group loading by name (spinnaker#2134) * fix(kubectl): --to-revision (spinnaker#2140) * feat(provider/kubernetes): allow OAuth token authentication (spinnaker#2147) * refactor(provider/openstack): Updated openstack4j to 3.1.0 and removed components that are available upstream (spinnaker#2103) * fix(elasticsearch): Force refresh when fetching entity tags from Front50 (spinnaker#2149) `elasticsearch_index_template.json` is the index template that should be applied in elastic search. * fix(provider/aws): Fix updating ALB listeners (spinnaker#2148) * feat(provider/google): Label instances so they can easily be grouped later. (spinnaker#2142) * feat(provider/kubernetes): more robust status (spinnaker#2152) * feat(provider/kubernetes): default to frigga (spinnaker#2156) * feat(provider/kubernetes): v2 configmap support (spinnaker#2157) * feat(provider/kuberentes): send deployed artifact to orca (spinnaker#2158) * feat(provider/kubernetes): v2 support secrets (spinnaker#2159) * feat(provider/kubernetes): register docker image replacers (spinnaker#2160) * feat(provider/kubernetes): v1 pod logs (spinnaker#2162) support pod logs for v1 provider. going to be useful when working on run job logs as well. * feat(provider/kubernetes): v2 support daemonset (spinnaker#2161) * fix(provider/gce): Tolerate failed backend service getHealth() calls. (spinnaker#2153) * fix(provider/gce): Break out labels. (spinnaker#2163) * fix(provider/aws): Should use edda to lookup target groups by name (spinnaker#2165) * ECS Container Instance caching classes and tests. (spinnaker#2143) * fix(provider/aws): Remove invalid `spinnaker:` tags (spinnaker#2168) Remove any `spinnaker:*` tags that are no longer valid (ie. cloning across stacks and `spinnaker:stack` no longer points at the correct stack). This is necessary as `deck` will _now_ supply existing tags when cloning. * feat(provider/kubernetes): runJob logs (spinnaker#2164) add support for capturing logs as part of `collectJob`. logs are added to the job status. i'm not sure how well this scales as log size grows so this may not be the best implementation, but it may work for the short term. since execution details are persisted to volitile storage storing them with the jobs status is best since the reference to the job pod is also stored there. * refactor(cf): Stop building CF provider until maintainer is found (spinnaker#1942) BREAKING CHANGE: Stops support of CF until maintainer is found * bug(provider/openstack) - LoadBalancer security groups are optional. (spinnaker#2154) * chore(dependencies): Bump to 0.123.0 (spinnaker#2169) * feat(provider/kubernetes): load artifacts alongside manifest (spinnaker#2166) * refactor(provider/kubernetes): allow api version subclassing (spinnaker#2167) * chore(jobs): turn down log-level (spinnaker#2171) * fix(web): Do not throw exception when sg not found (spinnaker#2172) * feat(provider/ecs): ECS Task Definition caching classes and tests. (spinnaker#2145) * ECS Task Definition caching classes and tests. * ObjectMapper is now injected. * refactor(entitytags): Support for alerts/notices on any entity type (spinnaker#2175) * feat(cats): Support for selectively enabling agents (spinnaker#2177) This is an assist to local development when you do not want _all_ agents to run by default. ``` redis: agent: enabledPattern: .*reservation.* ``` By default `enabledPattern` is `.*` and all agents will be scheduled. * Use setApiKey when using token from service account (spinnaker#2178) * fix(provider/kubernetes): v1 add docker registries in all cases (spinnaker#2176) * bug(provider/openstack) - Display VIP for LBs that do not have Floating IP instead of null.. (spinnaker#2170) * fix(provider/kubernetes): don't throw on invalid lb op (spinnaker#2184) * fix(provider/aws): Reservation caching agent, now with less RxJava (spinnaker#2180) * feat(provider/kubernetes): replace deployed configmap volumes (spinnaker#2183) * feat(xenial_debians): Add systemd service configuration. (spinnaker#2182) * feat(xenial_debians): Add systemd service configuration. * feat(xenial_builds): Fix typo. * feat(provider/ecs): ECS Service caching classes and tests. (spinnaker#2144) * ECS Service caching classes and tests. * Objectmapper is now injected. * feat(provider/aws): Support expected capacity constraint when resizing (spinnaker#2179) Support a constraint wherein the server groups' current capacity must match expected prior to a resize. There are scenarios wherein a user initiates a resize via the UI at the same time that an autoscaling event is happening behind the scenes. In such a scenario, the resize via UI != autoscaling and some churn will occur. `deck` will be changed to supply the capacity constraint. * fix(provider/aws): Retry createAutoScalingGroup() / updateAutoScalingGroup() (spinnaker#2185) * feat(provider/kubernetes): cache controller revisions (spinnaker#2138) * fix(provider/kubernetes): fix registry init (spinnaker#2188) if using the `namespaces` key for kubernetes accounts, reconfigure registries using those namespaces. * feat(provider/kubernetes): v2 support labels on delete & scale (spinnaker#2190) * Xenial builds (spinnaker#2189) * feat(xenial_debians): Add systemd service configuration. * fix(provider/kubernetes): fix onDemand cache update in controller agent. (spinnaker#2131) * chore(dependencies): update to latest spinnaker-dependencies Also updates gradle to 3.5 * fix(provider/kubernetes): fix registry init 2 (spinnaker#2191) the previous fix caused sever performance issues. this fix will only configure hard coded registries once, only reconfiguring failed registries on the next loop, if any. * fix(google): Retry on all 5xx errors from the platform. (spinnaker#2193) * fix(core): fix bean mismatch due to Spring update removes unnecessary corsfilter bean - clouddriver should not be hit by browser agents * chore(warnings): fix compiler warnings * feat(provider/ecs): ECS Task caching classes and tests. (spinnaker#2146) * ECS Task caching classes and tests. * ObjectMapper is now injected. * fix(cats): fix conditional instantiation of EurekaNodeStatusProvider * fix(build): fix spring-boot plugin usage * refactor(kubernetes): change OAuth token configuration (spinnaker#2196) * fix(provider/aws): s/contraints/constraints (spinnaker#2198) * fix(provider/kubernetes): fix envvar source (spinnaker#2199) the bump in client library broke the API for configMap and secret envvar sources. adds an extra property for optional. defaults to true to preserve backwards compatibility. shoutout major version bumps. * chore(logging): remove extra logging (spinnaker#2200) * feat(provider/gce): Support for RMIG zone selection. (spinnaker#2201) * chore(systemd_logs): Cleanup unneeded logging in systemd config. (spinnaker#2202) * feat(provider/ecs): ECS IAM Role caching classes and tests. (spinnaker#2150) * ECS IAM Role caching classes and tests. * Moved Role and the associated classes into clouddriver-aws. * AWS' default region is being used. Cleanup. * refactor(clouddriver-core) Fixed to use const value (spinnaker#2186) * fix(provider/kubernetes): registry init fix (spinnaker#2205) for real this time. the previous fix caused registry namespaces to get set to an empty list the second time the initialization occurred. moving it down into this method allows us to still use the full list of namespaces (`allNamespaces`) and setup the affected ones to be initialized. This would occur if a new namespace was added, for instance. * fix(amazon/loadBalancers): Be smarter about updating ALB listeners (spinnaker#2203) * fix(provider/dcos) Catch the correct exception for the proxy. (spinnaker#2207) * chore(dependencies): update to 0.128.0 (spinnaker#2208) * fix(core/search): Fix search query which matches non-conforming cache keys (spinnaker#2209) Certain cache keys were failing due to Keys.parse() returning null. Turns out there is no need to parse the cache key and then reconstruct it. * feat(provider/kubernetes): support envFrom (spinnaker#2213) adds support for envFrom for the v1 provider. envFrom can be either a secret or config map reference and will be injected into the pod as environment variables. * fix(provider/gce): Tolerate null instance template lists. (spinnaker#2216) * feat(aws/loadBalancer): Support on demand caching for target groups (spinnaker#2214) * refactor(provider/kubernetes): refactor delete pods operation (spinnaker#2210) * feat(artifacts): sets up artifact module (spinnaker#2218) My plan was to reuse the artifact work in appengine, but it would have been too messy to refactor. It didn't expose any artifact-centric interface or allow easy integration with other artifact sources without a fair bit of extra work, so I rewrote in a separate module. T he appengine GCS integration was left unchanged (with the small change of moving the storage utils into a 'AritifactUtils' class). * chore(dependencies): Bump spinnaker-dependencies version. (spinnaker#2220) Picks up the new compute library versions. * fix(provider/amazon): render IPv6 ingress security group rules (spinnaker#2212) * feat(provider/appengine): allow gcloud version to be configurable (spinnaker#2217) * feat(provider/kubernetes): init containers (spinnaker#2219) adds support for initContainer property of the Pod spec * fix(provider/kubernetes): s/name/manifestName (spinnaker#2222) * fix(provider/kubernetes):Server Group is disable in Daemonset when replic set to 0 (spinnaker#2206) * feat(provider/kubernetes): deploy from artifact (spinnaker#2223) * fix(provider/appengine): fix monikers for app engine server groups (spinnaker#2224) * fix(provider/amazon): correctly propagate ipv6 ingress security group rules on upsert (spinnaker#2225) * fix(provider/aws): block device configs for h1.* and i3.* (spinnaker#2227) * fix(registry): Handle situations where tag creation date cannot be fetched (spinnaker#2228) * fix(provider/amazon): ignore description when upserting security group rules (spinnaker#2229) * bug(provider/openstack) - filter images by region if provided. (spinnaker#2230) * fix(entitytags): Retry support when fetching entity tags from Front50 (spinnaker#2232) * fix(core): Create the 'RetrySupport' bean (spinnaker#2233) * feat(provider/kubernetes): return modified manifests (spinnaker#2234) * fix(dockerRegistry): handle 400 for token refresh (spinnaker#2215) By the spec https://docs.docker.com/registry/spec/api/#base, 401 should be used to indicate an expired token, but it seems 400 is used as well. * feat(provider/kubernetes): ds & ss stability conditions (spinnaker#2238) * fix(appengine): remove batch calls, revert client library (spinnaker#2221) This reverts commit 8db9514. * feat(provider/kubernetes): return deployed manifests (spinnaker#2235) * feat(provider/kubernetes): v2 make rollback dynamic (spinnaker#2237) * fix(provider/kubernetes): disable v1 controller caching (spinnaker#2239) This had a chance of deleting k8s resources from the cache when used with daemonsets & statefulsets, since there is no type information provided with a resource name. * chore(dependencies): bump spinnaker-dependencies to 0.131.0 (spinnaker#2242) - newer aws sdk * feat(artifacts): Download artifacts from GitHub (spinnaker#2231) * feat(provider/gcp): Add ability for default app creds to impersonate … (spinnaker#2240) * feat(provider/gcp): Add ability for default app creds to impersonate a service account * duftler comments * fix(provider/aws): Avoid being marked unhealthy after initialization (spinnaker#2245) Avoid being marked unhealthy should there be a temporary blip in connectivity to one or more accounts. This frequently occurs as a result of an AWS rate limit being hit. The `health.amazon.errors` metric can be used to alert on possible errors that would have previously caused an instance to go unhealthy. * feat(provider/appengine): enable flex deployments (spinnaker#2241) Allow App Engine deployments with a built container image and app.yaml. * fix(startup): make constructor bean optional (spinnaker#2247) * fix(clusters): Return all matching server groups for a provider (spinnaker#2249) * fix(provider/kubernetes): v2 Fix kubectl label selector flag (spinnaker#2252) Currently we set `-l=` when `labelSelectors` is null or is not empty, causing all commands that pass a null value for `labelSelectors` to fail. The `-l=somelabel` flag should be set if `labelSelectors` is not null and is not empty. * fix(provider/aws): handle spotPrice == in rollingpush (spinnaker#2251) * debug(provider/aws): Avoid thread pool when building reservation report (spinnaker#2253) Attempting to track down an issue wherein the reservation report caching agents _stop_ and are never rescheduled after some period of time. Will revisit and remove the thread pool configuration if this change happens to improve the stop/reschedule situation. * debug(provider/aws): Additional logging for `ReservationReportCachingAgent` (spinnaker#2254) * fix(provider/aws): Specify `connectionRequestTimeout` for edda (spinnaker#2255) Establishes an upper bounds on the amount of time to spend waiting on an available connection from the `edda` connection pool. Previously, a caching agent could wait _indefinitely_ for an available connection. The default is now 10000 (milliseconds) and can be overridden with `aws.edda.connectionRequestTimeout`. * fix(provider/amazon): do not try to create reserved tags (spinnaker#2256) * fix(provider/aws): Ensure that `HttpEntity` is closed on edda failure (spinnaker#2258) Also log an error when edda retries have been exhausted. * bug(provider/openstack) - allow for binding to multiple loadbalancers with the same port mappings. (spinnaker#2257) * fix(provider/aws): block device config for missing instance types (spinnaker#2266) This should provide coverage for all currently available instance types. Will look for better ways to manage this moving forward. * fix(provider/kubernetes): allow empty serviceName in statefulset (spinnaker#2261) * chore(provider/kubernetes): log malformed cache entries (spinnaker#2262) * feat(provider/kubernetes): v2 ignore caching flag (spinnaker#2268) * feat(provider/kubernetes): return resource create time (spinnaker#2269) * feat(provider/kubernetes): fail deploy if artifacts don't bind (spinnaker#2271) * feat(authz): requiredGroupMemberships to permissions migration (spinnaker#2272) update AllowedAccountsValidator to use permissions config in addition to requiredGroupMemberships * fix(provider/kubernetes): Replace artifact only if target found (spinnaker#2259) Do not attempt artifact replacement if no target was found. Reading a non-existing path can return an empty ArrayNode object, in which case we should not attempt any replacement. Add checks for bound artifacts in artifact replacement tests. * feat(provider/kubernetes): split out required & optional artifacts (spinnaker#2273) * fix(provider/docker): update bearer_token to access_token (spinnaker#2274) * fix(provider/kubernetes): Fixed statefulset and daemonset volumesource bugs. (spinnaker#2276) feat(provider/kubernetes): split out required & optional artifacts (spinnaker#2273) fix(provider/docker): update bearer_token to access_token (spinnaker#2274) * fix(provider/kubernetes): annotate artifacts with reference (spinnaker#2277) * feat(provider/kubernetes): only redeploy versioned artifacts on changes (spinnaker#2278) * fix(jobs): Fix bottleneck around JobLocalExecutor.startJob (spinnaker#2280) The `startJob` method submits your request to a threadpool, which it immediately starts polling. This isn't useful, and limits the number of jobs that can be started to the number of threads supported by your scheduler. As a result, submitting a lot of jobs at once causes (needlessly) a lot of these jobs to timeout. There is still a `sleep(500)` which doesn't seem to add anything in the function, but didn't feel confident removing since other spots in clouddriver make use of it. * fix(provider/kubernetes): v2 Set default namespace dynamically (spinnaker#2279) Set the default namespace according to the `kubectl` rules defined in https://github.com/kubernetes/kubernetes/blob/bd4d511a40e142ee65edff3b286e57de502aa790/pkg/kubectl/cmd/util/factory_client_access.go#L127 In short, the default namespace is set in this way: - If `/var/run/secrets/kubernetes.io/serviceaccount/namespace` exists, assume we are running in a Kubernetes container, and set the default namespace to the contents of this file. - Set the default namespace to the namespace of current `kubectl` config context, if it exists. `kubectl config view` is used to get this namespace, which takes care of the complicated merges and overrides. - Otherwise, set the default namespace to "default". This fixes the issue of manifests without a namespace specification being deployed in the namespace of clouddriver when Spinnaker has been deployed in Kubernetes. Previously manifests without a namespace were assumed to belong to the "default" namespace, but clouddriver would deploy them in the "spinnaker" namespace, where it is deployed by default by Halyard. Deploying such manifests would cause Spinnaker to get stuck waiting for a deployment to show up in the "default" namespace, since clouddriver would instead have created a deployment in the "spinnaker" namespace. * Added EcsCloudMetricAlarmCaching - Agent/Client/Tests. (spinnaker#2263) * Added TaskHealthCaching - Agent/Client/Tests. (spinnaker#2260) * feat(provider/ecs): ECS Loadbalancer cache client and tests. (spinnaker#2264) * Added EcsLoadbalancerCacheClient with a cache model and test. * Object mapper configuration moved to declaration line. * Clean up. * feat(provider/ecs): ECS Target Group cache client and tests. (spinnaker#2265) * Added EcsTargetGroupCacheClient with a model and tests. * Changes based on feedback. * feat(artifacts): add artifact download endpoint (spinnaker#2282) * refactor(provider/kubernetes): remove obsolete source check (spinnaker#2283) * fix(core): Allow for versioned providers in lb controller (spinnaker#2284) * perf(provider/gce): De-dupe getHealth() calls in LB caching agents. (spinnaker#2281) * fix(provider/kubernetes): load replicas for non replicaset server groups (spinnaker#2287) * feat(provider/kubernetes): look up namer by class name (spinnaker#2288) * fix(core): Support an empty NamerRegistry (spinnaker#2297) * poc(provider/aws): Index a subset of server group / instance details (spinnaker#2204) Mo bettah search! Specifically this PR introduces a brand new agent that fetches all server groups and instances in each aws account / region. The current breakdown of caching agents (per account and per region) are potentially too granular to create useful elastic search indexes off of. * fix(provider/kubernetes): prevent NPE if resource does not have creation timestamp (spinnaker#2299) * fix(provider/kubernetes): fix cast from double in replica count (spinnaker#2298) * fix(provider/aws): Handle ApplicationLoadBalancers with invalid actions (spinnaker#2300) * feat(provider/kubernetes): allow artifact types to be extended (spinnaker#2302) * fix(provider/kubernetes): v2 cache clusters for workloads only (spinnaker#2303) This ensures that only workload based resources are associated with a spinnaker cluster, reducing 'application' screen clutter * feat(provider/openstack): Add support for availability zones (spinnaker#2304) * fix(provider/kubernetes): only include app & cluster frigga details (spinnaker#2306) * feat(provider/ecs): Ecs instance cache client (spinnaker#2289) * Added EC2 Instance cache client. * Removed unneeded import. * Removed @qualifier from cache client constructors, and updated existing ones. * feat(provider/ecs): ECS Scalable Targets caching classes and tests (spinnaker#2290) * Added ScalableTarget cache - caching agent and client. * Removed @qualifier. * Added EcsClusterProvider. (spinnaker#2291) * Added EcsRoleProvider. (spinnaker#2292) * Added EcsApplicationProvider, EcsApplication model, and TestCredential.. (spinnaker#2293) * Added EcsCloudMetricProvider and EcsCloudMetricController. (spinnaker#2294) * poc(provider/aws): Index a subset of instance details (spinnaker#2204) (spinnaker#2308) Support searching for instances by ip address (and a handful of other attributes). Specifically this PR introduces a brand new agent that fetches all instances in each aws account / region. * feat(provider/kubernetes) - adds configMap replacer support for replicasets (spinnaker#2310) * fix(provider/kubernetes): v2 Make workloads authoritative for apps (spinnaker#2307) Let workload deployments have full control over (generated) applications coming and going. This is in line with spinnaker#2303 and fixes spinnaker/spinnaker#2259 In particular, applications will not be shown in the deck "Applications" view if the replicaset/daemonset/deployment/statefulset of the application has the `caching.spinnaker.io/ignore="true"` annotation, and will be removed from the list if the annotation added to an existing workload. * refactor(provider/appengine): artifact logic from orca (spinnaker#2305) Moves responsibility for artifact hydration during appengine deploys from orca into clouddriver. * fix(provider/kubernetes) : disable v1 controller caching spinnaker#2239 (spinnaker#2301) * fix(provider/kubernetes) : disable v1 controller caching spinnaker#2239 * fix(provider/kubernetes) : disable v1 controller caching spinnaker#2239) * feat(provider/kubernetes): v1 volume mount subpath support (spinnaker#2316) * fix(provider/kubernetes): v1 fix red/black for svgs without lbs (spinnaker#2317) * feat(provider/kubernetes): default artifact replacers (spinnaker#2318) * fix(artifacts): improve github artifact downloader error handling (spinnaker#2319) * fix(provider/gce): Fix typo in metric tag. (spinnaker#2320) * fix(provider/kubernetes): v1 eventual consistency bug in disable (spinnaker#2321) It turns out it was possible that we would 1. apply an annotation 2. toggle replica set labels 3. fail to wait for the label change to be applied Part 3. could happen because the generation of the replica set returned from the "toggle label" operation was that of the "apply annotation" operation. The "wait for consistency" step we usually do here was then skipped causing us to edit pod labels before the replica set owning them was finised updating. This doesn't happen often, but enough to be a real problem. * fix(gcs): Prevent NPE when gcs artifact account has no credentials path (spinnaker#2324) * Corrected capitalization for cloudMetrics endpoint. (spinnaker#2312) * feat(provider/ecs): ECR Image Provider (spinnaker#2309) * Added EcrImageProvider. * Moved EcrImageProvider to the correct package. * Made a filtering method. * Error switched to IllegalArgumentException. Grammar fixed. * Updated tests. * feat(provider/ecs): ECS Cluster Controller (spinnaker#2313) * Added EcsClusterController. * Camel-cased the endpoint. * fix(provider/amazon): propagate serverGroupNamesByRegion result on copyLastAsg (spinnaker#2270) * Added ContainerInformationService. (spinnaker#2314) * fix(provider/ecs): ECS Caching Agents - Account/Region awareness (spinnaker#2315) * Caching agents are now account and region aware. * getCluster account aware. * Formatting fixed. * EcsCredentialsInitializer now initializes NetflixAssumeRoleEcsCredentials. (spinnaker#2323) * fix(rrb): Disable percentage calculation should use Math.ceil() (spinnaker#2327) This aligns the calculation with what is also happening within `orca`. * fix(provider/kubernetes): v1 oom killed job has exit code 0 (spinnaker#2328) * Added EcsImagesController. (spinnaker#2329) * feat(provider/ecs): ECS Instance Provider (spinnaker#2330) * Added EcsInstanceProvider. Updated ContainerInstance and the caching client to include the availability zone. * Corrected variable names. * Added EcsCloudMetricService. (spinnaker#2331) * Added EcsServerClusterProvider. (spinnaker#2332) * feature(provider/openstack) implementing senlin zone policy and scheduler hints (spinnaker#2325) * feat(credentials): add expand parameter to /accounts endpoint (spinnaker#2326) * fix(entitytags): Increase default 'maxResults' from 100 -> 2000 (spinnaker#2335) * feat(provider/ecs): ECS Load Balancer Provider (spinnaker#2333) * Added EcsLoadBalancerProvider. * Feedback changes. * Changed x's to it's. * feat(provider/ecs): Destroy Service Atomic Operation (spinnaker#2334) * Added DestroyServiceAtomicOperation along with base deploy classes. * Removed invalid import. Formatting fixed. * Service changed to server group in the output messages. * Copyright year updated. * feat(provider/kubernetes): make kubectl configurable per-account (spinnaker#2337) * fix(provider/appengine): dont prepend "gs" to all repository URLs (spinnaker#2346) * fix(provider/kubernetes): v2 fix caching of malformed annotations (spinnaker#2340) * fix(provider/kubernetes): v2 gracefully handle unknown kind/version (spinnaker#2342) * fix(provider/kubernetes): v2 reduce noise when polling older clusters (spinnaker#2344) * feat(provider/kubernetes): v2 hpa support (spinnaker#2347) * Added TerminateInstancesAtomicOperation. (spinnaker#2339) * Added CreateServerGroup AO. (spinnaker#2338) * fix(*): Force using hashtags for dynomite connections (spinnaker#2348) * Added ResizeServiceAtomicOperation. (spinnaker#2341) * feat(provider/ecs): Enable Service Atomic Operation (spinnaker#2343) * Added EnableServiceAtomicOperation. * Deleted an unnecessary description. * feat(provider/ecs): Disable Service Atomic Operation (spinnaker#2345) * Added DisableServiceAtomicOperation. * Deleted an unnecessary validator. * refactor(provider/ecs): Caching agents enabled, code cleanup. (spinnaker#2349) * EcsProviderConfig now adds all of the caching agents. `expand` parameter being used properly in `EcsApplicationProvider`. Added missing copyrights. Added missing @component annotation. Minor syntax changes. * Lombok dependency added in clouddriver-ecs.gradle. * bug(provider/openstack) - cleanly handle errors when caching OST server groups. (spinnaker#2350) * fix(provider/kubernetes): concurrent modification of kind list (spinnaker#2353) Seems the iterator wasn't closed * fix(provider/kubernetes): Add top level `kind` to make Deck happy (spinnaker#2355) * feat(provider/kubernetes): v2 Extract artifact name from reference (spinnaker#2322) Allow for for using a regex to set the artifact name after finding artifacts in a manifest. This is used to add support in orca for spinnaker/spinnaker#2267 Currently only done for Docker images, but should be possible for configmaps and secrets as well. * Missing files. * fix(provider/appengine): surface timeouts to frontend during job execution (spinnaker#2356) * Added missing file back. Deleted a duplicate method.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Avoid being marked unhealthy should there be a temporary blip
in connectivity to one or more accounts.
This frequently occurs as a result of an AWS rate limit being hit.
The
health.amazon.errorsmetric can be used to alert on possibleerrors that would have previously caused an instance to go unhealthy.