Skip to content

@vito vito released this Jun 11, 2019

🔗 feature

  • @rliebz added the ability for put steps to git resources with the parameter merge: true to be further configured with the parameter returning: unmerged, which will ensure that concurrent changes to the same remote branch do not affect downstream jobs concourse/git-resource#262.

🔗 feature

🔗 feature

  • @stigtermichiel tuned our build process so that the size of the compressed frontend code is reduced by a factor of 3 - page loads should be accordingly faster #3915.

🔗 feature

  • If you want your web node to accept encrypted connections over HTTPS, you now have the option of connecting to an ACME server to automatically retrieve a certificate from an ACME server of your choice. By default it will reach the free public ACME service provided by Let's Encrypt.

    Thanks to @henderjm for the PR #3878!

🔗 feature

  • @agurney improved the info api endpoint (/api/v1/info) to return both the external URL and cluster name #3862.

    Hopefully this will enable future improvements to fly and the web UI!

🔗 feature

🔗 feature

  • When you pin a resource through the web UI, your name and current local time will now be automatically added as the pin comment #3743.

🔗 feature

  • The 'password' field on the login page now has an HTML property that will be ignored by most browsers, but helps Concourse pass some automated security scans #3839.

🔗 feature

  • @tvon gave the highlighted lines on the build page a slightly darker color so they really pop #3866.

🔗 fix

🔗 fix

🔗 fix

  • Addressed a bug where in rare circumstances, fly intercept -c would find multiple check containers for the same resource #3983.

🔗 fix

  • In service of a refactor, task caches are now decoupled from the worker(s) they are stored on. As a result, you may find that in multi-worker deployments, pipelines making use of task caches may consume duplicate storage across workers #3830, #3965.

🔗 fix

  • Corrected a surprising behaviour where, when a pipeline was paused for an extended period of time, many of its resources' versions would disappear #3963.

🔗 fix

  • There was a long-standing bug on the dashboard, where the existence of circular pipelines would cause the browser to crash. This was fixed in v5.2.0, but the fix caused a performance regression, which is now also fixed #3870, #3901.

🔗 fix

  • Fixed a bug, introduced in v5.1.0, where, on a build step with attempts: specified, only one attempt would be possible to view on the build page #3898.

🔗 fix

  • @gaelL Fixed a bug, which had existed since at least v4.2.3, where operators were prevented from retiring team-scoped workers with the concourse retire-worker command #3929.

🔗 fix

  • A bug on the reaped build logs screen was fixed, where a link was pointing to deprecated documentation #3931.

    This screen is visible when you visit a build that is older than its job's configured retention policy.

🔗 fix

  • Fixed a problem with one of the database migrations introduced in v5.0.0 which could cause the web node startup to fail in some cases. This won't affect users who have kept up to date until now, but if this is the first version of Concourse you are deploying since before v5.0.0, the upgrade should be a little more resilient #3996.

🔗 fix

  • @mockersf fixed a problem where timestamps weren't being properly returned when a get step or a put step finished. This caused some wacky durations to be displayed on build pages #3871.

🔗 fix

  • Fixed an issue with our build process that caused fly to stop working for some folks making use of a VPN with split DNS.

🔗 fix

  • Fixed a bug where the dashboard preview of a paused job with a pending build would show the color of the most recent build's status, when it should be blue #3718.
Assets 14
May 30, 2019
ci: wait for deployment rollout in k8s-deployment (#3947)
Previously, it could happen that `web` didn't have its deployment fully
complete, which used to make `port-forward` fail quite consistently.

By leveraging `rollout status` we can wait for the underlying pods to
get to a `READY` state (instead of plain `RUN`) before moving forward.

Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
May 30, 2019
Merge pull request #3943 from concourse/fix-helm-env-check
Only check for env variables in the templates dir

@vito vito released this May 16, 2019

🔗 feature, breaking

  • @ralekseenkov has implemented generic credential caching for all credential managers!

    This replaces the Vault-only caching functionality. To transition, you'll need to update the following flags.

    • --vault-cache is now --secret-cache-enabled

    • --vault-max-lease is now --secret-cache-duration

    As part of this change, credential managers now implement a simpler interface that will make it easier to look-up secrets in multiple paths.

    For more information, see Credential Management.

    To follow along with future planned improvements to credential management, check out concourse/rfcs#21.

🔗 fix, breaking

  • @stigtermichiel changed the short-flag for fly builds --team from -t from -n to make it consistent across fly. Consistency is key.

🔗 feature

🔗 feature

  • The web node can now be configured to enable audit logs, thanks to a PR by @loghen41!

    Auditing currently logs API calls to the default logger using flags to enable specific auditing groups.

🔗 feature

  • Like a phoenix from the ashes, the pipeline navigation sidebar has made its triumphant return. It was initially removed to focus our efforts on the dashboard as a navigation flow. We have concluded that one click is better than two.

    Expect more design/UX polish in future releases!

🔗 feature

  • @itsdalmo has introduced a new in_parallel step which can run steps in parallel with more control via additional config: limit which will limit the number of parallel steps, and fail_fast which will interrupt currently running steps and prevent scheduling pending steps.

    This sounds a lot like the aggregate step, only better in every way (e.g. it doesn't have a stupid name), so fly set-pipeline will now issue deprecation warnings for aggregate: usage.

🔗 feature

  • Added a tooltip to the pause toggle on the dashboard page and the pipeline page explaining why it might be disabled.

🔗 feature

  • @hprotzek added the ability to retain build logs for a specific time duration and/or build count. See build_log_retention for more details.

🔗 security

  • We have restricted the SSH MAC algorithms used by the web node to a more secure set, overriding the Go defaults which allow weaker algorithms.

🔗 feature

  • Concourse is now compatible with Credhub v2.x (except for 2.1 due to a bug)! CredHub v1.9.x is still supported, too.

🔗 feature

  • Added ability set a name for the Concourse cluster which will be displayed on the dashboard page by setting cluster-name flag.

🔗 feature

  • @cappyzawa added a new get-team subcommand to fly. It allows you to retrieve a single team's config.

🔗 feature

  • @rkoster added a new flag --external-garden-url to allow use of a separately-managed Garden server as a worker.

🔗 feature

  • @pivotal-kahin-ng added a way of retaining the build history of a job when renaming it, by updating the job name and specifying its old name as old_name. After the pipeline has been configured, the old_name field can be removed.

🔗 fix

  • We reduced the default concurrency settings for volume sweeping from 5 to 3 as a way of reducing the stress that volume deletion ends up putting on the system in some cases.

🔗 fix

  • @edtan fixed a panic caused by running concourse web without a --session-signing-key.

🔗 fix

  • The Concourse API now returns 401 Unauthorized when an expired/invalid token is used to access an endpoint which supports authenticated/unauthenticated views.

    Previously it would just return a 200 response with less data, as if you weren't logged in, which made the behavior somewhat ambiguous and made auto-relogin logic difficult to implement consistently.

🔗 fix

  • Fixed a bug with Dex CloudFoundry connector when the user is a member of many teams. Thanks to @daniellavoie!

🔗 fix

  • Fixed a bug where the user gets a "You are not authorized to view the details of this pipeline" while watching a build.

🔗 fix

  • Fixed a bug where aborting a started build prior to a web node re-attaching to it would result in an orphaned, still running, uncompleted build.

    Along the way, the general 'aborting' flow has been refactored and should fix up any oddities caused by aborting builds at...inopportune moments.

🔗 fix

  • fly prune-worker --all-stalled has been fixed to only return a warning if no stalled workers are found, instead of an error.

🔗 fix

  • concourse quickstart has been fixed to ignore the --worker-tsa-worker-private-key flag.

🔗 fix

🔗 fix

  • Multiple groups in the same pipeline can no longer use the same name. An error is now raised if attempted.

🔗 fix

  • Fixed a bug where fly execute --input would hang indefinitely after uploading the input directory as a consequence of the web node stopping.
Assets 14
May 15, 2019
Merge pull request #3865 from concourse/fix-artifact-errors
atc: fix stack overflow in artifact errors

@vito vito released this Apr 16, 2019

🔗 fix, breaking

  • tl;dr: concourse web --peer-url and concourse web --tsa-peer-ip are gone in favor of concourse web --peer-address

    We have been doing a lot of internal refactoring and decoupling between various components. One side effect of this is that the web nodes no longer need to stream user artifacts to one another, which was the only reason the concourse web --peer-url flag was needed, so it has been removed.

    However, the SSH gateways (the "TSAs"), which also run on the web nodes, still need their address for the forwarded worker connections advertised to other web nodes. This value used to be inferred by --peer-url, so we've added a new --peer-address flag for it.

🔗 security, breaking

  • The web node now defaults X-Frame-Options to deny to safeguard against clicjacking attacks. If you run Concourse in an iframe, you'll notice that it doesn't work anymore.

    To configure X-Frame-Options otherwise, see Ingress.

🔗 feature

🔗 feature

  • Each step in the build log will now show how long it took to initialize and run when hovering over the icon to the right of the header. Thanks for the PR, @mockersf!

🔗 feature

  • Resources can now be annotated with icon to put pretty little icons in your pipeline and make different resource types easier to distinguish. This was also a PR by @mockersf - thanks a bunch!

🔗 feature, security

  • Resource metadata will no longer be shown by default in exposed pipelines.

    Metadata should never contain credentials or any criticial information, but for some use cases it is not desireable to show e.g. commit messages and authors even though the pipeline is public.

    The resource must now have the public value set in order to show metadata, just like jobs. One caveat is build output: if a job is public, any get step and put steps will still show their metadata.

🔗 feature

  • fly execute will now upload inputs and download outputs in parallel.

🔗 feature

  • The Concourse BOSH release now packages Ubuntu-flavored images for each core resource type instead of Alpine. This is primarily for compliance reasons. Nothing should really be affected.

    The .tgz distribution continues to use Alpine so the tarball doesn't get even bigger. Once we minimize the amount of resource types we package with Concourse (see (RF)RFC #23) we'll be removing them and standardizing on Ubuntu for simplicity's sake.

🔗 feature

  • Generic oAuth can now be configured with different user ID/name keys. They default to user_id and user_name, just as before.

🔗 feature

  • Generic OIDC auth can now be configured with a different user name key. It defaults to username, just as before.

🔗 feature

  • Previously, workers would garbage collect containers in volumes sequentially, destroying containers first and then volumes. This meant that if a worker had thousands of volumes to remove, it would go through and destroy them one by one - meanwhile, containers were not being garbage-collected.

    Containers and volumes are now garbage-collected in parallel to each other, with a default max-in-flight of 5 containers and 5 volumes at a time. This speeds up garbage-collection overall and prevents an imbalance in volume/container counts from slowing each other down. This is especially important as workers are typically capped at 250 containers, but may have thousands of volumes and may even have a slow disk.

🔗 feature

🔗 feature

  • Pipelines now have a play/pause button at the top bar, so you don't have to go all the way back to the dashboard and find the pause button there. Thanks for the PR, @robwhitby!

🔗 feature

  • URLs in resource metadata are now clickable, thanks to a PR by @Twiknight!

🔗 fix, security

  • Fixed a minor information leak that would allow unauthenticated users to fetch the step names and structure for a build whose job is not marked public.

    This only exposed step names, but it was still a little weird to allow it to be fetched. It will now return a 401 Unauthorized instead.

🔗 fix

  • Previously, manually-triggered builds would cause resource checking to be performed in the job scheduling loop. This ensured that manually triggered builds ran with the latest versions available, but it also slowed down scheduling for every other job in the pipeline, because they're all scheduled one-by-one.

    In the worst case, this meant a hanging resource check could result in all builds in the pipeline being stuck in the "pending" state for a long period of time (or, "like, forever").

    So we changed things around a bit: instead, the scheduler just won't start a manually triggered build until the "last checked" timestamp of each of its resources is after the build's "created at" timestamp. And to make that go faster, when a build is manually triggered we'll short-circuit the checking interval for each of its input resources.

    With this change, if/when a resource check is hanging or slow it at least won't gum up the pipeline scheduling for all the other jobs.

    Expect more improvements in this area in the next few releases! We'll be making jobs schedule in parallel soon so they can't affect each other, and we're working on a new "algorithm" that should scale a lot better with pipelines that have a ton of data or versions.

🔗 fix

  • The above refactoring also fixed a race condition that could result in inputs configured with version: every having versions skipped when a build is manually triggered.

🔗 feature

🔗 feature

🔗 feature

🔗 fix

  • version on a get step will now take precedence over versions pinned via the web UI or via version on a resource definition.

🔗 fix

  • The HD dashboard view got a little weird in the last couple releases - it's fixed now.

🔗 fix

  • Fixed the spacing of the pipeline view so super tall pipelines don't get clipped by the top bar.

🔗 fix

  • Fixed the status:running search functionality on the dashboard view.

🔗 fix

  • When viewing a pipeline build by ID (/builds/123), the top bar will show the breadcrumb for its pipeline and job instead of being empty.

🔗 fix

  • The breadcrumb in the top bar now uses actual links, so they can be middle-clicked and right-clicked to your heart's content.

🔗 fix

  • The groups bar on the pipeline view now has hover states for each group.

🔗 fix

  • Fixed a bug that caused credential managers to be instantiated twice, resulting in two auth loops.

🔗 fix

  • When viewing a one-off build in the web UI, the build will now render instead of chucking errors in the browser console.

🔗 fix

  • The web UI is now up-to-date with Elm 0.19! You shouldn't really notice anything, but...yay!

🔗 fix

  • Fixed a crash that would occur when a build finished that produced outputs for a resource that had been un-configured from the pipeline in the meantime.

🔗 fix

  • The web node will now retry on unexpected EOF errors which could occur when a worker was restarted while a build was running a container on it.

🔗 fix

  • Fixed a bug with the Vault login re-try logic that caused it to go into a fast loop after reaching the maximum interval. Now it'll actually stay at the maximum interval.

🔗 fix

  • When viewing a build for a job that has a ton of builds, only the first batch of builds will be fetched and rendered instead of all of them. Older builds will be automatically loaded if the build being viewed is old, or as the user scrolls to see them.

🔗 feature

  • We're now consistently using Material Design icons everywhere in our UI - the last of the Font Awesome stragglers have been replaced!

🔗 fix

  • Fixed quite a few quirks with the dashboard search:

    • Team name autocomplete will now work even if you're not logged in.

    • Fixed the unstyled autosuggest menu in Chrome.

    • Hitting the escape key will now un-focus the search field.

    • The search autocomplete will now only appear if you press a key with the search field focused.

    • Typing ? into the search field will no longer bring up the hotkey help pane.

🔗 fix

  • fly execute will now print the correct URL for the build when running with -j.

🔗 fix

  • fly login will now create ~/.flyrc with stricter permissions (0600).

🔗 feature

  • We've added a (hopefully subtle) stripey animation to running builds in the build number list to help differentiate between errored and running builds.

🔗 feature

🔗 fix

  • With v5.0.0 we introduced a bit of a performance regression with loading the versions for a pipeline during scheduling. We've made an incremental change to make it a bit faster.

    This will also be fixed by the new input candidate algorithm mentioned previously.

🔗 fix

  • The dashboard will no longer crash when a pipeline is configured with a circular dependency.

🔗 fix

  • Fixed the rendering of many, many pipeline groups.
Assets 14

@vito vito released this Mar 25, 2019

🔗 fix, security

  • Fixed a bug when saving wacky versions generated by wacky resource types that let you put wacky arbitrary data in the version.

    The bug enables limited SQL injection, so we recommend that anyone running 5.0 upgrade to this version as soon as possible. It's a bit concerning that we've ended up with a SQL injection vulnerability in 2019, but this at least appears to be an isolated and easily verifiable case. More on that later.

    Thankfully, this is very difficult and impractical to exploit, and the impact is fairly low despite it being a SQL injection:

    • It is only possible to inject a single SELECT query, so there should be no loss of integrity or data.

    • The SELECTed value would only be inserted into an internal column which is never exposed to users - it is only used for internal bookkeeping and putting something bogus there will have no effect on the rest of the system.

    • This issue only affects resource types that put arbitrary user-specified data into the resource version. This is very unusual - almost all resource types have strict, simple versions (e.g. git refs, version numbers, sha256 digests).

    • No core resource types are affected, and most resource types shouldn't be either. The only known resource types that do this are sort of hacky ones that propagate arbitrary data through the pipeline via resource versions.

    How this exploit happened:

    Normally, we use a lightweight framework for constructing queries safely (Masterminds/squirrel), and we always pass all user data as params ($1, $2, etc) so that escaping is never even necessary. In this case however the query was slightly more complicated, so we had to pop open the hood and directly construct a query fragment using sq.Expr.

    Unfortunately the portion that we injected did so by concatenating the resource version JSON into the query fragment. As a result, versions with a single-quote (') in them would break out of the surrounding string and insert their own SQL query. We've changed it to use a param instead, and we've done an audit of all other uses of sq.Expr to verify that they are only ever being given static strings, trivial pre-formatted data, or params.

🔗 fix

  • The BOSH release now sets file permissions for its config values as 0600, which fixes Postgres certificate configuration. Thanks for the PR, @flavorjones!

🔗 fix

  • The BOSH release now correctly handles array-values for authorized worker keys. Sorry about that!
Assets 8

@vito vito released this Mar 6, 2019

This release is a doozy. You should probably read these release notes in full - there are a ton of substantial new features and a good (bad?) amount of breaking changes.

Sorry this took so long! The holiday season took its toll, but we also got a bit overzealous with piling feature work on master, and well, we restructured the entire project and re-created its pipeline from scratch, so that didn't help.

On the plus side, the project restructure is now done, and we'll be implementing a new release process soon that should prevent these kinds of hold-ups from happening again in the future.

Special thanks to the many individuals in the community who took part in this release - whether you submitted a PR, helped triage issues, helped people out on the forums or in Discord, or simply cheered us on, every little bit helps keep the project humming along. We deeply appreciate it, and look forward to delivering y'all a better and better CI system - hopefully, more continuously.

🔗 feature, breaking

  • We have done a major internal overhaul of how resource versions are stored. As a result, the version history for each resource across your pipelines will be re-set upon upgrading to v5.0.

    The upgrade does however preserve the state of which versions were disabled, and the data relating versions to builds they were inputs to and outputs of.

    In versions prior to v5.0, resource version history was associated to a pipeline resource by name. This meant that if you changed a resource's configuration or type, those old versions would actually stick around, even though they may technically no longer be appropriate.

    With v5.0, resource versions are now tied directly to an anonymous "resource config" - basically the source: and type: for the resource. Pipeline resources instead point to a config, and if their source: or type: changes, they'll point to a new config with its own version history.

    This improves the correctness of the system as a whole, eliminating the need to ever "purge" the history of a resource.

    In addition, now that versions are tied directly to their configs, check containers are also shared across teams, reducing the overall container count. As a result however we limited who can fly intercept check containers.

    Building on this change, we are currently experimenting with improvements that can now be made to reduce the overall checking overhead across a Concourse cluster that has many equivalent resource definitions across pipelines and teams. This is currently off by default while we learn more about the implications - see Global Resources for more information.

🔗 fix, breaking

  • We have removed --allow-all-users as almost every use has been a misuse. You must configure users explicitly now instead. This was done for development environments but even those were trivial to switch to a local user whitelist.

    If you were setting this flag before, you probably didn't mean to - setting this with GitHub oAuth configured, for example, would allow literally everyone to be a part of your team and manage your pipelines.

    After upgrading, any teams that had this configured will preserve the behavior from before - they will continue to allow all users. The next time the teams are configured, however, you will have to specify something else, as the CLI no longer has the flag.

🔗 feature, breaking

  • The concourse binary distribution has been rejiggered. Rather than a self-contained binary, we now ship it as a .tgz containing the binary and its dependencies pre-extracted. The .tgz should be extracted somewhere like /usr/local, resulting in /usr/local/concourse/bin/....

    The main benefit of this is simplification and faster startup. The concourse worker command no longer needs to extract resource types/etc. on start, so this speeds that up quite a bit.

    The concourse binary no longer directly embeds Garden-runC code, and instead ships alongside the gdn binary, copied from their releases. This simplifies the interface for configuring Garden and allows us to leverage their build process rather than risking deviation.

    The "breaking" aspect of this is that if you have been passing esoteric flags to Garden you'll have to switch to using a config file via --garden-config instead, or pass them as env vars (e.g. CONCOURSE_GARDEN_FOO_BAR) - flags are no longer supported as those relied on directly embedding their code.

🔗 feature, breaking

  • Workers can now be configured to periodically rebalance so that they don't end up all forwarding through a single web node. This is done by setting the --rebalance-interval flag on concourse worker. The rebalancing makes sure to drain in-flight connections and should not disrupt any in-flight builds.

    Along the way, we removed support for direct worker registration. The --peer-ip flag is no longer available on concourse worker. To transition to 5.0, just remove the flag - the worker will register via forwarding instead.

    Forwarding is more secure as it doesn't require opening your workers up to inbound traffic. It's easier for us to just focus on one registration method and make sure it works well.

    This also sets us up for enforcing TLS for all traffic to the forwarded workers in the future (#2415).

🔗 feature, breaking

  • The Concourse BOSH release has been redesigned and is now centered around the concourse binary.

    warning: Be sure to recreate your workers after or during the deploy, as the location that the worker stores volumes has changed and the old volume directory will not be cleaned up, effectively leaking disk usage.

    warning: The additional_resource_types property can no longer be configured. We plan to add another mechanism for co-located resources in future releases.

    The concourse release no longer needs to be deployed alongside a garden-runc BOSH release, and instead embeds the gdn binary directly.

    Along the way, we have adopted BPM and now use it for deploying the web node. We also enforce a higher nofile limit which should make large-scale deployments more...scaley.

🔗 fix, breaking

  • Two flags have been modified to be more consistent with other flag syntax:

    • concourse web --vault-auth-param foo=bar should now be specified as concourse web --vault-auth-param foo:bar (note the :).

    • concourse web --tsa-team-authorized-keys team=path/to/key should now be specified as concourse web --tsa-team-authorized-keys team:path/to/key (note the :).

🔗 feature

  • The Concourse GitHub repository has been completely restructured. This isn't really a feature per se, but it should make contributing a lot easier.

    More on this on our blog post: The Great Process Update of 2018.

🔗 feature

  • A new resource, the registry-image resource, has been added to the core. This resource is intended to replace the docker-image resource image for image pulling and pushing (but not building).

    This resource improves on the docker-image resource in a few ways:

    • It doesn't run Docker to fetch the image - it's written in pure Go, using the google/go-containerregistry package. This makes the implementation much less error-prone.

    • Because it doesn't run Docker, it doesn't need a privileged container. The fewer privileged containers in your cluster, the better - especially in light of [v4.2.3](https://github.com/concourse/recent CVE fixes/releases/tag/v4.2.3).

    • By focusing solely on fetching and pushing, the resource is much smaller and simpler. It also has test coverage!

    • The output has pretty colors.

    This all results in much faster, more efficient, and resilient image fetching. We recommend everyone to try switching your image_resources and Resource Types over - in most cases this is just a matter of replacing type: docker-image with type: registry-image.

    We intend to deprecate and phase out support for the docker-image resource in favor of the registry-image resource. We can't really do this until there's a solid direction for image building - preferably with a task, not a resource. This is a more natural split, and supports building images without pushing them - a long awaited ask of the docker-image resource.

    An experimental task for this is available at concourse/builder. This is not yet official, but we've using it in our own pipeline and it's been pretty solid. Feel free to give it a try!

    The next step from here is to actually kick off an RFC for reusable tasks - we're still collecting our thoughts for that in (RF)RFC #7. Once this is done we can formalize concourse/builder.

🔗 feature

  • We have introduced the first phase of role-based access control!

    Right now there are only a few statically defined roles. We started off by supporting the common request of having read-only team members ('team viewer'), and adding a slightly less powerful 'team member' role. See User Roles & Permissions for more information.

    Here's a quick rundown of how things have changed:

    • Existing team auth config will be transitioned to the Team Owner role - that is, anyone that can authenticate prior to the upgrade will now be authenticated as an owner of their team. This role is the closest equivalent to what they could do before.

    • The main team still has special admin power, with the slight tweak that only users that are an owner of the main team have admin capabilties.

    • Before, teams members could rename or destroy their own team. Team owners no longer have this power - only admins can do this.

    • The Team Member role is a new role that allows users to have full read and write powers within the team, except for being able to modify the team itself.

    • The Team Viewer role is a new role that allows users to browse the team's pipelines, builds, resources, etc. without permitting any sensitive operations (like fly get-pipeline or triggering builds).

    For a detailed breakdown of each role's capabilties, see the Permission Matrix. To learn how to configure these roles after upgrading, see Setting User Roles.

    If you're curious about the design process for this feature, check out RFC #3 (RBAC)!

🔗 feature

  • We have replaced resource pausing with resource pinning.

    Resource pausing had the effect of disabling the periodic checking for the paused resource. However we found that in most cases it was being used in combination with disabling versions to effectively pin a resource to the most recent available version.

    However, with global resource versions, each resource actually points to a shared history, so pausing checking wouldn't be enough - if any other pipelines had the same resource, new versions would still arrive!

    So instead, versions can now be pinned individually via the web UI or via the pipeline config (see version). Pinned resources will also skip periodic checking, but now even if the checking still happens (because some other pipeline had it un-pinned), the resource will stay pinned to the desired version.

    A comment can also be left on pinned versions for explaining to your team-mates why you decided to pin the resource.

    During the 5.0 upgrade, paused resources will be automatically transitioned to their pinned equivalent, by pinning the resource to the most recent available version. A comment will be left on any resources that are migrated so that it's clear to pipeline users.

🔗 feature

  • Task ((vars)) received a bit of an overhaul, thanks to a PR by @ralekseenkov!

    • Values for task ((vars)) can now be provided during fly execute!

    • In addition, values may be provided to a task step in a pipeline via vars.

    • Tasks can now have ((vars)) pretty much anywhere in their config, not just in image_resource.

    In all cases, vars can also be satisifed via a credential manager, the same as before.

    Admittedly, there is now some cause for confusion with params. This may see clarification with reusable tasks. In addition, pipeline ((params)) will now be referred to as pipeline ((vars)) instead, for consistency.

🔗 feature

🔗 feature

  • Any volumes or containers that disappeared from their worker (possibly due to a worker being re-created and then coming back under the same name) will now be automatically reaped from the database. This makes it easier for Concourse to recover from this situation rather than erroring with file not found or unknown handle errors.

🔗 feature

  • Logs emitted by Concourse components will now be...slightly prettier? They're still JSON (sorry), but the timestamps and log levels are at least human-readable.

    If you've got anything parsing your logs, make sure to update it accordingly!

🔗 feature

  • Concourse will now automatically retry fetching credentials when the request to the credential manager fails, thanks to a PR by @ralekseenkov!

    By default Concourse will retry 5 times, waiting 1 second between each attempt. This can be adjusted with the --secret-retry-attempts and --secret-retry-interval flags on concourse web.

🔗 feature

  • Tasks are now permitted to have inputs, outputs, and caches with overlapping paths. This was a hold-over from older versions of the container runtime that did not support this.

    This means that for simple tasks that e.g. make a commit a git repo, you no longer need to copy the input to the output. Yay!

🔗 feature

  • The put step can now be explicitly given a list of inputs to use, rather than using all of them. This can be used to dramatically speed up builds that have a ton of artifacts prior to a put.

🔗 feature

  • The fly login flow has been reworked a bit to better support logging in to a remote session. There's now a prettier landing page that detects when the token transfer fails by allowing you to copy the token to your clipboard instead.

    The auto-login prompt will also no longer ask for the token, because that disrupts the normal flow of the command. Previously it would ask for a token but then eat half of the keystrokes from then on. Now it just won't ask for a token.

🔗 feature

  • The concourse binary now has a generate-key subcommand to assist with - you guessed it - key generation. This is more portable to other platforms (I'm looking at you, Windows) and is more likely to generate keys that Concourse can actually accept (I'm looking at you, OpenSSH 7.8).

🔗 feature

  • The concourse worker command can now be given a --garden-use-houdini flag on Linux to use the "no-op" Houdini Garden backend for those odd cases where you don't really want containerization. (Use sparingly.)

🔗 feature

  • The timestamps shown in the build header will now transition to absolute instead of relative when the build is over 24 hours old. It wasn't very useful to see things like 128d 15h 30m ago when trying to compare old builds. Thanks for the PR, @Twiknight!

🔗 fix

  • You may have seen a scary error cropping up around your resources now and then. Something like worker_resource_config_check__resource_config_check_sessio_fkey references unreticulated spline.

    We fixed it. That thing doesn't even exist anymore. Don't worry about it.

🔗 fix

  • With Concourse 4.x configured with an oAuth provider such as GitHub, a user could log in via GitHub even if they weren't technically a member of any team. They couldn't do anything, mind you, but it was confusing that they were allowed to log in in the first place.

    This is no longer permitted.

    Similarly, fly login will also check to make sure you've successfully logged in to the target team and return an error if the team isn't in your token.

🔗 fix

  • The AWS SSM credential manager and the AWS SecretsManager credential manager previously had a turf war going on over the AWS_REGION environment variable. They both declared it as their own, meaning if you set it they would both try to be configured, which would fail.

    They now have separately namespaced env vars instead.

🔗 fix

  • fly intercept will now give a better error when it fails to execute the command (e.g. because bash isn't installed in the image).

🔗 feature

  • fly execute can now specify input mappings via -m, which is useful when running with --inputs-from-job when the job renames some inputs.

🔗 fix

  • fly execute with --include-ignored will no longer blow up when files are removed locally.

🔗 feature

  • The error message when a task's file refers to an unknown artifact source (i.e. the foo in foo/ci/task.yml) has been made more descriptive.

🔗 feature

  • There's a new fly command for landing workers remotely, called... fly land-worker. This will initiate the landing process via the API and will ultimately result in the worker process exiting. (Which may end up being re-started by whatever process monitor you use, but hey, it landed.)

🔗 feature

  • The web UI now explains why some get steps have a yellow icon, via a handy-dandy tooltip. (Spoiler: it means the job has never run with that version before!)

🔗 fix

🔗 feature

  • fly watch can now be called with --timestamps to show per-line timestamps in the build output. Thanks for the PR, @pivotal-kahin-ng!

🔗 fix

  • fly get-pipeline will now throw an error if the specified pipeline does not exist, rather than returning an empty pipeline config.

🔗 fix

  • Fixed various subtle UI issues with the dashboard page: #2430, #2434, #2435.

🔗 fix

  • fly login will no longer prompt for your auth method when a username/password are given via flags. It'll deduce that you're trying to do local auth.

🔗 fix

  • Task caches are now supported on Windows!

🔗 fix

  • Fixed an internal bug that made UNIQUE constraints for resource_configs ineffective (#2509). This was fairly low-impact, but database integrity matters!

🔗 feature

  • BitBucket auth support has been re-introduced thanks to PRs to Dex and Concourse by @edtan!

🔗 fix

  • The /api/v1/resources and /api/v1/jobs endpoints will now return [] instead of null when there are no resources or jobs, thanks to a PR by @pivotal-kahin-ng.

🔗 feature

  • The dashboard page will now indicate whether you are seeing a pipeline because it's exposed by showing an ominous "eye" icon.

🔗 fix

  • Fixed handling of auth configs set from empty env vars - previously this would result in bogus Dex configuration (e.g. github:, with no org or team) and sometimes cause things to misbehave.

🔗 fix

  • The legibility and anti-aliasing of text in the web UI has been improved.

🔗 fix

  • Cleaned up some dashboard behavior when there are no pipelines:

    • you can now see which team you're a member of, rather than one big 'no pipelines set' page

    • the bar along the bottom will now show up

    • there's a fancy ASCII art UI now

    • the search function is no longer shown (since there's nothing to search)

    • the HD view has been disabled and just redirects to / instead, since there was nothing for it to show

🔗 fix

  • The username part of the top bar will no longer detonate when viewed on a tiny mobile browser.

🔗 fix

  • When a resource's metadata is super wide, it will remain cordoned off to the side rather than uncomfortably squishing the resource's get output. Thanks for the fix, @stigtermichiel!

🔗 fix

  • Concourse will now send TCP keepalives for connections to the database. This will allow it to detect when the connection has been interrupted ungracefully. Thanks for the PR, @SimonXming!

🔗 fix

  • The manifest.json href in the web UI used to be relative to the URL, meaning it was broken on any page except /. This is now fixed.

🔗 fix

  • The web node used to leak both a connection and a goroutine for each build that completed when configured to drain build logs to syslog. This is now fixed. Sorry about that!

🔗 fix

  • The resources and resource types returned by fly get-pipeline will now be in a deterministic order, thanks to a PR by @edtan!

🔗 feature

  • fly curl is a new command to assist with (hopefully occasional) manual API requests to Concourse. Thanks for the PR and collaboration, @simonjohansson!

🔗 fix

  • The --tsa-authorized-keys flag is now optional, for situations where all authorized keys are associated to teams (via --tsa-team-authorized-keys). Thanks for the fix, @tlwr!

🔗 fix

  • The fly status command will now let you know if your token has expired, rather than happily reporting that everything is fine.

🔗 feature

  • A fly userinfo command has been added which will let you know which teams you are logged in to and which roles you have in each team.

🔗 fix

  • The positioning of the "no results" text when searching on the dashboard has been fixed.
Assets 8
Feb 25, 2019
Merge pull request #3375 from concourse/dont-stop-me-now
ci: stop docker only in the happy path

@vito vito released this Feb 15, 2019 · 0 commits to bcd4251a61445b139f731588a211f172e923ca47 since this release

🔗 fix, security

  • This release bumps our Garden-runC dependency to v1.18.2 which fixes CVE 2019-5736. We recommend that you upgrade your Concourse cluster to v4.2.3 to prevent this exploit from occurring.

    Concourse relies on Garden-runC to create containers for executing jobs and resource checks in pipelines. By default, all containers created by Concourse are unprivileged, and should be safe from CVE 2019-5736.

    However, if your pipelines configure privileged: true on tasks or privileged: true on resource types in your pipelines, these containers will be privileged, exposing the worker to the attack vector described in CVE 2019-5736. One common example of this is the docker-image resource, which is always privileged.

🔗 fix

  • The CF/UAA auth connector has been updated to use the authorization_endpoint so that the authentication flow can be completed successfully. Previously, authentication flows would fail whenever a third-party SAML redirect is required.
Assets 14
You can’t perform that action at this time.