Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow archiving / retiring pipelines #2518

Open
tkellen opened this issue Aug 21, 2018 · 15 comments
Labels

Comments

@tkellen
Copy link
Contributor

@tkellen tkellen commented Aug 21, 2018

Feature Request

I'd like the ability to archive a pipeline without deleting it so the build history / links are still usable but the UI isn't cluttered when a given pipeline is no longer needed.

A Modest Proposal

This seems like a pretty easy series of pull requests. Here's what I have in mind:

  1. Create a migration adding an archived boolean field to the pipelines table.
  2. Add support to the pipelines endpoint in atc to filter on archive status for GET and modify archive status with PUT.
  3. Add archive-pipeline and unarchive-pipeline commands to fly (basically a cut/paste job of hide/expose)
  4. Add a toggle in the UI (exactly the same as the high density toggle) to control if archived pipelines or active pipelines are currently being viewed.
  5. Mirror whatever testing practices are in place for all the aforementioned features to ensure new functionality is covered.

What do ya'll think?

PS: Thanks so much for making concourse, I am really enjoying using it!

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Aug 22, 2018

The PRs associated with this issue now provide minimal API and CLI support for this functionality.

No harm no foul if this turns out to be an undesirable feature. This was an interesting exploration of the concourse source code for me.

Assuming this is something ya'll would like to see through, more work needs to be done on test coverage, support needs to be added to the UI, and I'm sure there are plenty of implications I haven't considered that should be discussed.

Standing by!

@vito

This comment has been minimized.

Copy link
Member

@vito vito commented Aug 23, 2018

Neat, I like the idea. One thing to consider is that we should probably have archived pipelines become effectively paused, right?

Glad to see it wasn't too hard to dive in to the code! FWIW, we're planning to do a big restructuring that'll hopefully make it much easier.

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Aug 23, 2018

Oh good call. Yeah, pausing would definitely be warranted. Would unpausing be disabled until a pipeline is unarchived, or would unpausing implictly unarchive a pipeline?

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Aug 23, 2018

Also, I just read your note about restructuring. Kudos! The unix philosophy is fantastic. Applying it to git repositories is a nightmare for most, though. You can use folders to accomplish the same thing people 🤣! Monorepo for the win.

@vito

This comment has been minimized.

Copy link
Member

@vito vito commented Aug 28, 2018

@tkellen Hmm, it might be better to just have 'archived' state behave the same as 'paused' so that we never have to wonder things like "what happens if I unpause an archived pipeline" and so that people don't forget to pause it before archiving it.

The onus would be on the code to now have to handle both cases. Maybe they could just be 'or'ed together somewhere low-level enough (e.g. when pulling data from the db) and only expose something like .Active higher up, so it's hard to make the mistake of only checking for one or the other. So a pipeline would still have paused and archived columns in the database, but things like the scheduler and radar package would just check if it's active.

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Aug 28, 2018

@vito Got it. We'd also probably want this as a derived field in the API responses so we can have the UI similarly check for "active" when displaying paused state, right?

@vito

This comment has been minimized.

Copy link
Member

@vito vito commented Aug 28, 2018

@tkellen I'm not sure. I figure archived pipelines should show up differently in the UI already (possibly not at all on some pages, as I suspect that's your goal), and maybe that visual cue can make it obvious that it's not "active" without making it look like it's been explicitly paused (as that could be misleading).

Maybe @Lindsayauchin has ideas here?

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Aug 28, 2018

Archived status having a wholly separate (and superseding) representation works for me and keeps the implementation a bit more straightforward. I think the toggle still makes sense (perhaps only showing active or inactive, though rather than combined), but I am happy to hold off on any UI work until @Lindsayauchin weighs in.

I can proceed with further updates to support archive mode implicitly pausing. Can you tell me precisely which layer of abstraction you'd like .Active to be represented?

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Sep 6, 2018

Any thoughts @Lindsayauchin? Thanks in advance!

@Lindsayauchin

This comment has been minimized.

Copy link
Contributor

@Lindsayauchin Lindsayauchin commented Sep 19, 2018

@tkellen - Awesome idea! Great that you are working on this!

Couple questions for you on your use case:

  • Can you give an example of the last time you wanted to do this and what did you do to work around this?
  • How often are would you use this feature?
  • Are you expecting a view of all the archived pipelines in your team or across the instance?
  • How deep are you expecting to go into the read-only pipeline? ie. how far in the history of builds?
  • Are you expecting to unarchive a pipeline from the UI?
@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Sep 19, 2018

Hi @Lindsayauchin. Thanks for the reply 💃, I'll jump right in:

  1. Example of last time I wanted to do this: when I was ready to discard a pipeline that is no longer needed technically but necessary to retain the history of for audit purposes. The workaround was renaming it. My actual use case is watching a repo and creating a new pipeline for each branch and archiving it when the branch is deleted (maybe someday #1707 will obviate this need but it looks to be very far off).
  2. Daily.
  3. By team.
  4. Access to the full history seems warranted?
  5. No.
@Lindsayauchin

This comment has been minimized.

Copy link
Contributor

@Lindsayauchin Lindsayauchin commented Sep 20, 2018

@tkellen @vito

With the introduction of the tags for a 'member' or 'exposed' teams, we could use the area beside this element to show if there are archived pipelines for that team with a counter.

A user could then select this to display the archived pipelines with the current active pipelines.

screen shot 2018-09-20 at 12 06 59 pm

screen shot 2018-09-20 at 12 06 46 pm

The pipeline cards on the dashboard view would have some new UI elements such as: an icon for archived displayed where the timestamp normally appears, the display of the thumbnail dimmed down to indicated it is no longer active, the running animation stripe displaying grey to represent the pipeline being inactive.

screen shot 2018-09-20 at 11 44 52 am

As the user drills down to the buildpage from the dashboard the breadcrumbs will display the archived icon preceeding the pipeline name.

pipeline_archived

User will have a read-only view of the build page and the trigger button will not be displayed.

pipeline_build_page_archived

Asset:

baseline-archive-24px.svg.zip

@vito

This comment has been minimized.

Copy link
Member

@vito vito commented Sep 20, 2018

@tkellen

This comment has been minimized.

Copy link
Contributor Author

@tkellen tkellen commented Sep 20, 2018

Nice! I think I can run with that, thank you so much @Lindsayauchin!

I tried bringing up the new monorepo to re-open my PR and ran into:
maxbrunsfeld/counterfeiter#94

@vito, will there be an announcement of some kind when you're ready for contributors again? I am following #2534.

@vito

This comment has been minimized.

Copy link
Member

@vito vito commented Sep 20, 2018

@tkellen Yeah, I'll post an announcement somewhere when things are all ready. Probably the #contributors channel on Discord, and maybe a few other places. I can help you along if you want to keep working on it soon. I'll update the contributing docs once things are smoother.

tkellen added a commit to tkellen/concourse that referenced this issue Sep 24, 2018
tkellen added a commit to tkellen/concourse that referenced this issue Sep 24, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
tkellen added a commit to tkellen/concourse that referenced this issue Sep 25, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
tkellen added a commit to tkellen/concourse that referenced this issue Sep 25, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
tkellen added a commit to tkellen/concourse that referenced this issue Sep 26, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
tkellen added a commit to tkellen/concourse that referenced this issue Oct 2, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
tkellen added a commit to tkellen/concourse that referenced this issue Oct 12, 2018
adds functionality in fly and api for archiving and unarchiving
pipelines, in support of concoursegh-2518.

replicates:
concourse/go-concourse#17
concourse/fly#258
concourse/atc#300
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.