Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(In Progress) Composing Actions Within Actions Proof of Concept #612

Closed
wants to merge 27 commits into from

Conversation

ethanchewy
Copy link
Contributor

@ethanchewy ethanchewy commented Jul 23, 2020

In this PR, I show a proof of concept for calling actions within actions.

For calling another action, in this PR, it will only support with, uses, and env).

If need be, I'll create another ExecutionContext for Composite Actions in this PR as well.

@ethanchewy ethanchewy marked this pull request as draft Jul 23, 2020
@ethanchewy ethanchewy changed the title Composing Actions Within Actions Proof of Concept (In Progress) Composing Actions Within Actions Proof of Concept Jul 23, 2020
@ethanchewy
Copy link
Contributor Author

ethanchewy commented Aug 3, 2020

TODO: Don't forget to hide with an environment flag or some sort of check like 1 != 0.

@Shane32
Copy link

Shane32 commented Feb 12, 2021

What's the status of this?

@afirth
Copy link

afirth commented Feb 27, 2021

@hross since you were active in #646. Since Ethan left it seems like these are totally parked. uses is a major blocker for me.

Use case, deploy k8s manifests to EKS, with the manifests living with each services' code.

Getting the credentials is a hot mess from AWS, which they've had the good grace to package up for us. I saw workflow templates, but they are only available for Enterprise and One. This type of thing is already available in travis, buildkite, circleci, etc.

It's also disappeared from the roadmap. Is this all bundled up into github/roadmap#98? Will that be rolled out for non-enterprise at some point too?

Actions is awesome, but this is stopping it from challenging the existing best-in-class.

@hross
Copy link
Contributor

hross commented Feb 28, 2021

Thanks for reaching out @afirth. We would still like to do this and I agree it is a pain point with actions. I can't comment too much on the roadmap so I will reach out to one of our PMs in this space.

At this time we don't have plans to work on this item in the near term (next month) but we would like to invest here as soon as we can get some higher priority items off of our plate.

@afirth
Copy link

afirth commented Feb 28, 2021

Thanks for reaching out @afirth. We would still like to do this and I agree it is a pain point with actions. I can't comment too much on the roadmap so I will reach out to one of our PMs in this space.

At this time we don't have plans to work on this item in the near term (next month) but we would like to invest here as soon as we can get some higher priority items off of our plate.

thanks for the update

@cpumanaz
Copy link

cpumanaz commented Apr 16, 2021

Thanks for reaching out @afirth. We would still like to do this and I agree it is a pain point with actions. I can't comment too much on the roadmap so I will reach out to one of our PMs in this space.

At this time we don't have plans to work on this item in the near term (next month) but we would like to invest here as soon as we can get some higher priority items off of our plate.

@hross It's been a couple months, and this has stalled our GH Actions migration from Jenkins. I'm considering going with some third party tooling to make this a better experience. nested using in composite actions will improve the DRY. I can have 10 projects that do the same thing with different inputs use external actions etc. Instead of having to duplicate all the using and code and only getting marginal benefit with the composite steps.

Should I wait? (and for how long) or should I go with this third party?(#646 (comment)) or go back to Jenkins?

GH Actions is so close to being awesome, but until a product owner understands that DevOps people (your primary customers) really hate duplication, GH Actions will continue to be stalled, and you will not grow your userbase with this feature.

@hross
Copy link
Contributor

hross commented Apr 16, 2021

@cpumanaz we are restarting some work in this area, actually. I can't make any guarantees on product timelines but we expect to work on it in the coming months. Have you seen this ADR? If you have any feedback there we would love to hear it.

@davidkarlsen
Copy link

davidkarlsen commented Apr 16, 2021

+1 are you pouring enough resources into actions? If we look at ADRs they move really slow, for instance this trivial change: actions/setup-java#98 took 8-9 months. I would guess composites takes more effort - and if that is linear will take over a year, it's already been considerable time.

@cpumanaz
Copy link

cpumanaz commented Apr 16, 2021

+1 are you pouring enough resources into actions? If we look at ADRs they move really slow, for instance this trivial change: actions/setup-java#98 took 8-9 months. I would guess composites takes more effort - and if that is linear will take over a year, it's already been considerable time.

I agree @davidkarlsen. Azure DevOps has this now, CircleCI has this now, TravisCI has this. GitHub could have this in a quarter and could strip business from CloudBees, CircleCI, Travis and others, because they offer the whole package. Plus the community actions make GitHub sticky, that means more reoccurring revenue, more reason to pay for higher tiers of service.

GitHub is a business like any other. As a DevOps Manager, I see our developers, and my team take what is prioritized in the backlog. This need likely is not prioritized. I have been reaching out to sales and product owners on LinkedIn in hopes that I can get some attention here. I don't work for the largest company, but if we have a few large companies, or many companies asking product for this feature, I imagine it would work its way to the top of the backlog in no time.

If anyone from a Fortune 100 company reads this, work with your leadership team to partner with GitHub to get this prioritized. This can benefit your internal self-hosted GitHub Enterprise as well.

-Shoosh

@MrGraversen
Copy link

MrGraversen commented May 9, 2021

GH Actions is so close to being awesome, but until a product owner understands that DevOps people (your primary customers) really hate duplication, GH Actions will continue to be stalled, and you will not grow your userbase with this feature.

Well put, @cpumanaz. Initially starting out with GitHub Actions is awesome, don't get me wrong, but as the project scales, GitHub Actions are having a hard time keeping up, as consolidation of workflows through actions are hindered by their lack of composability.

@aavileli
Copy link

aavileli commented May 30, 2021

Whats the new issue number for this feature ? Its a very important feature to keep configs DRY.
People are now coming up with other workarounds as GA looks to have no timeline or status on this issue
hacks :: https://github.com/marketplace/actions/github-workflow-sync

@AlexDHoffer
Copy link

AlexDHoffer commented Jul 1, 2021

Please get this on the roadmap. This is a sorely needed feature for anybody who doesn't want to copy-paste their workflows everywhere.

@cpumanaz
Copy link

cpumanaz commented Jul 1, 2021

I'm done with GH Actions.

https://github.com/argoproj/argo-workflows

Kubernetes native, inheritance, shared code, good community stuff. No proprietary stuff. 4 months of no action, or no communication isn't helpful.

Modern look/feel compared to Jenkins.

@nokite
Copy link

nokite commented Jul 26, 2021

How can this be a prioritization issue, how can Github not have enough resources for a feature this important? I'm actually surprised Github Actions was released at all - without any solution for code reuse.

I feel trapped in Github Actions now, having recently migrated to it, invested in it, and already considering if that was the right choice. Sorry for the drama, but it feels deserved in this case. It's a paid solution, not a 3-person open-source library 🙂

@pablotrinidad
Copy link

pablotrinidad commented Aug 5, 2021

Is there any way the community can get involved into solving this? This feature will really take GitHub Actions to the next level.

@ChristopherHX
Copy link
Contributor

ChristopherHX commented Aug 6, 2021

Most of this feature reached the (pre)release 2.280.1, but is behind the feature flag DistributedTask.EnableCompositeActions (which is still disabled and hosted runners are still running 2.279.x). If expressions in composite actions are still missing...

If you want to try this brand new feature locally, you could try my fork of this runner it has a console client for the runner.
This is a sample of the new composite actions in runner 2.280.1, you can also call other composite actions.

newcomposite-sample Repository

.github/comp1/action.yml

runs:
  using: composite
  steps:
  - uses: actions/setup-node@v2
    with:
      node-version: '16'
  - run: |
      console.log(process.version);
      console.log("Hi from node");
    shell: node {0}

.github/workflows/main.yml

on: push
jobs:
  comp:
    runs-on: self-hosted
    steps:
    - uses: actions/checkout@v2
      with:
        path: testa
    - uses: ./testa/.github/comp1

@davidkarlsen
Copy link

davidkarlsen commented Aug 8, 2021

How does one activate the DistributedTask.EnableCompositeActions flag?

@ChristopherHX

This comment has been minimized.

@thboop
Copy link
Collaborator

thboop commented Aug 9, 2021

How does one activate the DistributedTask.EnableCompositeActions flag?

We will enable it for everyone once the feature is ready! Thank you for your patience.

@mikebroberts
Copy link

mikebroberts commented Nov 17, 2021

For anyone else that gets this far, runs.steps[*].uses is now supported, which was sufficient for me. https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-composite-actions

@thboop
Copy link
Collaborator

thboop commented Nov 17, 2021

This is live, as shown in the docs above

@jackton1
Copy link

jackton1 commented Jan 28, 2022

Does anyone know if this feature is supported on self-hosted runners?

Log output:

Current runner version: '2.286.1'
Runner name: 'actions-runner-deployment-jznqh-h6x2h'
Runner group name: 'Default'
Machine name: 'actions-runner-deployment-jznqh-h6x2h'
GITHUB_TOKEN Permissions
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'actions/checkout@v2' (SHA:ec3a7ce113134d7a93b817d10a8272cb61118579)
Download action repository 'tj-actions/changed-files@v14.1' (SHA:445bbefc90b76c794710efc73b01e1438611cad3)
Download action repository 'actions/github-script@v2' (SHA:44b873bc975058192f5279ebe7579496381f575d)
Download action repository 'actions/setup-java@v2' (SHA:f0bb91606209742fe3ea40199be2f3ef195ecabf)
Download action repository 'gradle/wrapper-validation-action@v1' (SHA:e6e38bacfdf1a337459f332974bb2327a31aaf4b)
Download action repository 'C2E/action-surefire-report@master' (SHA:f01db0d1e61e196ed61ebe54e2a39e3337c645b9)
Download action repository 'EnricoMi/publish-unit-test-result-action@v1' (SHA:bffeee8009970dbb7444de4027c06333c97a8a54)
Download action repository 'actions/upload-artifact@v2' (SHA:82c141cc518b40d92cc801eee768e7aafc9c2fa2)
Download action repository 'docker/setup-buildx-action@v1' (SHA:94ab11c41e45d028884a99163086648e898eed25)
Download action repository 'mathieudutour/github-tag-action@v6.0' (SHA:d745f2e74aaf1ee82e747b181f7a0967978abee0)
Download action repository 'docker/login-action@v1' (SHA:42d299face0c5c43a0487c477f595ac9cf22f1a7)
Error: `uses:` keyword is not currently supported.

@ClenchPaign
Copy link

ClenchPaign commented Jan 28, 2022

Additional content on top of @jackton1 comment.
This is how the action is used in the workflow:

- name: Get changed files
  id: changed_files
  uses: tj-actions/changed-files@v14.1
  with:
    base_sha: ${{ steps.head_commit.outputs.sha }}
    sha: ${{ steps.latest_tag_sha.outputs.sha }}
    since_last_remote_commit: true
    files: |
      common/libs/**
      tests/**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet