Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some releases fail to clone. Run action shows set-output warning instead. #14

Closed
jrbe228 opened this issue May 13, 2023 · 14 comments
Closed
Assignees
Labels
enhancement New feature or request feature New functionality. python Pull requests that update Python code

Comments

@jrbe228
Copy link

jrbe228 commented May 13, 2023

I get the following output from building an Action -

Adding v0.5.2-beta to ...
Adding v0.5.1-beta to ...
Adding v0.5.0-beta to ...
Adding v0.4.0-beta to ...
Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Of the 7 releases in the source repo, the first 4 now appear as drafts in my target repo. The other 3 fail to clone. Of the 4 repos which did manage to transfer as drafts, all of them were missing binaries.

My YAML -

name: Clone
on: push
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Run action
        # Put your action repo here
        uses: andrewthetechie/gha-clone-releases@v1.5.0
        # Put an example of your mandatory inputs here
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          src_repo: <internal_org_repo>

Expected behavior
All 7 repos transfer, including binaries.

@andrewthetechie
Copy link
Owner

Hi. Thanks for the issue

The action does not copy files from the releases on the target repo to your repo. It only creates releases on your repo that match the release on the target. If you want to copy release files, you would need another workflow to run on release

I'll look into the set output issue

@jrbe228
Copy link
Author

jrbe228 commented May 13, 2023

I can edit the ReadME to note that info. Right now the text says "clone releases from a source repo to your repo". So it's unclear if binaries are included or not. Could that feature be added as an option? Including binaries is very helpful in most cases I can think of.

Another minor item I just noticed. The "prerelease" checkbox is checked on source repo, but appears not to carry through to the target repo's drafts.

@andrewthetechie
Copy link
Owner

I'm taking a look at this today and will see what I can do.

@andrewthetechie
Copy link
Owner

#15 - Fixes the set-output issue
#17 - Adds draft to the fields copied from source Releases to the created Release.

These will be available in the v1.6.0 release once the release PR is merged

I'm still looking into copying release assets.

@andrewthetechie andrewthetechie added enhancement New feature or request feature New functionality. python Pull requests that update Python code labels May 13, 2023
@andrewthetechie andrewthetechie self-assigned this May 13, 2023
@andrewthetechie
Copy link
Owner

Please try out the 1.6.0 release and the new copy_assets input and let me know if that works for your needs.

@jrbe228
Copy link
Author

jrbe228 commented May 14, 2023

Having an issue accessing the 1.6.0 release -

Pull down action image 'ghcr.io/andrewthetechie/gha-clone-releases:v1.6.0'
  /usr/bin/docker pull ghcr.io/andrewthetechie/gha-clone-releases:v1.6.0
  Error response from daemon: Head "https://ghcr.io/v2/andrewthetechie/gha-clone-releases/manifests/v1.6.0": unauthorized
  Warning: Docker pull failed with exit code 1, back off 2.8[9](https://github.com/Ansys-DEE/td-mc-plugin-releases/actions/runs/4973417576/jobs/8899249193#step:2:9)3 seconds before retry.

Same issue going to that link in a browser -
https://ghcr.io/v2/andrewthetechie/gha-clone-releases/manifests/v1.6.0 ---> Chrome returns --->
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required"}]}

@andrewthetechie
Copy link
Owner

I missed a checkbox to enable un-authed access to the ghcr package. Please give it a try again and it should work for you now.

@jrbe228
Copy link
Author

jrbe228 commented May 15, 2023

Several improvements over last time!

  • Container access issue fixed
  • Pre-release checkbox now checked
  • Tags are now migrating (just noticing this was not the case in v1.5.0).
  • Each release migrates as a completed release! Not a draft.
  • Some assets are migrated.
  • set-output() logging issue fixed.

Remaining possible improvements.

  • Still only migrating 4 of the 7 releases. This may be Ok for my specific case. The 3 repos being missed are much older releases... as long as recent (and all future) releases are migrated, I'm happy.
  • Only 2 of the 3 assets were migrated. Here is the output log -
##[debug]mcd-plugin has 7 releases
##[debug]td-mc-plugin-releases has 3 releases
##[debug]Limit: 0
Adding v0.5.0-beta-mc.22.2 to td-mc-plugin-releases
Error: Error while copying asset TD_MC_v050_BetaMC22_2.zip. 404 Client Error: Not Found for url: https://github.com/mcd-plugin/releases/download/v0.5.0-beta-mc.22.2/TD_MC_v050_BetaMC22_2.zip
##[debug]Docker Action run completed with exit code 0
##[debug]Set output addedReleases = v0.5.0-beta-mc.22.2
##[debug]Set output skippedReleasesCount = 0
##[debug]Set output addedReleasesCount = 1
##[debug]Finishing: Run action

Notes.

  • The message td-mc-plugin-releases has 3 releases is because I manually deleted v0.5.0-beta-mc.22.2 before running the Action. Otherwise it would have all 4 releases from previous testing.
  • The 2 assets which migrated successfully were the source code ZIP and TAR.GZ. The failed asset TD_MC_v050_BetaMC22_2.zip contains compiled binaries.
  • Lastly, when I click the GitHub URL in the error message Error while copying asset TD_MC..., a download immediately starts for that asset. You wouldn't be able to follow the same link. I sanitized the error logs, removing company-specific info.

@andrewthetechie
Copy link
Owner

When I try to open that URL, it returns a 404. GitHub returns a 404 when you don't have permission to access a resource rather than a 403 to prevent enumeration. I'd suggest you check out the permissions of the token you're using with the action to see if, for some reason, it doesn't have access to that file.

As for the other issues, I added more debug logging in v1.6.5 (ignore the interim releases). That debug logging might give you more insight into why releases are not being cloned.

@jrbe228
Copy link
Author

jrbe228 commented May 16, 2023

Got some useful info!

Failure to migrate the first 3 of the 7 releases

  • This was caused by our early naming convention. Versions were only noted in the Tag. Names were repeated for Releases 1 - 4. So the Action only migrated Release 4, but ignored Releases 1 - 3.
  • For Release 5 - 7, we were noting version number in both the Name and Tag. So all of those releases were uniquely named and the Action migrated them.
  • It's Ok for my use case. We can name releases uniquely going forward. No "fix" required unless you think other devs would run into the same issue.

Releases are migrated without assets

  • This one I've partially figured out. Permissions are part of the story. Using secrets.GITHUB_TOKEN results in no visibility to other Private or Internal repos. No releases get migrated. This is a known bug - GITHUB_TOKEN does not have access to other private packages  actions/setup-node#49.
  • Using secrets.<PAT> works better. That leads to the situation above, where 4 of 7 releases are migrated. Previously I thought 2 of 3 assets were migrating. That was incorrect. The source code ZIP and TAR.GZ were essentially empty, only containing the README.md and .github folder from the destination repo. Nothing from the source repo. So actually, no assets are currently migrating when using Private or Internal repos.
  • I've tried using different PATs with various access scopes. Also tried running the action from the source repo instead of destination repo. No luck.
  • One idea after some Googling. Could this be caused by how the Python "requests" library formats GET requests? 404 error using github hashable URL jazzband/pip-tools#947 (comment)

@andrewthetechie
Copy link
Owner

Please try v1.7.0 and let me know if it works as expected.

I created a private repo and was able to clone a release and copy over the assets using a Personal Access Token (PAT). You'll have to use a PAT with access to the source repo for this to work. Thank you for the link to the pip-tools issue; that helped.

Just to comment on something from earlier discussion, the action is not copying the Source Code zip and tar.gz from the source repo. Those assets are created automatically when you create a release. They will contain the source code of the destination repo, at the time the release and tag are created. I don't think you can delete these.

@jrbe228
Copy link
Author

jrbe228 commented May 16, 2023

v1.7.0 is working!! This is extremely helpful for our team. Now we can keep source code in the private repos, while copying releases to internal / public repos. All in a simple automated way.

No further code changes to suggest. A few comments on the docs.

  • Note that Releases must be uniquely named or they will fail to copy.
  • Need to note using PATs instead of GITHUB_TOKEN with private / internal repos. I think you could just post this link rather than explain the backstory - GITHUB_TOKEN does not have access to other private packages  actions/setup-node#49. Otherwise each new user would likely take a few tries before realizing they need to use a PAT.
  • Consider on: push for the example workflow. This is the most common use case I can think of. Placing the workflow in the source repo ensures that any new releases are migrated as soon as they are uploaded to GitHub.
  • For the example workflow. In an earlier version, repo was a required parameter. Now it's src_repo.
  • For the example workflow. Consider locking onto a specific ubuntu version rather than ubuntu-latest. This way a bad ubuntu build will never break your code. It's happened to me before with ubuntu 22.04 breaking some of my legacy software builds. But on the other hand, pulling ubuntu-latest should mean less Linux kernel vulnerabilities. I'm on the fence about this one.

@andrewthetechie
Copy link
Owner

Glad it's working, I'll update some docs to mention the new feature and update that example you found.

@all-contributors please add @jrbe228 for bug and ideas

@allcontributors
Copy link
Contributor

@andrewthetechie

I've put up a pull request to add @jrbe228! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature New functionality. python Pull requests that update Python code
Projects
None yet
Development

No branches or pull requests

2 participants