Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pushing is slower than expected (> 2min) #386

Closed
melink14 opened this issue Apr 3, 2022 · 12 comments · Fixed by #423
Closed

Pushing is slower than expected (> 2min) #386

melink14 opened this issue Apr 3, 2022 · 12 comments · Fixed by #423
Assignees
Labels
status: pinned Should not be labeled as stale type: feature New feature or feature request

Comments

@melink14
Copy link

melink14 commented Apr 3, 2022

Describe the bug
I created a workflow which pushes updated visual regression screenshots when those tests fail, pushing the images takes twice as long as running the entire presubmit lint/test suite.

Workflow used
https://github.com/melink14/rikaikun/blob/23fc0df1f33bd32456a73e09ddc16e19c52a3087/.github/workflows/presubmit.yml

Expected behavior
Based on pushing from my computer and pushing that happens during other Actions (like semantic release) I expect it to be quite fast to push a single commit to git.

Logs
https://github.com/melink14/rikaikun/runs/5803997253?check_suite_focus=true

@melink14 melink14 added the status: pending More info is needed before deciding what to do label Apr 3, 2022
@stale
Copy link

stale bot commented Apr 23, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the status: stale Inactive issues and PRs label Apr 23, 2022
@EndBug EndBug added status: pinned Should not be labeled as stale and removed status: stale Inactive issues and PRs labels Apr 23, 2022
@kachkaev
Copy link

kachkaev commented Aug 11, 2022

Same here in a private repo. I am pushing to a lightweight branch in a repo that has a lot of heavy branches. It feels as if the command pulls all branches first before pushing. Just judging by its time – it takes 6+ minutes.

UPD I removed the pull option but the action did not become faster. It was originally pull: "--rebase --autostash". There must be a way of pulling just a single branch, but performance is not a high priority in my case. It’d be great if the action was as fast as running git commit & git push though.

@EndBug
Copy link
Owner

EndBug commented Aug 18, 2022

Hi @kachkaev, thanks for writing in
Would you be able to provide some logs? I'm trying to replicate this behavior since I should have some time to look into it, but the logs from the issue author have expired.

@kachkaev
Copy link

👋 @EndBug! Here are my logs:

2022-08-05T05:57:16.6131225Z ##[group]Run EndBug/add-and-commit@v9
2022-08-05T05:57:16.6131487Z with:
2022-08-05T05:57:16.6131694Z   author_name: GitHub Actions
2022-08-05T05:57:16.6132032Z   author_email: github.actions@my-project.example.org
2022-08-05T05:57:16.6132372Z   message: Update seed data
2022-08-05T05:57:16.6132626Z   pull: --rebase --autostash
2022-08-05T05:57:16.6132856Z   push: true
2022-08-05T05:57:16.6133332Z   cwd: some-cwd
2022-08-05T05:57:16.6133559Z   add: .
2022-08-05T05:57:16.6133782Z   default_author: github_actor
2022-08-05T05:57:16.6134041Z   pathspec_error_handling: ignore
2022-08-05T05:57:16.6134593Z   github_token: ***
2022-08-05T05:57:16.6134799Z ##[endgroup]
2022-08-05T05:57:19.9912054Z 
2022-08-05T05:57:19.9917541Z 
2022-08-05T05:57:19.9918073Z 
2022-08-05T05:57:19.9918542Z 
2022-08-05T05:57:19.9919006Z 
2022-08-05T05:57:19.9919470Z 
2022-08-05T05:57:20.0064685Z Running in /home/runner/work/repo-name/repo-name/some-cwd
2022-08-05T05:57:20.0087082Z Add input parsed as single string, running 1 git add command.
2022-08-05T05:57:20.0088165Z > Using 'GitHub Actions <github.actions@my-project.example.org>' as author.
2022-08-05T05:57:20.0090565Z > Using "Update seed data" as commit message.
2022-08-05T05:57:20.0092763Z ##[group]Internal logs
2022-08-05T05:57:20.0097351Z > Staging files...
2022-08-05T05:57:20.0098326Z > Adding files...
2022-08-05T05:57:20.1205370Z > No files to remove.
2022-08-05T05:57:20.1205889Z > Checking for uncommitted changes in the git working tree...
2022-08-05T05:57:20.1341700Z > Found 2 changed files.
2022-08-05T06:03:26.9984275Z {
2022-08-05T06:03:26.9985182Z   raw: '',
2022-08-05T06:03:26.9986195Z   remote: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:26.9986697Z   branches: [
2022-08-05T06:03:26.9987775Z     { name: 'another-branch', tracking: 'origin/another-branch' },
2022-08-05T06:03:26.9988218Z     {
2022-08-05T06:03:26.9988706Z       name: 'some-other-branch-01',
2022-08-05T06:03:27.0058350Z       tracking: 'origin/some-other-branch-01'
2022-08-05T06:03:27.0058647Z     },
2022-08-05T06:03:27.0058842Z     {
2022-08-05T06:03:27.0059165Z       name: 'some-other-branch-02',
2022-08-05T06:03:27.0059557Z       tracking: 'origin/some-other-branch-02'
2022-08-05T06:03:27.0059799Z     },
2022-08-05T06:03:27.0059988Z     {
2022-08-05T06:03:27.0060306Z       name: 'some-other-branch-03',
2022-08-05T06:03:27.0060708Z       tracking: 'origin/some-other-branch-03'
2022-08-05T06:03:27.0060961Z     }

============================ ≈ 350 lines trimmed (≈ 95 more branches) ============================

2022-08-05T06:03:27.0174817Z     {
2022-08-05T06:03:27.0175135Z       name: 'some-other-branch-98',
2022-08-05T06:03:27.0175531Z       tracking: 'origin/some-other-branch-98'
2022-08-05T06:03:27.0175783Z     },
2022-08-05T06:03:27.0175967Z     {
2022-08-05T06:03:27.0176294Z       name: 'some-other-branch-99',
2022-08-05T06:03:27.0176720Z       tracking: 'origin/some-other-branch-99'
2022-08-05T06:03:27.0176975Z     },
2022-08-05T06:03:27.0177162Z     ... 10 more items
2022-08-05T06:03:27.0177367Z   ],
2022-08-05T06:03:27.0177561Z   tags: []
2022-08-05T06:03:27.0177812Z }
2022-08-05T06:03:27.0178066Z > Pulling from remote...
2022-08-05T06:03:34.4661967Z {
2022-08-05T06:03:34.4663005Z   raw: '',
2022-08-05T06:03:34.4663820Z   remote: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:34.4709698Z   branches: [],
2022-08-05T06:03:34.4709925Z   tags: []
2022-08-05T06:03:34.4711665Z }
2022-08-05T06:03:34.8203641Z xt {
2022-08-05T06:03:34.8204521Z   remoteMessages: It { all: [] },
2022-08-05T06:03:34.8205425Z   created: [],
2022-08-05T06:03:34.8205835Z   deleted: [],
2022-08-05T06:03:34.8206162Z   files: [],
2022-08-05T06:03:34.8206500Z   deletions: {},
2022-08-05T06:03:34.8206907Z   insertions: {},
2022-08-05T06:03:34.8212049Z   summary: { changes: 0, deletions: 0, insertions: 0 }
2022-08-05T06:03:34.8212361Z }
2022-08-05T06:03:34.8212633Z > Checking for conflicts...
2022-08-05T06:03:34.8273839Z nr {
2022-08-05T06:03:34.8274410Z   not_added: [],
2022-08-05T06:03:34.8274803Z   conflicted: [],
2022-08-05T06:03:34.8275184Z   created: [],
2022-08-05T06:03:34.8275547Z   deleted: [],
2022-08-05T06:03:34.8275941Z   ignored: undefined,
2022-08-05T06:03:34.8276853Z   modified: [ 'file1.txt', 'file2.txt' ],
2022-08-05T06:03:34.8278160Z   renamed: [],
2022-08-05T06:03:34.8278557Z   files: [
2022-08-05T06:03:34.8278994Z     sr {
2022-08-05T06:03:34.8279632Z       path: 'file1.txt',
2022-08-05T06:03:34.8280111Z       index: 'M',
2022-08-05T06:03:34.8280603Z       working_dir: ' '
2022-08-05T06:03:34.8281244Z     },
2022-08-05T06:03:34.8281833Z     sr { path: 'file2.txt', index: 'M', working_dir: ' ' }
2022-08-05T06:03:34.8282098Z   ],
2022-08-05T06:03:34.8282570Z   staged: [ 'file1.txt', 'file2.txt' ],
2022-08-05T06:03:34.8282870Z   ahead: 0,
2022-08-05T06:03:34.8283078Z   behind: 0,
2022-08-05T06:03:34.8283385Z   current: 'selected-branch',
2022-08-05T06:03:34.8283767Z   tracking: 'origin/selected-branch',
2022-08-05T06:03:34.8284035Z   detached: false,
2022-08-05T06:03:34.8284270Z   isClean: [Function]
2022-08-05T06:03:34.8284499Z }
2022-08-05T06:03:34.8284730Z > No conflicts found.
2022-08-05T06:03:34.8285034Z > Re-staging files...
2022-08-05T06:03:34.8830473Z > Creating commit...
2022-08-05T06:03:34.8966565Z {
2022-08-05T06:03:34.8968001Z   author: null,
2022-08-05T06:03:34.8968763Z   branch: 'selected-branch',
2022-08-05T06:03:34.8969683Z   commit: '9c3d66591e8861166029c5549f7778755321b50c',
2022-08-05T06:03:34.8970167Z   root: false,
2022-08-05T06:03:34.8972559Z   summary: { changes: 2, insertions: 2, deletions: 0 }
2022-08-05T06:03:34.8973215Z }
2022-08-05T06:03:34.8974278Z 
2022-08-05T06:03:34.8974977Z 
2022-08-05T06:03:34.9009007Z 
2022-08-05T06:03:34.9012440Z > No tag info provided.
2022-08-05T06:03:34.9014303Z > Pushing commit to repo...
2022-08-05T06:03:42.3433112Z 
2022-08-05T06:03:42.3434666Z {
2022-08-05T06:03:42.3434945Z   pushed: [],
2022-08-05T06:03:42.3435157Z   update: {
2022-08-05T06:03:42.3435364Z     head: {
2022-08-05T06:03:42.3435749Z       local: 'refs/heads/selected-branch',
2022-08-05T06:03:42.3436150Z       remote: 'refs/heads/selected-branch'
2022-08-05T06:03:42.3436399Z     },
2022-08-05T06:03:42.3436777Z     hash: { from: 'd3adbe864c9eb', to: '9c3d66591e886' }
2022-08-05T06:03:42.3437026Z   },
2022-08-05T06:03:42.3437466Z   repo: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:42.3437988Z   ref: { local: 'refs/remotes/origin/selected-branch' },
2022-08-05T06:03:42.3438315Z   remoteMessages: It { all: [] }
2022-08-05T06:03:42.3438536Z }
2022-08-05T06:03:42.3438756Z > No tags to push.
2022-08-05T06:03:42.3443077Z ##[endgroup]
2022-08-05T06:03:42.3443378Z > Task completed.
2022-08-05T06:03:42.3445184Z ##[group]Outputs
2022-08-05T06:03:42.3445440Z committed: true
2022-08-05T06:03:42.3445802Z commit_long_sha: 9c3d66591e8861166029c5549f7778755321b50c
2022-08-05T06:03:42.3446181Z commit_sha: 9c3d66591e8861166029c5549f7778755321b50c
2022-08-05T06:03:42.3446443Z pushed: true
2022-08-05T06:03:42.3446663Z tagged: false
2022-08-05T06:03:42.3446901Z tag_pushed: false
2022-08-05T06:03:42.3447287Z ##[endgroup]

Total task duration was 6:26 in this case. As you can see from the logs, the problem is likely to do with syncing all branches, not just one. I have a rather ‘wide’ repo, i.e. 100+ branches with unrelated stuff in them. Syncing everything is what probably takes a while.

In the meantime, I’ve temporarily switched to:

      - name: Commit and push
        run: |
          git add --all \
           && git commit --message "Update seed data" \
           && git push \
           || true
        env:
          GIT_AUTHOR_EMAIL: github.actions@my-project.example.org
          GIT_AUTHOR_NAME: GitHub Actions
          GIT_COMMITTER_EMAIL: github.actions@my-project.example.org
          GIT_COMMITTER_NAME: GitHub Actions
        working-directory: some-cwd

Hope this info helps! 🙌

@EndBug
Copy link
Owner

EndBug commented Aug 20, 2022

Thanks for the logs! Yeah, it's definitely the fetch that's slowing down the action: I was able to reproduce this on a fork of the repo the author was having problems too.

I think I'll add an option to disable fetching so that large repos can use the action too. Disabling the fetch step will impact branch and tag creation, so I'll make sure to add some documentation in the README

EndBug added a commit that referenced this issue Aug 20, 2022
@EndBug EndBug added type: feature New feature or feature request and removed status: pending More info is needed before deciding what to do labels Aug 22, 2022
@EndBug EndBug self-assigned this Aug 22, 2022
@EndBug
Copy link
Owner

EndBug commented Aug 22, 2022

Alright, I've opened a PR (#423) with a new input that would let you customize or skip the git fetch command.
@kachkaev could you review it?

@kachkaev
Copy link

Great, thank @EndBug! The diff makes sense 👍

EndBug added a commit that referenced this issue Aug 22, 2022
* chore: add additional log notes

* feat: add `fetch` input

Ref #386

* fix: add warnings about not fetching

* docs: `fetch` input & large repos FAQ

* chore: fix typo
@EndBug
Copy link
Owner

EndBug commented Aug 22, 2022

@all-contributors please add @melink14 and @kachkaev for their bug report

@allcontributors
Copy link
Contributor

@EndBug

I've put up a pull request to add @melink14! 🎉

@EndBug
Copy link
Owner

EndBug commented Aug 22, 2022

@all-contributors please add @kachkaev for their bug report

@allcontributors
Copy link
Contributor

@EndBug

I've put up a pull request to add @kachkaev! 🎉

@EndBug
Copy link
Owner

EndBug commented Aug 22, 2022

Alright, I've just published the feature in v9.1.0 (also v9 and latest)
Thanks everyone 🎉

melink14 added a commit to melink14/rikaikun that referenced this issue Dec 4, 2022
…flow

- Migrates to new `set-output` syntax to avoid deprecation warning.
- Marks `fetch: false` when adding commits for significant speed up.
- Excludes /screenshots directory when commiting lint fixes. This avoid inadvertantly committing failed screenshot metadata when visual regression tests fail. (Requiring a manual revert)

See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
See EndBug/add-and-commit#386
mergify bot pushed a commit to melink14/rikaikun that referenced this issue Dec 5, 2022
…flow (#1306)

- Migrates to new `set-output` syntax to avoid deprecation warning.
- Marks `fetch: false` when adding commits for significant speed up.
- Excludes /screenshots directory when commiting lint fixes. This avoid inadvertantly committing failed screenshot metadata when visual regression tests fail. (Requiring a manual revert)

See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
See EndBug/add-and-commit#386
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: pinned Should not be labeled as stale type: feature New feature or feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants