Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-17293: [Java][CI] Prune java nightly builds #13839

Merged
merged 5 commits into from Aug 30, 2022

Conversation

davisusanibar
Copy link
Contributor

Prune java nightly builds

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

env:
KEEP: ${{ github.event.inputs.keep || 14 }}
run: |
for i in `ls -t repo/org/apache/arrow`; do; find repo/org/apache/arrow/$i -mindepth 1 -maxdepth 1 -type d -print0 | xargs -0 ls -t -d | tail -n +$((KEEP + 1)) | xargs rm -rf; done
- name: Show repo contents
run: tree repo
- name: Upload Files
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need to update this step with the new action and switches otherwise the locally deleted files wont be deleted from the server.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @assignUser , just added

- shell: bash
name: Show local repo sync from remote
run: |
for i in `ls -t repo/org/apache/arrow`; do; echo "- $i: $(find repo/org/apache/arrow/$i -mindepth 1 -maxdepth 1 -type d | wc -l | xargs) versions available"; done
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use multiple lines for readability?

for i in ...; do
  echo ...
done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also change this line too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

for i in `ls -t repo/org/apache/arrow`; do
echo "- $i: $(find repo/org/apache/arrow/$i -mindepth 1 -maxdepth 1 -type d \
| wc -l \
| xargs) versions available"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this | xargs needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xarg in this case help us with one-line string trimming report about number of version available:

- flight-sql:        3 versions available
- arrow-vector:        7 versions available
- flight-grpc:        4 versions available

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks.

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou kou merged commit d7eea09 into apache:master Aug 30, 2022
@ursabot
Copy link

ursabot commented Aug 30, 2022

Benchmark runs are scheduled for baseline = 15d6959 and contender = d7eea09. d7eea09 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Failed] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] d7eea09b ec2-t3-xlarge-us-east-2
[Failed] d7eea09b test-mac-arm
[Failed] d7eea09b ursa-i9-9960x
[Failed] d7eea09b ursa-thinkcentre-m75q
[Finished] 15d69596 ec2-t3-xlarge-us-east-2
[Failed] 15d69596 test-mac-arm
[Failed] 15d69596 ursa-i9-9960x
[Finished] 15d69596 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Aug 30, 2022

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

anjakefala pushed a commit to anjakefala/arrow that referenced this pull request Aug 31, 2022
Prune java nightly builds

Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
Prune java nightly builds

Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
fatemehp pushed a commit to fatemehp/arrow that referenced this pull request Oct 17, 2022
Prune java nightly builds

Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants