Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use one upload to script #499

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

roxell
Copy link
Collaborator

@roxell roxell commented Mar 1, 2024

Use one upload script for both artifactorial and for squad.

# SQUAD_ARCHIVE_SUBMIT_TOKEN is used for uploading authentication,
# and must be defined by the submitter as one profile managed token
SQUAD_UPLOAD_URL: ""
UPLOAD_URL: "https://archive.validation.linaro.org/artifacts/team/qa/"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is related to the interface change, it would be better to support all of them for a while and drop them after that period.
Like the current status, there are jobs using ARTIFACTORIAL_URL(old jobs not working at the moment) and SQUAD_UPLOAD_URL(work now), and then will submit jobs to support "UPLOAD_URL" if we have this PR merged right now, then the jobs with SQUAD_UPLOAD_URL will fail, and it needs to resubmit them again. if both SQUAD_UPLOAD_URL and UPLOAD_URL are supported here, then the current jobs will be able to run, and it's possible to submit new jobs with UPLOAD_URL as well. ARTIFACTORIAL_URL support will be the similar.

@@ -45,7 +42,6 @@ run:
# Upload test output to artifactorial.
- cp "./output/${TEST_NAME}/result.txt" "./output/result.txt"
- tar caf "output-${TEST_NAME}.tar.xz" "./output"
- if [ -n "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-squad.sh -a "output-${TEST_NAME}.tar.xz" -u "${SQUAD_UPLOAD_URL}"; fi
- if [ -z "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-artifactorial.sh -a "output-${TEST_NAME}.tar.xz" -u "${ARTIFACTORIAL_URL}" -t "${ARTIFACTORIAL_TOKEN}"; fi
- ../../utils/upload-artifacts.sh -a "output-${TEST_NAME}.tar.xz" -u "${UPLOAD_URL}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to support my concern, there could be done like this:

- if [ -n "${UPLOAD_URL}" ]; ../../utils/upload-artifacts.sh -a "output-${TEST_NAME}.tar.xz" -u "${UPLOAD_URL}"; fi
- if [ -n "${SQUAD_UPLOAD_URL}" ]; ../../utils/upload-artifacts.sh -a "output-${TEST_NAME}.tar.xz" -u "${SQUAD_UPLOAD_URL}"; fi

And after like several weeks or a month, drop the SQUAD_UPLOAD_URL line when no SQUAD_UPLOAD_URL jobs will be run.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense, to keep it around for a while. Good catch.

@mwasilew
Copy link
Contributor

mwasilew commented Mar 7, 2024

LGTM. I share @liuyq worries about backward compatibility. As I'm not using any uploads right now I'll leave this to you to decide.

Add support for upload to squad in upload-to-artifactoral.sh and create
a symlink upload-to-squad.sh to upload-to-artifactorial.sh.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Explaining that the artifactoral token should be set by the submitters
LAVA profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Explaining that the artifactoral token should be set by the submitters
LAVA profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Explaining that the artifactoral token should be set by the submitters
LAVA profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add a more generic script name for uploading artifacts to artifactorial
or SQUAD.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
@@ -70,8 +70,8 @@ run:
- sudo dmesg > ./output/dmesg-host.txt || true
- if ! tar caf tradefed-output-$(date +%Y%m%d%H%M%S).tar.xz ./output; then error_fatal "tradefed - failed to collect results and log files [$ANDROID_SERIAL]"; fi
- ATTACHMENT=$(ls tradefed-output-*.tar.xz)
- if [ -n "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-squad.sh -a "${ATTACHMENT}" -u "${SQUAD_UPLOAD_URL}"; fi
- if [ -z "${SQUAD_UPLOAD_URL}" ]; then ../../utils/upload-to-artifactorial.sh -a "${ATTACHMENT}" -u "${URL}" -t "${TOKEN}"; fi
- if [ -n "${SQUAD_UPLOAD_URL}" ]; then UPLOAD_URL="${SQUAD_UPLOAD_URL}" fi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not quite sure, but I think it's this line that has the syntax error, like reported by the lava job here:
https://lkft.validation.linaro.org/scheduler/job/7371910#L8593

the right one should be like this:
if [ -n "${SQUAD_UPLOAD_URL}" ]; then UPLOAD_URL="${SQUAD_UPLOAD_URL}"; fi

Use script upload-artifacts.sh, just have to specify the correct
UPLOAD_URL and set the correct TOKEN in the submitters profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Use script upload-artifacts.sh, just have to specify the correct
UPLOAD_URL and set the correct TOKEN in the submitters profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Use script upload-artifacts.sh, just have to specify the correct
UPLOAD_URL and set the correct TOKEN in the submitters profile.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
No need to keep symlink upload-to-squad.sh, upload-artifacts.sh should
be used instead.

Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants