Skip to content

Deploy metadata in end to end tests#2186

Merged
bert-e merged 4 commits intodevelopment/2.11from
improvement/ZENKO-4414
Feb 20, 2025
Merged

Deploy metadata in end to end tests#2186
bert-e merged 4 commits intodevelopment/2.11from
improvement/ZENKO-4414

Conversation

@welansari
Copy link
Copy Markdown

@welansari welansari commented Jan 21, 2025

  • Deploy metadata from chart
  • Wait for metadata components to start in order : repd -> bucketd -> cloudserver
  • Manually add the proper endpoint in Cloudserver config
  • Only deploy metadata in end2end-sharded

Issue: ZENKO-4414

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jan 21, 2025

Hello kerkesni,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@welansari welansari force-pushed the improvement/ZENKO-4414 branch 3 times, most recently from e03ffd7 to 15d54c7 Compare January 22, 2025 11:41
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jan 22, 2025

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@welansari welansari force-pushed the improvement/ZENKO-4414 branch 9 times, most recently from 2468755 to ac95b9c Compare January 23, 2025 17:28
@welansari welansari marked this pull request as ready for review January 24, 2025 15:15
@welansari welansari force-pushed the improvement/ZENKO-4414 branch 3 times, most recently from 0efddbf to 9055c5f Compare January 24, 2025 16:06
@francoisferrand francoisferrand changed the base branch from improvement/ZENKO-4966-1.29 to development/2.10 January 28, 2025 11:32
@welansari welansari force-pushed the improvement/ZENKO-4414 branch 3 times, most recently from 8e060ea to 5a84228 Compare January 28, 2025 11:55
# wait for all bucketd pods to start serving port 9000
wait_for_all_pods_behind_services metadata-bucketd metadata 9000 60

# manually add "s3c.local" to the rest endpoints list as it's not configurable in the chart
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need s3c.local ?

i did not need such patch on my artesca cluster (in namespace default i think), just installed the chart and created the artesca location pointed at s3c-cloudserver.default.svc.cluster.local

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloudserver responds with an error when the host used is not declared in its config file. I've tried with the k8s service endpoint and it doesn't work.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this comes from the namespace ("metadata") you deploy to, which may not be taken into account?
in my tests this is using the default namespace, with no issue...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No even with the default namespace i got the same behaviour, which to me, is the expected behaviour. The normal cloudserver we deploy in Zenko has all the k8s endpoints in its config.

@welansari welansari force-pushed the improvement/ZENKO-4414 branch from 5a84228 to f6d88ec Compare February 10, 2025 17:16
Kerkesni added 3 commits February 11, 2025 10:17
Metadata components need to be started in the order
repd -> bucketd -> cloudserver
as they are not able to recover on their own when a
component they depend on is not available at startup.

Issue: ZENKO-4414
Configuration of the test environement is done through an image built in the build-test-image job,
this image contains the list of location details used to initialize the locations. This list isn't
updated when only retrying the end to end test jobs, and the bucket is create using the env var passed
to the pod. So we endup creating the bucket with the updated retry count but configure the location
with the old bucket name which fails as pensieve won't be able to find that bucket.

Issue: ZENKO-4414
@welansari welansari force-pushed the improvement/ZENKO-4414 branch 2 times, most recently from 0707209 to 0deadb5 Compare February 11, 2025 11:41
@welansari welansari force-pushed the improvement/ZENKO-4414 branch from fc13857 to 0deadb5 Compare February 19, 2025 10:47
@francoisferrand francoisferrand changed the base branch from development/2.10 to development/2.11 February 20, 2025 13:51
@scality scality deleted a comment from bert-e Feb 20, 2025
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Feb 20, 2025

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@scality scality deleted a comment from bert-e Feb 20, 2025
@scality scality deleted a comment from bert-e Feb 20, 2025
@scality scality deleted a comment from bert-e Feb 20, 2025
@welansari
Copy link
Copy Markdown
Author

/approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Feb 20, 2025

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/2.11

The following branches will NOT be impacted:

  • development/2.10
  • development/2.5
  • development/2.6
  • development/2.7
  • development/2.8
  • development/2.9

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Feb 20, 2025

Queue build failed

The corresponding build for the queue failed:

  • Checkout the status page.
  • Identify the failing build and review the logs.
  • If no issue is found, re-run the build.
  • If an issue is identified, checkout the steps below to remove
    the pull request from the queue for further analysis and maybe rebase/merge.
Remove the pull request from the queue
  • Add a /wait comment on this pull request.
  • Click on login on the status page.
  • Go into the manage page.
  • Find the option called Rebuild the queue and click on it.
    Bert-E will loop again on all pull requests to put the valid ones
    in the queue again, while skipping the one with the /wait comment.
  • Wait for the new queue to merge, then merge/rebase your pull request
    with the latest changes to then work on a proper fix.
  • Once the issue is fixed, delete the /wait comment and
    follow the usual process to merge the pull request.

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Feb 20, 2025

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/2.11

The following branches have NOT changed:

  • development/2.10
  • development/2.5
  • development/2.6
  • development/2.7
  • development/2.8
  • development/2.9

Please check the status of the associated issue ZENKO-4414.

Goodbye kerkesni.

@bert-e bert-e merged commit a88ec9a into development/2.11 Feb 20, 2025
28 checks passed
@bert-e bert-e deleted the improvement/ZENKO-4414 branch February 20, 2025 16:20
@bert-e bert-e restored the improvement/ZENKO-4414 branch February 20, 2025 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants