Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade tests #503

Merged
merged 1 commit into from
Jun 1, 2021
Merged

Upgrade tests #503

merged 1 commit into from
Jun 1, 2021

Conversation

fao89
Copy link
Member

@fao89 fao89 commented May 7, 2021

@pulpbot
Copy link
Member

pulpbot commented May 7, 2021

Attached issue: https://pulp.plan.io/issues/8719

@fao89 fao89 changed the title Upgrade tests WIP - Upgrade tests May 7, 2021
@fao89 fao89 force-pushed the 8719 branch 27 times, most recently from 7baf0b2 to 2bb5447 Compare May 11, 2021 15:30
@fao89 fao89 force-pushed the 8719 branch 8 times, most recently from 76e7b4f to d3e084b Compare May 11, 2021 20:02
@fao89 fao89 changed the title WIP - Upgrade tests Upgrade tests May 11, 2021
Copy link
Member Author

@fao89 fao89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Main challenges:

  • how to deal with additional_plugins and its old dependencies?
  • how to deal with external commands or specific versions of the CI scripts?
  • how to deal with dependencies? e.g. django

Comment on lines 139 to 145
include:
- FROM_BRANCH: 1.4
FROM_PULPCORE_BRANCH: "3.9"
- FROM_BRANCH: 1.5
FROM_PULPCORE_BRANCH: "3.10"
- FROM_BRANCH: 1.6
FROM_PULPCORE_BRANCH: "3.11"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be built according to template_config.yml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can be a challenge when we have additional_plugins, should we declare each additional plugin dependency?

Comment on lines 166 to 163
echo "FROM_BRANCH=${{ matrix.FROM_BRANCH }}" >> $GITHUB_ENV
echo "FROM_PULPCORE_BRANCH=${{ matrix.FROM_PULPCORE_BRANCH }}" >> $GITHUB_ENV
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can be a challenge when we have additional_plugins, should we declare each additional plugin dependency?

Comment on lines 30 to 40
if [[ "$TEST" == "upgrade" ]]; then
git checkout -b ci_upgrade_test
cp -R .github /tmp/.github
cp -R .ci /tmp/.ci
git checkout $FROM_BRANCH
rm -rf .ci .github
cp -R /tmp/.github .
cp -R /tmp/.ci .
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to checkout old branches, but we need to keep the current CI scripts

Comment on lines +110 to +123
sed -i "s/true/false/g" tests/settings.toml
mkdir ~/.config/pulp
cp tests/settings.toml ~/.config/pulp/settings.toml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not so familiar with pulp-cli, but I did this because I wanted to use the CLI to expose we still have data after migration

Comment on lines 126 to 127
# Pin pulpcore deps
sed -i "s/~/=/g" requirements.txt
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dealing with dependencies from previous versions is challenging, on this case I did this hack for pinning django

Comment on lines +109 to +125
cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-api"
cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-content"
cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-resource-manager"
cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-worker@1"
cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-worker@2"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restarting services

cmd_prefix bash -c "s6-svc -r /var/run/s6/services/pulpcore-worker@2"

echo "Restarting in 60 seconds"
sleep 60
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to wait services to start before making requests

echo "Restarting in 60 seconds"
sleep 60

pulp status
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for verifying if it indeed upgraded to current version

sleep 60

pulp status
pulp file content list
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for exposing if the migration was successful and we still having data

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for pulp_file 1.5 it returned an empty list, I believe it was because of that migration that needed an external command

Comment on lines +121 to +142
cd ../pulp-openapi-generator
./generate.sh pulpcore python
pip install ./pulpcore-client
./generate.sh pulp_file python
pip install ./pulp_file-client
./generate.sh pulp_certguard python
pip install ./pulp_certguard-client
cd $REPO_ROOT
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to rebuild bindings after upgrading

@fao89 fao89 force-pushed the 8719 branch 8 times, most recently from 9f7b6bd to eddf1b1 Compare May 19, 2021 14:07
@fao89 fao89 marked this pull request as ready for review May 19, 2021 17:52
@fao89
Copy link
Member Author

fao89 commented May 20, 2021

@pulp/core it is ready for review now!

@fao89
Copy link
Member Author

fao89 commented May 25, 2021

@pulp/core can I have a review?

Copy link
Member

@bmbouter bmbouter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also looks good. Thanks @fao89 !

@dkliban dkliban merged commit e75111f into pulp:master Jun 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants