Single container #210

mdellweg · 2020-04-21T09:34:36Z

No description provided.

templates/travis/.travis/start_container.yaml

templates/travis/.travis/install.sh.j2

mdellweg · 2020-04-22T13:33:20Z

templates/travis/.travis/settings.py.j2.copy

+AWS_S3_ENDPOINT_URL = "http://minio:9000"
+DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"
+AWS_DEFAULT_ACL = "@none None"
+{% endif %}


Please include the pulp_settings from the template config.

This wasn't as trivial as previously thought.
I added an ansible filter around pythons repr to make it work.

templates/travis/.travis/smash-config.json

mikedep333 · 2020-04-22T13:54:58Z

@mdellweg Per the meeting, give me a little more time to finish my review.

pep8speaks · 2020-04-22T16:20:37Z

Hello @mdellweg! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-04-28 12:36:37 UTC

templates/travis/.travis/Containerfile.ci_base

mikedep333 · 2020-04-23T14:29:52Z

templates/travis/.travis/install.sh.j2

+plugins:
+  - name: pulpcore
+    source: pulpcore{{ pulpcore_pip_version_specifier | default(omit, true) }}


FYI: Later on, we're going to need special logic for pulpcore. I suppose we could always change this later:
https://pulp.plan.io/issues/5864

mikedep333 · 2020-04-23T14:39:16Z

templates/travis/.travis/install.sh.j2

+plugins:
+  - name: pulpcore
+    source: pulpcore{{ pulpcore_pip_version_specifier | default(omit, true) }}
+  - name: {{ plugin_name }}
+    source: ./{{ plugin_name }}
+{%- for item in additional_plugins %}
+  - name: {{ item.name }}
+    source: {{ item.name }}{{ item.pip_version_specifier | default(omit, true) }}


So we currently have 3 different data structures for our list of plugins to install across our installers. The addition of the 2nd & 3rd are my fault, but we should still try to consolidate them, even if certain args are ignored in different installers.

pulp_installer pulp_install_plugins

pulpcore.git/containers plugins

template_config.yml additional_plugins

That said, I do approve of your design though, because it is based on additional_plugins.

mikedep333 · 2020-04-23T14:47:27Z

templates/travis/.travis/before_install.sh.j2

+{%- for repository in ["pulpcore", "pulp-smash", "pulp-openapi-generator"] + additional_plugins | map(attribute="name") | list %}
+  export {{ repository | upper | replace("-", "_") }}_PR_NUMBER=$(echo $COMMIT_MSG | grep -oP 'Required\ PR:\ https\:\/\/github\.com\/pulp\/{{ repository }}\/pull\/(\d+)' | awk -F'/' '{print $7}')
+{%- endfor %}
 else
-  export PULP_PR_NUMBER=
-  export PULP_SMASH_PR_NUMBER=
-  export PULP_ROLES_PR_NUMBER=
-  export PULP_BINDINGS_PR_NUMBER=
-  export PULP_OPERATOR_PR_NUMBER=
+  {%- for repository in ["pulpcore", "pulp-smash", "pulp-openapi-generator"] + additional_plugins | map(attribute="name") | list %}
+  export {{ repository | upper | replace("-", "_") }}_PR_NUMBER=
+{%- endfor %}


Nice! I was just brainstorming ways to make this more generic for https://pulp.plan.io/issues/6557

mikedep333 · 2020-04-23T15:00:19Z

templates/travis/.travis/before_install.sh.j2

+# Install ansible with the boto3 tags to dict fix
+pip install git+https://github.com/mdellweg/ansible.git@fix_boto3_tags_dict


Do we have a PR against ansible?

I looked for one.

It also looks like ec2.py is no longer in the repo as of 2.10 devel (moved to a partner collection?)

Yeah, i did not find that file either in master. My guess is the humongous community-collection.
But actually i am unsure, whether the root problem here is with ansible, or minio.

I see what you mean.

I found the repo for Ansible 2.10 (git log -- ./deleted/file/path), the line is not fixed:
https://github.com/ansible-collections/amazon.aws/blob/master/plugins/module_utils/ec2.py#L458

It is published on Galaxy here, per Ansible's plan to separate partner modules from ansible core:

Could you please try reporting it to 1 or the other?

Will do. Thank you for finding it.

ansible-collections/amazon.aws#37

Great. Please do 1 of the following so we don't end up with this forever:

Put a comment above it with a link to the PR.

Create a task in Pulp's redmine to make sure the PR gets accepted there (or in minio if they refuse.)

Comment in place. I mentioned the move of the code to the collection, because this might be interesting anyway, when using the next version of ansible.

templates/travis/.travis/before_script.sh.j2

mikedep333

👏

fixes #6477 https://pulp.plan.io/issues/6477

mdellweg force-pushed the single_container branch from 2adb4b9 to 7ffb9c4 Compare April 21, 2020 12:04

mdellweg mentioned this pull request Apr 21, 2020

Single container pulp/pulpcore#660

Merged

dkliban approved these changes Apr 22, 2020

View reviewed changes

mdellweg commented Apr 22, 2020

View reviewed changes

templates/travis/.travis/start_container.yaml Outdated Show resolved Hide resolved

mdellweg commented Apr 22, 2020

View reviewed changes

templates/travis/.travis/install.sh.j2 Outdated Show resolved Hide resolved

mdellweg commented Apr 22, 2020

View reviewed changes

templates/travis/.travis/smash-config.json Outdated Show resolved Hide resolved

mdellweg force-pushed the single_container branch from 210f72d to cd3079e Compare April 22, 2020 16:20

mdellweg force-pushed the single_container branch from cd3079e to 31495d4 Compare April 22, 2020 16:25

mikedep333 requested changes Apr 23, 2020

View reviewed changes

mdellweg force-pushed the single_container branch 4 times, most recently from b86eec4 to 9dd39a6 Compare April 27, 2020 11:37

mikedep333 approved these changes Apr 27, 2020

View reviewed changes

mdellweg added 2 commits April 28, 2020 14:35

Allow to copy templates

a8ce72a

Transform travis script to run single container

cb86c09

fixes #6477 https://pulp.plan.io/issues/6477

mdellweg force-pushed the single_container branch from 9dd39a6 to cb86c09 Compare April 28, 2020 12:36

mdellweg merged commit 40f158d into pulp:master Apr 28, 2020

mdellweg deleted the single_container branch April 28, 2020 15:34

Single container #210

Single container #210

mdellweg commented Apr 21, 2020

mdellweg Apr 22, 2020

mdellweg Apr 22, 2020

mikedep333 commented Apr 22, 2020

pep8speaks commented Apr 22, 2020 •

edited

mikedep333 Apr 23, 2020

mikedep333 Apr 23, 2020

mikedep333 Apr 23, 2020

mikedep333 Apr 23, 2020

mdellweg Apr 23, 2020

mikedep333 Apr 23, 2020

mdellweg Apr 24, 2020

mdellweg Apr 24, 2020

mikedep333 Apr 27, 2020 •

edited

mdellweg Apr 28, 2020

mikedep333 left a comment

		# Install ansible with the boto3 tags to dict fix
		pip install git+https://github.com/mdellweg/ansible.git@fix_boto3_tags_dict

Single container #210

Single container #210

Conversation

mdellweg commented Apr 21, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedep333 commented Apr 22, 2020

pep8speaks commented Apr 22, 2020 • edited

Comment last updated at 2020-04-28 12:36:37 UTC

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedep333 Apr 27, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedep333 left a comment

Choose a reason for hiding this comment

pep8speaks commented Apr 22, 2020 •

edited

mikedep333 Apr 27, 2020 •

edited