Skip to content
This repository has been archived by the owner on Dec 7, 2022. It is now read-only.
/ devel Public archive

use pulp-manager to reset db #91

Merged
merged 2 commits into from
Sep 22, 2017
Merged

use pulp-manager to reset db #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
257274e
Remove the use of db-reset script
asmacdo Sep 15, 2017
65f86c3
Skip plugin install if repo DNE
asmacdo Sep 22, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions ansible/library/pulp_facts.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,6 @@ def main():
# and to make it clear that those facts were provided by this module.
facts = {'pulp_available_repositories': available_repositories(module)}

# filter the plugins list against the available repos to get available plugins
facts['pulp_available_plugins'] = list(filter(lambda repo: repo in plugins,
facts['pulp_available_repositories']))

# filter the requirements list against the available repos, removing unavailable repos
facts['pulp_requirements_files'] = {
k: v for k, v in requirements_files.items() if k in facts['pulp_available_repositories']
Expand Down
4 changes: 2 additions & 2 deletions ansible/pulp-from-source.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
- broker
- lazy
- dev
- plugins
- reset_db
- {role: plugin, plugin_name: 'pulp_file', app_label: 'pulp_file'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey so I found an issue with this. If you provide the names of the plugins up-front instead of letting them be enumerated, this fails if the plugin isn't in the user's devel directory.

Specifically, I forgot to clone the pulp_file plugin, and the ansible script failed out with

TASK [plugin : Install Plugin packages into core Pulp Python environment] ******
fatal: [pulp3_dev]: FAILED! => {"changed": false, "cmd": "/home/vagrant/.virtualenvs/pulp/bin/pip3 install -e /home/vagrant/devel/pulp_file", "failed": true, "msg": "\n:stderr: /home/vagrant/devel/pulp_file should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+\nYou are using pip version 8.1.2, however version 9.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n"}

I assume we want to keep the previous workflow where you don't necessarily need to have the plugins cloned - but let me know if that assumption is invalid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have 2 choices. We can use the task to assert that the repo exists:
https://github.com/pulp/devel/blob/3.0-dev/ansible/roles/common/tasks/require_repo.yml

Or we can leave this role commented out as the default. I'm inclined to go that route. Pretty sure you could clone, uncomment and run vagrant provision.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of letting the presence of pulp_file be optional, which would mean having this line of code commented out by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair, although it means that devs would need to uncomment/list all plugins they want to be installed themselves, which is probably fine at this point in the Pulp 3 lifecycle. I feel like we would eventually want to go back to the old model installing everything it could find, but maybe now isn't the time to worry about it.

Thoughts?

Copy link

@Ichimonji10 Ichimonji10 Sep 21, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True enough. I do like the idea of making plugin installation as seamless and easy as possible. We both want the same end goal.

I want to get to a place where we can install pulpcore (with development extras) with a single button press as soon as possible. This will be much easier to do if we start by putting a pulpcore-only playbook in front of the user by default.

EDIT: One specific possible solution is to create a role (or roles) for installing plugins, and to make that role(s) call this role after the plugin(s) is installed. That would place the logic for installing the plugin near the logic for running that plugin's migrations. Regardless of how exactly it happens, the end procedure is the same: get one-button pulpcore installations working now, and get one-button plugin installations working later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long story short, we were already hardcoding pulp_file into the ansible install, so for now, I think its best to move that out of the role and into the playbook.

Id like to comment out the pulp_file install in the default playbook and use require_repo to fail with an easily readable message. (Probably also a small docs change). That seems like the safest default to me, and we can continue this conversation on the pulp-dev list to align our longer term ideas.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me. Let me know when those changes have been made and I will take another look

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long story short, we were already hardcoding pulp_file into the ansible install, so for now, I think its best to move that out of the role and into the playbook.

This step?

TASK [plugins : Install Plugin packages into core Pulp Python environment] ********************************************
 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ [item] | issubset(pulp_available_plugins) }}                        

skipping: [fedora-25-pulp3-pypi] => (item=pulp_file)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ichimonji10, yes, and that part also depends on a hardcoded list in pulp_facts. I prefer that if someone wants to modify their dev environment to install their new plugin, they should only need to change the playbook.

# - {role: plugin, plugin_name: 'pulp_example', app_label: 'pulp_example'}
- systemd
# - pulp_ca
# - smash
Expand Down
5 changes: 4 additions & 1 deletion ansible/roles/dev/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,4 +101,7 @@
state: directory
owner: apache
group: apache
mode: 0755
mode: 0755

- include: roles/django_db/tasks/makemigrations.yml app_label=pulp_app
- include: roles/django_db/tasks/reset.yml
3 changes: 3 additions & 0 deletions ansible/roles/django_db/tasks/makemigrations.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- name: makemigrations
command: "{{pulp_venv}}/bin/pulp-manager makemigrations {{app_label}}"
become: false
7 changes: 7 additions & 0 deletions ansible/roles/django_db/tasks/reset.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
- name: migrate
command: "{{pulp_venv}}/bin/pulp-manager {{item}}"
with_items:
- "reset_db --noinput"
- "migrate auth --noinput"
- "migrate --noinput"
become: false
20 changes: 20 additions & 0 deletions ansible/roles/plugin/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
- stat:
path: "{{ pulp_devel_dir }}/{{ plugin_name }}"
register: repo_path

- debug:
msg: "{{ pulp_devel_dir }}/{{ plugin_name }} does not exist, skipping!"
when: repo_path.stat.isdir is not defined or not repo_path.stat.isdir

- block:
- name: Install Plugin packages into core Pulp Python environment
pip:
name: "{{ pulp_devel_dir }}/{{ plugin_name }}"
extra_args: "-e"
virtualenv: "{{ pulp_venv }}"
become: false

# variables plugin_name and app_label are passed implicitly
- include: roles/django_db/tasks/makemigrations.yml
- include: roles/django_db/tasks/reset.yml
when: repo_path.stat.isdir is defined and repo_path.stat.isdir
13 changes: 0 additions & 13 deletions ansible/roles/plugins/tasks/main.yml
View file
Open in desktop

This file was deleted.

4 changes: 3 additions & 1 deletion ansible/roles/pulp-user/templates/alias.bashrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ preset() {

pclean() {
workon pulp
platform/pulpcore/app/db-reset.sh
pulp-manager reset_db --noinput
pulp-manager migrate auth --noinput
pulp-manager migrate
}
_pclean_help="Restore pulp to a clean-installed state"
# can get away with not resetting terminal settings here since it gets reset in phelp
Expand Down
2 changes: 0 additions & 2 deletions ansible/roles/reset_db/tasks/main.yml
View file
Open in desktop

This file was deleted.