#2637 and #878 #18

mibanescu · 2017-04-06T14:19:51Z

No description provided.

… in new sync

… mirror repo sync code

Change-Id: Ia3b9c10decf840b39ffbdb1f25e37217010d4354

Change-Id: I8892fad1d5eef784740c6cb6a94469b9502befab Sem-Ver: feature

Change-Id: I6615fdd60070bd1fb97faf0a0bb0f289b578510c

There are limitations: only one component/arch can be synced at this time. Change-Id: I54671d265143121b42fb5517aa15bfe400c2f9fe

closes #2637 Change-Id: Id5eb5bd0f4cef304688824b301f1a94c72372281

pulpbot · 2017-04-06T14:19:59Z

Can one of the admins verify this patch?

pulpbot · 2017-04-06T14:20:01Z

Can one of the admins verify this patch?

bmbouter · 2017-04-07T18:59:08Z

extensions_admin/pulp_deb/extensions/admin/contents.py

@@ -0,0 +1,143 @@
+# Copyright (c) 2012 Red Hat, Inc.
+#
+# This software is licensed to you under the GNU General Public


FYI we've been removing these header license blocks from our files. We've been informed that having one license file at the top is sufficient.

Not a problem - I will need to do any suggested changes as a different commit, since we use git internally and it won't let me force push. Just FYI :-)

Cool that makes perfect sense. Additional commits are no problem.

bmbouter · 2017-04-07T19:47:20Z

plugins/pulp_deb/plugins/db/models.py

+    REL_FIELDS = ['breaks', 'conflicts', 'depends', 'enhances', 'pre_depends',
+                  'provides', 'recommends', 'replaces', 'suggests']
+
+    breaks = mongoengine.DynamicField()


Is there any way these DynamicFields could become more strongly typed? The strong data validation we get with Mongoengine isn't as strong when these fields can take on a variety of data types. I don't know enough about what is in them, but I wanted to ask.

The way I've chosen to represent the deps, they are either a list of dict or a list of (list of dict) (in the case of a dependency like foo OR bar).
I believe I tried ListField and it failed for a different reason - I had a duplicated field. If you would rather have it as a ListField, I can do that, as long as each item of the list can be either a dict or a list of dicts.

bmbouter · 2017-04-07T19:49:34Z

ok test

bmbouter · 2017-04-07T19:57:45Z

Our Jenkins results are not yet viewable by the community but @dkliban is actively working on that. For now I can say that the PR test runner is failing early on with:

19:53:04 Traceback (most recent call last):
19:53:04   File "./run-tests.py", line 8, in <module>
19:53:04     from pulp.devel.test_runner import run_tests

How are you running your tests locally?

mibanescu · 2017-04-10T13:57:26Z

Everything has been coded against pulp 2.10.3 (I believe I copied run-tests.py from pulp_rpm from the 2.10-release branch). However, I see that import line to be the same in master. Am I missing a dependency to pull in pulp.devel?

dkliban · 2017-04-11T21:00:27Z

ok test

dkliban

You need to bring back pulp-dev.py so that this plugin can be installed for development. This script is also used by Jenkins when running unit tests.

Also updated test_requirements.txt and dist_list.txt Change-Id: I195703d7bd61eab38f95d2e6a7955e5a92aa6fa7

Change-Id: Ic89510ba9c279158c0a94e5042c846df5882ae0f

dkliban · 2017-04-12T16:02:11Z

ok test

dkliban · 2017-04-12T18:27:23Z

pulp-deb.spec

 # ---- Plugins -----------------------------------------------------------------
 %package plugins
 Summary: Pulp Debian plugins
 Group: Development/Languages
-Requires: python-pulp-common >= %{pulp_version}
-Requires: python-pulp-deb-common >= %{version}
-Requires: pulp-server >= %{pulp_version}


You need to keep the version of pulp-server specified so that the plugin is installed with a new enough version of the platform. The unit test jenkins job also relies on this information in the spec file to know which platform bits to install before testing the plugin.

dkliban · 2017-04-12T19:44:25Z

ok test

mibanescu · 2017-04-27T15:47:48Z

ok test

mibanescu · 2017-04-27T21:44:38Z

ok test

mibanescu · 2017-04-27T21:59:14Z

ok test

mibanescu · 2017-04-27T22:09:50Z

ok test

bmbouter · 2017-04-27T22:46:58Z

When you run the tests locally do you also experience the error?

dkliban · 2017-05-02T20:39:55Z

plugins/test/unit/plugins/distributors/test_distributor.py

+    @mock.patch("pulp_deb.plugins.distributors.distributor.aptrepo.debpkg.debfile.DebFile")
+    @mock.patch("pulp.server.managers.repo._common.task.current")
+    @mock.patch('pulp.plugins.util.publish_step.repo_controller')
+    def test_publish_repo(self, _repo_controller, _task_current, _DebFile,


This is a very elaborate test. I was hoping to find an example in the RPM plugin that would help mocking the SELinux restorecon statement, but we don't have one.

Patching the restorecon statement should get you further.

@mock.patch('pulp.plugins.util.publish_step.selinux.restorecon')

mibanescu · 2017-05-12T20:06:40Z

ok test

mibanescu · 2017-05-12T20:34:23Z

The tests succeeded on RHEL7. Not sure how to make them run on f25 (I don't think pulp 2.10 was running on it) or f24 (I think it's not finding the repo, for some reason).

dkliban · 2017-05-12T22:48:29Z

@mibanescu This is so great! @goosemania has already filed an issue for the problem you are experiencing with the unit tests. https://pulp.plan.io/issues/2751 We hope to address it soon. You can consider the unit tests as passing at this point.

bmbouter · 2017-05-15T17:33:30Z

extensions_admin/pulp_deb/extensions/admin/copy_commands.py

@@ -0,0 +1,93 @@
+# This software is licensed to you under the GNU General Public


This license block can be removed.

bmbouter · 2017-05-15T17:34:01Z

extensions_admin/pulp_deb/extensions/admin/deb_repo/pulp_cli.py

@@ -0,0 +1,102 @@
+# -*- coding: utf-8 -*-
+#
+# This software is licensed to you under the GNU General Public


This license block can be removed.

bmbouter · 2017-05-15T17:34:30Z

extensions_admin/pulp_deb/extensions/admin/remove.py

@@ -0,0 +1,53 @@
+# This software is licensed to you under the GNU General Public


This license block can be removed.

bmbouter · 2017-05-15T17:35:38Z

extensions_admin/pulp_deb/extensions/admin/structure.py

@@ -0,0 +1,200 @@
+# This software is licensed to you under the GNU General Public


All the license blocks can be removed. Since these are all the same I'll stop leaving the same comment over and over.

bmbouter · 2017-05-15T18:06:23Z

dist_list.txt

@@ -1 +1 @@
-el6 el7 fc21 fc22
+el6 el7 fc24


Can this be updated to the current support matrix? I believe that would be el7 fc24 fc25.

mibanescu · 2017-05-15T19:32:33Z

ok test

…nstalls pulp-dev.py can now install packages via pip test/setup.py added just so pip can be installed. pulp_rpm is no longer a dependency. Removed copyright statements. Change-Id: I8b6954f62fa635335a57f4fedaeaa543b45f16f3

mibanescu · 2017-05-15T20:01:31Z

ok test

mibanescu · 2017-05-16T21:23:03Z

What are the next steps for this PR?

bmbouter

@mibanescu Thank you so much for this PR. You've implemented all of the changes I requested. The unit tests are all passing. The last thing we need is a sign off from @dkliban.

dkliban

Thank you!

mibanescu · 2017-05-17T20:50:25Z

Thank you!

sjtindell and others added 13 commits May 10, 2016 23:41

Added content unit model for mongodb

7a1a3a2

Updated admin extension with additional download-root option

76f8d4e

Updated httpd conf, removed SSL options

eff1fa2

Updated importer sync to use model for unit

95b837d

Updated distributor to increment save-units sucess

83ca41b

Updated admin extension with download-root option

f418075

Added check during import to unassociate and remove repos not present…

c06c179

… in new sync

Removed download-root option from admin client, updated importer with…

77e3a08

… mirror repo sync code

Support for Debian packages (BAREOS-2009)

625b644

Change-Id: Ia3b9c10decf840b39ffbdb1f25e37217010d4354

Added signing suport (BAREOS-2172)

862d79b

Change-Id: I8892fad1d5eef784740c6cb6a94469b9502befab Sem-Ver: feature

Signing code has shifted in debpgrk.

f3be61c

Change-Id: I6615fdd60070bd1fb97faf0a0bb0f289b578510c

Added support for syncing repos (BAREOS-2171)

0a6e9b4

There are limitations: only one component/arch can be synced at this time. Change-Id: I54671d265143121b42fb5517aa15bfe400c2f9fe

Save dependency relationships in the unit (BAREOS-2407)

60323a5

closes #2637 Change-Id: Id5eb5bd0f4cef304688824b301f1a94c72372281

bmbouter reviewed Apr 7, 2017

View reviewed changes

dkliban requested changes Apr 12, 2017

View reviewed changes

mibanescu force-pushed the master branch from 893ab85 to 7305c9d Compare April 12, 2017 15:23

Refactor #878 added back pulp-dev.py to enable jenkins to run tests.

6123969

Also updated test_requirements.txt and dist_list.txt Change-Id: I195703d7bd61eab38f95d2e6a7955e5a92aa6fa7

mibanescu force-pushed the master branch from 7305c9d to 6123969 Compare April 12, 2017 15:29

Refactor #878 Removed copyright statements as suggested by bmbouter

05b7cfc

Change-Id: Ic89510ba9c279158c0a94e5042c846df5882ae0f

dkliban requested changes Apr 12, 2017

View reviewed changes

mibanescu force-pushed the master branch from 1957f9e to 31c6776 Compare April 12, 2017 19:39

mibanescu force-pushed the master branch from 14a9675 to 4f46b76 Compare April 27, 2017 15:47

mibanescu force-pushed the master branch from 4f46b76 to a1fc720 Compare April 27, 2017 21:44

mibanescu force-pushed the master branch from a1fc720 to 4b1b019 Compare April 27, 2017 21:59

mibanescu force-pushed the master branch from 4b1b019 to a473cbc Compare April 27, 2017 22:09

dkliban reviewed May 2, 2017

View reviewed changes

mibanescu force-pushed the master branch from a473cbc to 27d854f Compare May 12, 2017 20:05

bmbouter reviewed May 15, 2017

View reviewed changes

mibanescu force-pushed the master branch 2 times, most recently from 215d7f8 to f417a0e Compare May 15, 2017 19:28

mibanescu force-pushed the master branch from f417a0e to 4df79d8 Compare May 15, 2017 19:58

bmbouter approved these changes May 17, 2017

View reviewed changes

dkliban approved these changes May 17, 2017

View reviewed changes

dkliban merged commit de4c554 into pulp:master May 17, 2017

#2637 and #878 #18

#2637 and #878 #18

mibanescu commented Apr 6, 2017

pulpbot commented Apr 6, 2017

pulpbot commented Apr 6, 2017

bmbouter Apr 7, 2017

mibanescu Apr 7, 2017

bmbouter Apr 7, 2017

bmbouter Apr 7, 2017

mibanescu Apr 10, 2017

bmbouter commented Apr 7, 2017

bmbouter commented Apr 7, 2017

mibanescu commented Apr 10, 2017

dkliban commented Apr 11, 2017

dkliban left a comment

dkliban commented Apr 12, 2017

dkliban Apr 12, 2017

dkliban commented Apr 12, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

bmbouter commented Apr 27, 2017

dkliban May 2, 2017

mibanescu commented May 12, 2017

mibanescu commented May 12, 2017

dkliban commented May 12, 2017

bmbouter May 15, 2017

bmbouter May 15, 2017

bmbouter May 15, 2017

bmbouter May 15, 2017

bmbouter May 15, 2017

mibanescu May 15, 2017

mibanescu commented May 15, 2017

mibanescu commented May 15, 2017

mibanescu commented May 16, 2017

bmbouter left a comment

dkliban left a comment

mibanescu commented May 17, 2017

		@@ -0,0 +1,93 @@
		# This software is licensed to you under the GNU General Public

		@@ -0,0 +1,53 @@
		# This software is licensed to you under the GNU General Public

		@@ -0,0 +1,200 @@
		# This software is licensed to you under the GNU General Public

#2637 and #878 #18

#2637 and #878 #18

Conversation

mibanescu commented Apr 6, 2017

pulpbot commented Apr 6, 2017

pulpbot commented Apr 6, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bmbouter commented Apr 7, 2017

bmbouter commented Apr 7, 2017

mibanescu commented Apr 10, 2017

dkliban commented Apr 11, 2017

dkliban left a comment

Choose a reason for hiding this comment

dkliban commented Apr 12, 2017

Choose a reason for hiding this comment

dkliban commented Apr 12, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

mibanescu commented Apr 27, 2017

bmbouter commented Apr 27, 2017

Choose a reason for hiding this comment

mibanescu commented May 12, 2017

mibanescu commented May 12, 2017

dkliban commented May 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mibanescu commented May 15, 2017

mibanescu commented May 15, 2017

mibanescu commented May 16, 2017

bmbouter left a comment

Choose a reason for hiding this comment

dkliban left a comment

Choose a reason for hiding this comment

mibanescu commented May 17, 2017