rpm generation process update #1980

szemere · 2018-04-11T13:57:15Z

RPM generation process update from @czanik.
This is the first step in a long process, the final goal is to harmonize Peter's work and /dbld

remove syslog-ng-add-contextual-data.pc from file list
update to 3.14.1
re-enable amqp & mongodb on RHEL only
keep tcp wrappers support only on RHEL

Fixes: #1857

kira-syslogng · 2018-04-11T14:32:31Z

success

bazsi

It looks good to me.

bazsi · 2018-04-11T16:05:20Z

dbld/images/required-epel/all.txt

@@ -2,3 +2,5 @@
 libdbi
 libdbi-drivers
 riemann-c-client-devel
+syslog-ng-java-deps


Where does this package come from?

From my Copr repo: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/package/syslog-ng-java-deps/

I think it's not the githead branch in your repo, but version "3.11":
https://github.com/balabit/syslog-ng/blob/4ab34df81a7f97740e60aa5b0748c54b6518338b/dbld/images/helpers/functions.sh#L14-L17
This is used during docker image creation.

bazsi · 2018-04-11T16:09:09Z

dbld/rpm

+# CzP's spec file expect it in the pwd
+# where the build was initiated
+cp packaging/rhel/syslog-ng.* /build
+


My understanding is that they need to be in the rpmbuild source directory. Isn't that the case?

It is looking for the files where rpmbuild is initiated. In our case it is '''/build'''

I was experimenting with it as well.
rpmbuild is using the dist tarball, isn't it?
Why don't we include it our Makefile instead of this copy? I have tried it out and it works, it would be similar to what our debian packaging does.

gaborznagy · 2018-04-12T13:41:48Z

dbld/rpm

+# CzP's spec file expect it in the pwd
+# where the build was initiated
+cp packaging/rhel/syslog-ng.* /build
+


I was experimenting with it as well.
rpmbuild is using the dist tarball, isn't it?
Why don't we include it our Makefile instead of this copy? I have tried it out and it works, it would be similar to what our debian packaging does.

gaborznagy · 2018-04-12T13:51:59Z

dbld/images/required-epel/all.txt

@@ -2,3 +2,5 @@
 libdbi
 libdbi-drivers
 riemann-c-client-devel
+syslog-ng-java-deps


I think it's not the githead branch in your repo, but version "3.11":
https://github.com/balabit/syslog-ng/blob/4ab34df81a7f97740e60aa5b0748c54b6518338b/dbld/images/helpers/functions.sh#L14-L17
This is used during docker image creation.

gaborznagy · 2018-04-12T15:06:29Z

This is a huge step to resolve #1857 👍

MrAnno · 2018-04-13T14:40:05Z

Hi,

We've just merged a CI-related pull request into our master branch.
Since the current Travis CI checks in your PR are outdated, I'd like to ask you to rebase against master.

kira-syslogng · 2018-04-16T09:59:48Z

success

kira-syslogng · 2018-04-16T12:37:56Z

success

czanik · 2018-04-19T09:48:51Z

Did a quick test in a completely clean environment. Running dbld/rules rpm-centos7 fails with dependency problems only if dbld/rules image-centos7 is not run first. Thanks to @szemere it now also works if it was invoked as root.

As discussed IRL with @gaborznagy , my githead repo should be used instead of 311 to make sure that the right version of dependencies are used.

gaborznagy · 2018-04-19T11:12:37Z

@czanik : by completely clean environment you mean you did not have any of the docker images prior testing, right?
We've rebuilt the centos7 docker images on dockerhub a couple of days ago, automatic docker pull should be working now.

About the changes:

I have tested rpm generating in dbld and it worked when I built the image prior to testing.
I have not tested it dbld with your githead repo so far.
I still need to find resolution for the problems I found when I included the packaging/rhel/ dir's content in the dist tarball:

There is another spec in the repo, I have removed it from the dist tarball for testing, so there will be only the packaging/rhel spec file in the tarball.
I tried to remove many unnecessary copying of the packaging dir (this is common in dbld).
Your spec file expects files under the build/ dir which now has to be copied (this is what's in this PR), I try to change this to use files from the tarball instead.

szemere · 2018-04-23T11:17:10Z

Running dbld/rules rpm-centos7 fails with dependency problems only if dbld/rules image-centos7 is not run first.

You are absolutely right. A little background information:
Because of a configuration mistake in Docker Hub, the latest TAG pointed to an outdated version of the image, from the time before #1876 . (Obviously with outdated dependencies.)
By running dbld/rules image-centos7 the image was (re)built locally and overrode whatever was downloaded from Docker Hub.
I corrected the Docker Hub config on April 17th, so the latest tag will represent the last commit from the master branch. By running docker pull balabit/syslog-ng-centos7 manually you can override your local version without the long build process.

furiel

I reviewed and it looks good to me in general. If Gabors comment on wrong version: 3.11 is resolved, approve from my side.

"dbld/rpm" will work with this patch and it's awaited. However it's not a sophisticated solution, it needs more work.

gaborznagy · 2018-04-26T10:48:40Z

This PR is awaited.
Let's test "githead" repo instead of "3.11" and if it works, merge the PR in.

The resolution of the many copying of the packaging dir in dbld will be resolved later.

- remove syslog-ng-add-contextual-data.pc from file list - update to 3.15.1 - re-enable amqp & mongodb on RHEL only - keep tcp wrappers support only on RHEL - updated epel: centos7->git_head

szemere · 2018-05-08T15:30:35Z

Updated epel repo url to git head.
Version bumped the spec file.

kira-syslogng · 2018-05-08T15:55:42Z

success

furiel

I tried the patchset manually. I could build image-centos7 successfully. The rpm's are generated successfully. I could successfully install the generated rpm-s inside image-centos7, and syslog-ng started with an stdin source stdout destination, log are generated as expected.

All looks good to me! Thanks for putting this together.

lbudai added the in progress label Apr 11, 2018

bazsi approved these changes Apr 11, 2018

View reviewed changes

gaborznagy previously requested changes Apr 12, 2018

View reviewed changes

gaborznagy changed the title ~~rpm generation process update:~~ [WIP] rpm generation process update Apr 16, 2018

szemere force-pushed the update_centos_build branch from c945bd9 to 53ca55c Compare April 16, 2018 08:51

szemere force-pushed the update_centos_build branch from 53ca55c to 8e14af4 Compare April 16, 2018 12:02

kira-syslogng changed the title ~~[WIP] rpm generation process update~~ rpm generation process update Apr 18, 2018

furiel reviewed Apr 26, 2018

View reviewed changes

furiel changed the title ~~rpm generation process update~~ [WIP] rpm generation process update May 7, 2018

rpm generation process update:

d51bba8

- remove syslog-ng-add-contextual-data.pc from file list - update to 3.15.1 - re-enable amqp & mongodb on RHEL only - keep tcp wrappers support only on RHEL - updated epel: centos7->git_head

szemere force-pushed the update_centos_build branch from 8e14af4 to d51bba8 Compare May 8, 2018 15:29

szemere changed the title ~~[WIP] rpm generation process update~~ rpm generation process update May 8, 2018

szemere added needs-review and removed in progress labels May 8, 2018

furiel approved these changes May 9, 2018

View reviewed changes

gaborznagy merged commit 3ae93e3 into syslog-ng:master May 9, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rpm generation process update #1980

rpm generation process update #1980

szemere commented Apr 11, 2018 •

edited by gaborznagy

kira-syslogng commented Apr 11, 2018

bazsi left a comment

bazsi Apr 11, 2018

czanik Apr 11, 2018

gaborznagy Apr 12, 2018

bazsi Apr 11, 2018

czanik Apr 11, 2018

gaborznagy Apr 12, 2018

gaborznagy Apr 12, 2018

gaborznagy Apr 12, 2018

gaborznagy commented Apr 12, 2018

MrAnno commented Apr 13, 2018

kira-syslogng commented Apr 16, 2018

kira-syslogng commented Apr 16, 2018

czanik commented Apr 19, 2018

gaborznagy commented Apr 19, 2018 •

edited

szemere commented Apr 23, 2018

furiel left a comment

gaborznagy commented Apr 26, 2018

szemere commented May 8, 2018

kira-syslogng commented May 8, 2018

furiel left a comment

rpm generation process update #1980

rpm generation process update #1980

Conversation

szemere commented Apr 11, 2018 • edited by gaborznagy

kira-syslogng commented Apr 11, 2018

bazsi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaborznagy commented Apr 12, 2018

MrAnno commented Apr 13, 2018

kira-syslogng commented Apr 16, 2018

kira-syslogng commented Apr 16, 2018

czanik commented Apr 19, 2018

gaborznagy commented Apr 19, 2018 • edited

szemere commented Apr 23, 2018

furiel left a comment

Choose a reason for hiding this comment

gaborznagy commented Apr 26, 2018

szemere commented May 8, 2018

kira-syslogng commented May 8, 2018

furiel left a comment

Choose a reason for hiding this comment

szemere commented Apr 11, 2018 •

edited by gaborznagy

gaborznagy commented Apr 19, 2018 •

edited