Refactor .deb builds #1700
Refactor .deb builds #1700
Conversation
|
Weird. All of the Debian and Ubuntu builds in the CI at https://ci.columnstore.mariadb.net/mariadb-corporation/mariadb-columnstore-engine/1402 are passing, but the OpenSUSE, Centos 8 and Centos 9 builds are failing on test columnstore/basic.mcs38_select_limit. |
We have different mtr results between deb and rpm platforms. And for that reason we don't have mtr's step on debs pipelines. I hope we will fix tests/configuration very soon to get the same results on debs. I masked mcs39_select_limit test (it will be skipped) for a while (needs investigation). |
|
this copy of debian/autobake-deb.sh was made for testing of transition some deb packaging stuff to that repo. and now it is not used at all. we just forgot to delete it. |
ottok
force-pushed
the
ok-10.5-rewrite-deb-build
branch
4 times, most recently
from
86f26f7
to
0130e6b
Compare
| @@ -11,17 +11,27 @@ local server_ref_map = { | |||
| }; | |||
|
|
|||
| local builddir = 'verylongdirnameforverystrangecpackbehavior'; | |||
| local cmakeflags = '-DCMAKE_BUILD_TYPE=RelWithDebInfo -DPLUGIN_COLUMNSTORE=YES -DPLUGIN_XPAND=NO -DPLUGIN_MROONGA=NO -DPLUGIN_ROCKSDB=NO -DPLUGIN_TOKUDB=NO -DPLUGIN_CONNECT=NO -DPLUGIN_SPIDER=NO -DPLUGIN_OQGRAPH=NO -DPLUGIN_SPHINX=NO'; | |||
|
|
|||
| // This is stupid use of PLUGIN_*=NO syntax. It should just skip building | |||
There was a problem hiding this comment.
do you know the proper path how to reach that? it looks stupid indeed. but I don't know other ways to skip plugins. you could address that to mariadb server project. debian/autobase-deb.sh uses the same stupid way.
There was a problem hiding this comment.
For autobake-deb.sh it is not stupid, since it wants to build as much of the server as possible and "blacklist" only a few ones that take too long.
In ColumnStore you want to avoid building anything, and only "whitelist" ColumnStore to be built.
This makes the current Drone CI build .deb partspass and also simplifies the config so that future maitenance of it will be easier and less error prone. The mcs39_select_limit test failure still remains for .rpm. Also remove autobake-deb.sh as it is unfit for this git repository. The debian/autobake-deb.sh is designed as a build system for the MariaDB Server that automatically adapts the build dependencies according to each platform the MariaDB Server supports. This is not in any way designed or fit for use with ColumnStore. It was copied to this repo in commit 68f15c9 without any motivations on why and accoring to Roman it was just a temporary thing that is safe to remove.
ottok
force-pushed
the
ok-10.5-rewrite-deb-build
branch
from
d71374e
to
28eb4c9
Compare
|
@mariadb-RomanNavrotskiy Removed commit d71374e as it only made the rpm MTR fail even more. Squashed this PR into a single commit. Feel free to merge now or request further changes (different target branch, other "trimming"?). |
| // _everything_ else apart from the things that ColumnStore actually needs. | ||
| local cmakeflags = '-DCMAKE_BUILD_TYPE=RelWithDebInfo -DPLUGIN_COLUMNSTORE=YES -DPLUGIN_XPAND=NO -DPLUGIN_MROONGA=NO -DPLUGIN_ROCKSDB=NO ' + | ||
| '-DPLUGIN_TOKUDB=NO -DPLUGIN_CONNECT=NO -DPLUGIN_SPIDER=NO -DPLUGIN_OQGRAPH=NO -DPLUGIN_SPHINX=NO ' + | ||
| '-DPLUGIN_PERFSCHEMA=NO -DWITH_EMBEDDED_SERVER=OFF -DWITH_WSREP=OFF'; |
There was a problem hiding this comment.
IIRC our bulk insertion buffer depends on perfschema so we can't compler w/o it.
There was a problem hiding this comment.
Apparently you can compile and the test suite passes as the CI shows? Or is the CI unreliable, it does not actually run a ColumnStore test suite?
There was a problem hiding this comment.
Yes, ATM the feature is in testing so production MTR that is also under development doesn't have tests for the feature.
There was a problem hiding this comment.
I've added perfschema back in PR#1702 for develop-5, which you should merge, since you seem to merge from develop-5 onwards to develop and this PR will become obsolete.
|
Many thanks for the help with debian packaging @ottok |
|
This is superseded by #1702 |
Refactor and simplify .deb build steps in Drone CI config (duplicate of #1700 for develop-5 branch)
Feel free to merge this. At least the build works now, but it is far from ideal yet. I might continue with this project in #1701 if I have time.
Before:
After:
The rpm builds are still failing on test columnstore/basic.mcs38_select_limit which is unrelated to this PR.