Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CONC-456: Introduce INSTALL_PLUGINDIR_CLIENT #130

Merged
merged 1 commit into from Mar 28, 2020

Conversation

ottok
Copy link
Contributor

@ottok ottok commented Mar 26, 2020

If there is a INSTALL_PLUGINDIR_CLIENT it will override whatever value
INSTALL_PLUGINDIR had. This is useful in MariaDB Server builds where this
repository exists as a git submodule in directory libmariadb/.

Normally, when a INSTALL_PLUGINDIR is set (which happens in the server) it
will override whatever values are in the client library. The client
library does define its own layout for INSTALL_LAYOUT=DEB, but it is
near useless, since the values are the same as in the default, apart for
INSTALL_LIBDIR, which points to /usr/lib//libmariadb3/plugins, but
is overridden. With this new flag, it will not be overridden, and will
actually be set as we want it to be in new Debian builds.

Why /usr/lib//libmariadb3/plugins?

  • , e.g. x86_64-linux-gnu, is needed to allow co-installation of
    32-bit and 64-bit on Debian, and is relevant for libmariadb3 pacakge.
  • The "libmariadb" part separates these client library plugins from the
    server plugins, which are now in /usr/lib//mysql/plugins, but likely
    in /usr/lib/
    /mariadb/plugins in the future, and at that point we don't
    want them to get mixed with the client ones.
  • The "3" in in "libmariadb3/plugins" is needed since the libmariadb3 can
    in far future be co-installed with package libmariadb4, since there could
    be on a running system programs built either against libmariadb3 or
    libmariadb4, at least during some transitioning period. When different
    versions of a package are co-installable, both the pacakge name and the
    file paths need to have a version component.
    Note that package libmariadb-dev does not have a version nor versioned
    paths, as for any given point in time when new builds are made, they
    are always made against the latest library version.

In addition, the PLUGINDIR_DEB option is removed since to our knowledge
nothing ever used it and it is thus obsolete.

Is this backwards compatible?

  • Since the next release will be 3.1.8 (a minor release) all changes need
    to be fully backwards compatible.
  • The version 3.1.8 will be included in all new MariaDB versions 10.2,
    10.3, 10.4 and 10.5. Of these, only 10.5 is under development and allows
    functional changes to be made. This means, that only in MariaD 10.5 there
    can be paths changed. Since this is a new option introduced, and in 10.5
    the option to use CONC_INSTALL_LAYOUT=DEB will be introduced, this code
    path is only active in MariaDB 10.5 and later with MariaDB Connector C
    3.1.8 or later. This has no effect on 10.2, 10.3 and 10.4, and is thus
    safe to apply.

This also fixes CONC-304, which was partially already fixed, but never
applied in server builds due to the same buggy build flag handling.

In addition, some line ending whitespace was removed.

Change tested with full builds of both the client and server and comparing
full deb file listings:

$ diff -u filelist-ad1a0660dfd.log filelist-1f57e8a0f64.log
--- filelist-ad1a0660dfd.log	2020-03-26 22:04:19.821001386 +0200
+++ filelist-1f57e8a0f64.log	2020-03-27 00:02:12.156576964 +0200
@@ -2,13 +2,13 @@
 drwxr-xr-x root/root ./
 drwxr-xr-x root/root ./usr/
 drwxr-xr-x root/root ./usr/lib/
-drwxr-xr-x root/root ./usr/lib/mysql/
-drwxr-xr-x root/root ./usr/lib/mysql/plugin/
--rw-r--r-- root/root ./usr/lib/mysql/plugin/client_ed25519.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/dialog.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/mysql_clear_password.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/sha256_password.so
 drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/client_ed25519.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/dialog.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/mysql_clear_password.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/sha256_password.so
 -rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb.so.3
 drwxr-xr-x root/root ./usr/share/
 drwxr-xr-x root/root ./usr/share/doc/
@@ -514,7 +514,8 @@
 -rw-r--r-- root/root ./usr/include/mariadb/server/typelib.h
 drwxr-xr-x root/root ./usr/lib/
 drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
--rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.a
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb.a
+lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.a
 lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.so
 lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadb.so
 -rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmysqlservices.a
@@ -741,9 +742,10 @@
 drwxr-xr-x root/root ./
 drwxr-xr-x root/root ./usr/
 drwxr-xr-x root/root ./usr/lib/
-drwxr-xr-x root/root ./usr/lib/mysql/
-drwxr-xr-x root/root ./usr/lib/mysql/plugin/
--rw-r--r-- root/root ./usr/lib/mysql/plugin/auth_gssapi_client.so
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/auth_gssapi_client.so
 drwxr-xr-x root/root ./usr/share/
 drwxr-xr-x root/root ./usr/share/doc/
 drwxr-xr-x root/root ./usr/share/doc/mariadb-plugin-gssapi-client/

The MariaDB Server part will be fixed in MDEV-22053.

When and in what order can this be rolled out?

  • This change only affects server builds once the MDEV-22053 is fixed, so
    this can be merged on the MariaDB Connector C at any time.
  • The server MDEV-22053 can be fixed at any time, as it only starts to
    apply to the server once MariaDB Connector C 3.1.8 is released and
    the MariaDB Server source code submodule libmariadb is updated, at which
    point the Debian packaging is affected and needs to be updated.

Are there resources to carry out this change?

  • Otto made this change and will also fix MDEV-22053.
  • Otto will update MariaDB Server 10.5 packaging as soon as MariaDB
    Connector C 3.1.8 is out and updated as submodule in server.

@9EOR9
Copy link
Collaborator

9EOR9 commented Mar 27, 2020

Hi Otto,

thanks for your contribution. Unfortunately travis fails since my last commit from yesterday evening - I need to check the reason first before I can merge your pull request..

/Georg

@ottok
Copy link
Contributor Author

ottok commented Mar 27, 2020

3.1 branch history on Travis-CI is unstable due to threadpool mtr test failure in server code:
image

�];mtr: spent 0:01:15 on 203 tests. 0:04:53 (796 tests) left�main.thread_pool_info                    w8 [ fail ]
        Test ended at 2020-03-26 23:10:01
CURRENT_TEST: main.thread_pool_info
--- /home/travis/build/mariadb-corporation/workdir-server/mysql-test/main/thread_pool_info.result	2020-03-26 22:47:22.355276000 +0000
+++ /home/travis/build/mariadb-corporation/workdir-server/mysql-test/main/thread_pool_info.reject	2020-03-26 23:10:01.466362999 +0000
@@ -55,7 +55,7 @@
 FLUSH THREAD_POOL_STATS;
 SELECT SUM(DEQUEUES_BY_LISTENER+DEQUEUES_BY_WORKER)  FROM INFORMATION_SCHEMA.THREAD_POOL_STATS;
 SUM(DEQUEUES_BY_LISTENER+DEQUEUES_BY_WORKER)
-1
+0
 SELECT SUM(POLLS_BY_LISTENER+POLLS_BY_WORKER) FROM INFORMATION_SCHEMA.THREAD_POOL_STATS;
 SUM(POLLS_BY_LISTENER+POLLS_BY_WORKER)
 2
mysqltest: Result content mismatch
 - saving '/home/travis/build/mariadb-corporation/workdir-server/mysql-test/var/8/log/main.thread_pool_info/' to '/home/travis/build/mariadb-corporation/workdir-server/mysql-test/var/log/main.thread_pool_info/'
Only  277  of 1080 completed.
--------------------------------------------------------------------------
The servers were restarted 110 times
Spent 358.643 of 75 seconds executing testcases
Failure: Failed 1/184 tests, 99.46% were successful.
Failing test(s): main.thread_pool_info

@ottok
Copy link
Contributor Author

ottok commented Mar 27, 2020

@ottok
Copy link
Contributor Author

ottok commented Mar 27, 2020

@9EOR9 Do you have any comments about the contents of this PR? Is the approach OK? Any details you would like me to amend?

If there is a INSTALL_PLUGINDIR_CLIENT it will override whatever value
INSTALL_PLUGINDIR had. This is useful in MariaDB Server builds where this
repository exists as a git submodule in directory libmariadb/.

Normally, when a INSTALL_PLUGINDIR is set (which happens in the server) it
will override whatever values are in the client library. The client
library does define its own layout for INSTALL_LAYOUT=DEB, but it is
near useless, since the values are the same as in the default, apart for
INSTALL_LIBDIR, which points to /usr/lib/<arch>/libmariadb3/plugins, but
is overridden. With this new flag, it will not be overridden, and will
actually be set as we want it to be in new Debian builds.

Why /usr/lib/<arch>/libmariadb3/plugins?
- <arch>, e.g. x86_64-linux-gnu, is needed to allow co-installation of
  32-bit and 64-bit on Debian, and is relevant for libmariadb3 pacakge.
- The "libmariadb" part separates these client library plugins from the
  server plugins, which are now in /usr/lib/*/mysql/plugins, but likely
  in /usr/lib/*/mariadb/plugins in the future, and at that point we don't
  want them to get mixed with the client ones.
- The "3" in in "libmariadb3/plugins" is needed since the libmariadb3 can
  in far future be co-installed with package libmariadb4, since there could
  be on a running system programs built either against libmariadb3 or
  libmariadb4, at least during some transitioning period. When different
  versions of a package are co-installable, both the pacakge name and the
  file paths need to have a version component.
  Note that package libmariadb-dev does not have a version nor versioned
  paths, as for any given point in time when new builds are made, they
  are always made against the latest library version.

In addition, the PLUGINDIR_DEB option is removed since to our knowledge
nothing ever used it and it is thus obsolete.

Is this backwards compatible?
- Since the next release will be 3.1.8 (a minor release) all changes need
  to be fully backwards compatible.
- The version 3.1.8 will be included in all new MariaDB versions 10.2,
  10.3, 10.4 and 10.5. Of these, only 10.5 is under development and allows
  functional changes to be made. This means, that only in MariaD 10.5 there
  can be paths changed. Since this is a new option introduced, and in 10.5
  the option to use CONC_INSTALL_LAYOUT=DEB will be introduced, this code
  path is only active in MariaDB 10.5 and later with MariaDB Connector C
  3.1.8 or later. This has no effect on 10.2, 10.3 and 10.4, and is thus
  safe to apply.

This also fixes CONC-304, which was partially already fixed, but never
applied in server builds due to the same buggy build flag handling.

In addition, some line ending whitespace was removed.

Change tested with full builds of both the client and server and comparing
full deb file listings:

$ diff -u filelist-ad1a0660dfd.log filelist-1f57e8a0f64.log
--- filelist-ad1a0660dfd.log	2020-03-26 22:04:19.821001386 +0200
+++ filelist-1f57e8a0f64.log	2020-03-27 00:02:12.156576964 +0200
@@ -2,13 +2,13 @@
 drwxr-xr-x root/root ./
 drwxr-xr-x root/root ./usr/
 drwxr-xr-x root/root ./usr/lib/
-drwxr-xr-x root/root ./usr/lib/mysql/
-drwxr-xr-x root/root ./usr/lib/mysql/plugin/
--rw-r--r-- root/root ./usr/lib/mysql/plugin/client_ed25519.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/dialog.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/mysql_clear_password.so
--rw-r--r-- root/root ./usr/lib/mysql/plugin/sha256_password.so
 drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/client_ed25519.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/dialog.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/mysql_clear_password.so
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/sha256_password.so
 -rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb.so.3
 drwxr-xr-x root/root ./usr/share/
 drwxr-xr-x root/root ./usr/share/doc/
@@ -514,7 +514,8 @@
 -rw-r--r-- root/root ./usr/include/mariadb/server/typelib.h
 drwxr-xr-x root/root ./usr/lib/
 drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
--rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.a
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb.a
+lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.a
 lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadbclient.so
 lrwxrwxrwx root/root ./usr/lib/x86_64-linux-gnu/libmariadb.so
 -rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmysqlservices.a
@@ -741,9 +742,10 @@
 drwxr-xr-x root/root ./
 drwxr-xr-x root/root ./usr/
 drwxr-xr-x root/root ./usr/lib/
-drwxr-xr-x root/root ./usr/lib/mysql/
-drwxr-xr-x root/root ./usr/lib/mysql/plugin/
--rw-r--r-- root/root ./usr/lib/mysql/plugin/auth_gssapi_client.so
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/
+drwxr-xr-x root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/
+-rw-r--r-- root/root ./usr/lib/x86_64-linux-gnu/libmariadb3/plugin/auth_gssapi_client.so
 drwxr-xr-x root/root ./usr/share/
 drwxr-xr-x root/root ./usr/share/doc/
 drwxr-xr-x root/root ./usr/share/doc/mariadb-plugin-gssapi-client/


The MariaDB Server part will be fixed in MDEV-22053.

When and in what order can this be rolled out?
- This change only affects server builds once the MDEV-22053 is fixed, so
  this can be merged on the MariaDB Connector C at any time.
- The server MDEV-22053 can be fixed at any time, as it only starts to
  apply to the server once MariaDB Connector C 3.1.8 is released _and_
  the MariaDB Server source code submodule libmariadb is updated, at which
  point the Debian packaging is affected and needs to be updated.

Are there resources to carry out this change?
- Otto made this change and will also fix MDEV-22053.
- Otto will update MariaDB Server 10.5 packaging as soon as MariaDB
  Connector C 3.1.8 is out and updated as submodule in server.
@9EOR9 9EOR9 merged commit 21a9d12 into mariadb-corporation:3.1 Mar 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants