New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure connecting Heidi 9.5 to MySQL 8.0 due to missing plugin: caching_sha2_password #163

Open
ebyrob opened this Issue Feb 26, 2018 · 69 comments

Comments

Projects
None yet
@ebyrob

ebyrob commented Feb 26, 2018

Steps to reproduce this issue

  1. Download and install MySQL 8.0 RC
  2. Verify MySQL up and running on default port.
  3. Download and install HeidiSQL 9.5
  4. Attempt to connect unencrypted to "localhost" / 127.0.0.1 (if IPv4)

Current behavior

---------------------
     Error
---------------------
Authentication plugin 'caching_sha2_password' cannot be loaded: The specified module could not be found.
          [ OK ]
---------------------

Expected behavior

Successful connection.

Possible solution

Find or build missing plugin file: caching_sha2_password.dll
or perhaps plugin dll has slightly different name, I can't find it.

Environment

  • HeidiSQL version:
    9.5.0.5196
  • Database system and version:
    mysql Ver 8.0.4-rc for Win64 on x86_64 (MySQL Community Server (GPL))
  • Operating system:
    Windows 7 64-bit.

Thanks for a great program!!! I love HeidiSQL, but realized I can't build it without a Borland compiler I never bought, but I won't panic.

Thank you in advance for your help,
Robert Eby

@rentalhost

This comment has been minimized.

Collaborator

rentalhost commented Feb 26, 2018

Could you try it again by using the last nightly version?

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Feb 26, 2018

There is just no caching_sha2_password.dll in HeidiSQL's plugins folder. You need to copy it from the server distribution.

Not sure if I should add some more plugins to the installer, along with the both dialog.dll and auth_gssapi_client.dll

@ebyrob

This comment has been minimized.

ebyrob commented Feb 26, 2018

I have no idea what should be in HeidiSQL, I'm just glad it works so well.

I wanted to copy a file called caching_sha2_password.dll when I saw the error message but I couldn't find it anywhere in the pre-built binaries for MySQL. I also tried copying all pre-built plugins from MySQL 8.0 RC main release but that did not work either.

A trusted colleague thought perhaps it could have been repackaged under another name, but I thought it was likely that mysql.exe was using it from static include and didn't need the DLL.

@G3ph4z

This comment has been minimized.

Contributor

G3ph4z commented Feb 26, 2018

I think you should upgrade the libmysql.dll (MySQL Connector lib). I will check that.

@rentalhost

This comment has been minimized.

Collaborator

rentalhost commented Feb 26, 2018

It was reported on forum too: https://www.heidisql.com/forum.php?t=24939

Seems related to MySQL 8.x, but I can't found where we can download it. Maybe just update libmysql.dll.

@G3ph4z

This comment has been minimized.

Contributor

G3ph4z commented Feb 26, 2018

@rentalhost Doesnt help. I tried with 8.0.3 version libmysql.
-> http://prtsc.denes.cloud/vmware_2018-02-26_22-42-08.png

If a client or connector is available that has been updated to know about caching_sha2_password, using it is the best way to ensure compatibility when connecting to a MySQL 8.0 server configured with caching_sha2_password as the default authentication plugin.

These clients and connectors have been upgraded to support caching_sha2_password:

The libmysqlclient client library in MySQL 8.0.4 or higher. Standard MySQL clients such as mysql and mysqladmin are libmysqlclient-based, so they are compatible as well.

MySQL Connector/J 8.0.9 or higher.

MySQL Connector/Net 8.0.10 or higher (through the classic MySQL protocol).

MySQL Connector/Node.js 8.0.9 or higher. 

But If I try with 8.0.4 libmysqlclient, I get this:
http://prtsc.denes.cloud/vmware_2018-02-26_22-48-13.png

DLL:
http://prtsc.denes.cloud/vmware_2018-02-26_22-51-25.png

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Feb 27, 2018

@ebyrob you should install the latest built installer once, then you will get an additional libmariadb.dll in your HeidiSQL folder, which Heidi then uses. I added that lib after the last 9.5 release, so most likely you don't have it yet. It's from the MariaDB 10.2.12 GA release.
The remaining old libmysql.dll is still there and used as a fallback.

@ebyrob

This comment has been minimized.

ebyrob commented Feb 28, 2018

Could you try it again by using the last nightly version?

I did try this, and I'm amazed how easy grabbing nightly is, but no luck.

I think you should upgrade the libmysql.dll (MySQL Connector lib). I will check that.

I couldn't find RC for MySQL Connect/C, but I also didn't see a readily available "RC" version on the website. I may need to see if I can build it myself. And indeed, I was told on MySQL forums that the sever's copy of mysql.exe uses a static include so it isn't there either.

The libmysqlclient client library in MySQL 8.0.4 or higher.

I did find libmysqlclient.lib in the server binaries, it seems likely I could static link to that but I wasn't sure how that would plug in to HeidiSQL.

you should install the latest built installer once, then you will get an additional libmariadb.dll...

I'm not 100% but libmariadb.dll seems to get exactly the same error. This is very interesting. Notes: I renamed libmysql.dll to avoid loading it. My other upgrade test machine is happily running on latest MariaDB though comparing MariaDB release to MySQL RC probably isn't fair.

@ebyrob

This comment has been minimized.

ebyrob commented Mar 14, 2018

Sorry for the long delay, I finally had time to look at this again and this time I'm getting the following:

Error

SSL connection error: error:00000001:lib(0):func(0):reason(1)

[OK]

When trying to connect. This only happens after I copy the 3 necessary dll's from MySQL 8.0 RC build to replace libmysql.dll. I'm suspicious the problem is I need to re-build Heidi, but I'm not sure where to get the right Pascal compiler. (I do have a test application connecting just fine but I can rebuild that all I need to)

@G3ph4z

This comment has been minimized.

Contributor

G3ph4z commented Mar 14, 2018

@ebyrob You only can compile with RAD Studio XE5 or newer versions of RAD Studio.

Delphi XE5 is required for building HeidiSQL. Older Delphi versions will most likely fail; newer Delphi versions may work or fail. Unfortunately, Larazus or one of the other free compilers cannot currently compile HeidiSQL.

@ebyrob

This comment has been minimized.

ebyrob commented Mar 14, 2018

So Embarcadero RAD Studio for $2,200 would work? It seems XE5 isn't available at any price, but I didn't exactly go digging for it.

https://www.lazarus-ide.org/ won't work? Ok got it. Ty. Just want to be clear here.

@G3ph4z

This comment has been minimized.

Contributor

G3ph4z commented Mar 14, 2018

You have right. But yes, its so sad, the Embarcadero dont want to release free version (like Visual Studio) for OpenSource Projects, so.... I think you can use trial version for compile, the 10.x.x working fine too.

Btw, the Lazarus is good enough if you want to create some applications in Pascal.

@SpyroTEQ

This comment has been minimized.

SpyroTEQ commented Mar 20, 2018

It won't fix the HeidiSQL issue, but until then and for development environment, one can still set their MySQL 8.0 to use the previous SQL password plugin.

Either by altering/creating a user with elder credentials and then use it in Heidi (ALTER USER ... IDENTIFIED WITH mysql_native_password BY password that can be done command-line using bin/mysql --user=...) or by setting up the MySQL 8.0 instance to use the old hashing plugin ([mysqld]default_authentication_plugin=mysql_native_password + bin/mysqld --initialize-insecure so you can then connect to your freshly setup MySQL 8.0 instance using Heidi)

Still, that's just a workaround for those (like me!) who would like to try MySQL8.0 new features in their development environment (using old hashing system in production is not what I would recommend :/ )

@ebyrob

This comment has been minimized.

ebyrob commented Mar 21, 2018

@littleearth

This comment has been minimized.

littleearth commented Apr 27, 2018

From some testing I have done, there does not seem to be a version of libmysql.dll for 32bit for Version 8 of Mysql

@binarymaster

This comment has been minimized.

binarymaster commented Apr 28, 2018

https://stackoverflow.com/a/49935803/4381493

This answer helped me to overcome the issue.

@littleearth

This comment has been minimized.

littleearth commented Apr 29, 2018

Thanks, yes I have also worked around but I was trying to implement the new security in my own application and it so far does not seem possible for 32bit and the libmysql.dll as there is no version 8 of it. 64bit works fine so in theory should for heidisql but I have not tested.

@mlocati

This comment has been minimized.

mlocati commented May 3, 2018

I've been able to configure a development machine so that we can connect to in with HeidiSQL:

  1. I added this line to the the MySQL Server configuration file (and restarted it)
[mysqld]
default_authentication_plugin=mysql_native_password
  1. With the MySQL command line client (bin/mysql) I updated the password of the user I use with HeidiSQL:
ALTER USER 'username'@'hostname' IDENTIFIED WITH mysql_native_password BY 'password';
FLUSH PRIVILEGES;
@natachabox

This comment has been minimized.

natachabox commented Jun 25, 2018

I was able to connect by using the command line :
Change the auth method for your passbolt user on your mysql 8 installation:
ALTER USER 'username'@'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Jul 22, 2018

I had just luck with libmysql.dll from the MySQL 8.0.11 server release, after also synchronizing some structures from mysql.h into HeidiSQL. With that dll both above errors are fixed (caching_sha2_password missing + ssl connection error).
But I have problems in finding a 32bit version of that libmysql.dll. HeidiSQL normally supports both 64 + 32 bit Windows versions. On the download page I can just find a 64bit version. Can someone point me to the right place please? Or has MySQL stopped support for 32bit systems in the Community Edition?

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Jul 22, 2018

Btw, MySQL Workbench had the same problem some months ago, and Mike Lischke posted a relevant comment on StackOverflow.

@shincejose

This comment has been minimized.

shincejose commented Jul 22, 2018

I installed the latest version of Mysql 8.1 community edition just for sha2 encryption. But most answers suggests using the old encryption. Is there a solution yet? I'm also trying to connect using Heidisql and getting the error "caching_sha2_password cannot be loaded on connect screen itself. Heidisql 9.5.0.5280 . updated today.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Jul 22, 2018

I'm just a tick away from a solution here, see my previous comments. I just need the 32bit version of the libmysql.dll v8.0.11.

@danitxo

This comment has been minimized.

danitxo commented Jul 22, 2018

It seems that the ZIP package is x64 only ("x86, 64-bit", that text is a little confusing). The MSI package, however, is 32 and 64 bit (it puts "x86, 32 & 64 bit").

Here is the link for the MSI package, and here is the direct link for the 8.0.11 release. Microsoft Visual C++ 2015 Redist are needed.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Jul 22, 2018

I already tried that 32bit .msi installer, but ended up with a 64bit server version afterwards. Also visible during the install - there is just a 64bit server to install, no 32bit available.

@littleearth

This comment has been minimized.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Jul 23, 2018

Too bad. Probably MariaDB also has support for the same stuff. I'll check that.

@psychelzh

This comment has been minimized.

psychelzh commented Nov 14, 2018

Will it be fixed in future releases? The workarounds are not so perfect in face of dealing with the two kinds of MySQL password settings simultaneously.

@ebyrob

This comment has been minimized.

ebyrob commented Nov 20, 2018

Sorry if this is old news but...

I've just been testing with latest master mariadb/connectr-c from github. I don't think it's in pre-built release yet, but there now seems to be support for plugin caching_sha2_password in libmariadb.dll !!

I'm a bit confused that mariadb connector 3.1 is newer than mariadb 10.0, but I think this is new stuff.

I've just been testing in a small 32-bit C++ application. The fact it works for everything is awesome.

Now if only I can figure out how to change the lib/mariadb/plugin/ sub-folder under running executable where it is looking for caching_sha2_password.dll

@mlocati

This comment has been minimized.

mlocati commented Nov 20, 2018

@ebyrob I compiled auth_gssapi_client.dll from version 3.0.7 (released on 2018-11-08, that is 12 days ago) of https://github.com/MariaDB/mariadb-connector-c, and I also can't find any reference to caching_sha2_password.
BTW, by using Dependencies, I don't see any reference to missing DLLs.

The build script is at https://github.com/mlocati/auth_gssapi_client (and I pre-built the 32-bit and 64-bit binaries of it, just in case someone is able to test if they work with HeidiSQL - see https://github.com/mlocati/auth_gssapi_client/releases/download/1.0.0/binaries.zip)

@ebyrob

This comment has been minimized.

ebyrob commented Nov 21, 2018

@mlocati Sorry, I didn't mean to confuse anyone.

This was in the latest master branch of the repository. Probably it is not yet available in any pre-built or release version of the connector. (I did verify that 3.0.7 definitely DOES NOT have it yet, I only noticed this fix because someone had closed a related trouble ticket and I went looking for the fixed version)

I don't know the mariadb repository well, but I see a change went in on Oct 10:
130732d on Oct 10 - @9EOR9 9EOR9 CONC-312: Implementation of caching_sha2_plugin

In fact, CONC-312 is the trouble ticket I noticed as closed.

This file seems to be most of the fix:
mariadb-connector-c/plugins/auth/caching_sha2_pw.c

I'm not sure whether something from October 10 should have gone out November 08 with 3.0.7, but in master branch it sure seems to work great. Perhaps they're still testing it before it will go into an official release.

I must confess, I have no idea what gssapi is or why it would be relevant to MySQL 8.0's default security configuration.

@mlocati

This comment has been minimized.

mlocati commented Nov 21, 2018

@ebyrob by using master I've been able to compile caching_sha2_password.dll both for 32 and 64 bits.
It required some patches, though:

Furthermore, the MariaDB implementation of caching_sha2_password.dll uses the CryptImportPublicKeyInfoEx2 Windows function, which is available since Windows Vista.

So I had to add this patch too.

I didn't check the built DLLs (I don't know how). If someone is willing to try them (both for 32 and 64 bit Windows), simply clone https://github.com/mlocati/auth_gssapi_client and run the build-auth_gssapi_client command (on an Ubuntu virtual machine or on Windows Subsystem for Linux - which is much slower); otherwise you can use the binaries I compiled myself, available at https://github.com/mlocati/auth_gssapi_client/releases/download/1.0.1/binaries.zip.

@mlocati

This comment has been minimized.

mlocati commented Nov 21, 2018

Now if only I can figure out how to change the lib/mariadb/plugin/ sub-folder under running executable where it is looking for caching_sha2_password.dll

@ebyrob It should be as easy as setting the MARIADB_PLUGIN_DIR environment variable to C:\Path\To\HeidiSQL\plugins (see https://github.com/MariaDB/mariadb-connector-c/blob/9e1fef0bf2c47b6f75b68fc6ce3ed15495222da9/client/ma_plugin_info.c#L114)

@OliverBailey

This comment has been minimized.

OliverBailey commented Dec 5, 2018

Glad I came across this thread. Thanks.

@9EOR9

This comment has been minimized.

9EOR9 commented Dec 5, 2018

Guys,

please don't use master - default branch is 3.1 - The caching_sha2_password plugin is also available in latest 3.0.8 release which should be available tomorrow.

@OliverBailey

This comment has been minimized.

OliverBailey commented Dec 6, 2018

@9EOR9 Sorry if I'm being stupid - But currently version is 9.5.0.5196 - Where's the 3.0.8 release you're seeing?

@psychelzh

This comment has been minimized.

psychelzh commented Dec 6, 2018

@OliverBailey That means mariadb-connector-c, not HeidiSQL. BTW, when will HeidiSQL be able to update?

@OliverBailey

This comment has been minimized.

OliverBailey commented Dec 6, 2018

Right - With you. Didn't realise he was on about that!

Temporary fix for the meantime however is just copy the libmysql.dll from mysql 8.0 to the heidisql file located in program files.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 6, 2018

I am on the way to fix the last bugs for the next HeidiSQL release. Can I add the binaries from @mlocati or ..? I guess it's a good idea to also update libmariadb.dll for that release.

@mlocati

This comment has been minimized.

mlocati commented Dec 6, 2018

@ansgarbecker they patched the code accordingly to my suggestions, and added some other fixes.
We should be able to use the official 3.0.8 binaries. If they are not available from the MariaDB website I can of course compile them.

@ansgarbecker ansgarbecker removed the question label Dec 6, 2018

ansgarbecker added a commit that referenced this issue Dec 6, 2018

Issue #163: update MariDB libraries and some code structures, with th…
…ose from MariaDB Connector 3.0.8 (= MariaDB server 10.3.6)
@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 6, 2018

I have just updated all plugins, copied not yet existing ones in HeidiSQL\plugins directory, and updated structures from mysql.h. All from the Connector 3.0.8. I guess it should now be possible to connect to MySQL 8 via HeidiSQL out of the box. Only requirement is to download the latest built installer (r9.5.0.5372, 32 or 64 bit), compiled in a few minutes.

@mlocati

This comment has been minimized.

mlocati commented Dec 6, 2018

@ansgarbecker Great!
I saw that the DLLs from https://downloads.mariadb.com/Connectors/c/connector-c-3.0.8/ are quite big.. for example, caching_sha2_password.dll for 32-bit is 423 KB, whereas the one built by my script is only 19 KB. I guess that they keep in it some data helpful for debugging...
If you want to strip that data out you can use the strip command of MXE (that is, i686-w64-mingw32.static-strip for 32-bit, x86_64--w64-mingw32.static-strip for 64-bit). I don't know if there's an equivalent way with the standard Microsoft build tools.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 6, 2018

Ok, that was too quick :) I get

Plugin caching_sha2_password could not be loaded: The module was not found. Library path is 'lib/mariadb/plugin/caching_sha2_password.dll'

For MySQL plugins, HeidiSQL already sets its own plugin path:

mysql_options(FHandle, MYSQL_PLUGIN_DIR, 'c:\path\to\heidisql\plugins\');

But that seems to be ignored for MariaDB plugins. Instead, MARIADB_PLUGINDIR is a non-overridable constant in mariadb_version.h. Is there some way to change that?

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 6, 2018

@mlocati : I never compiled the dlls by mysqlf, as HeidiSQL is written in Delphi, not VC. Perhaps the server distribution has these dlls without debug information?

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 9, 2018

Now if only I can figure out how to change the lib/mariadb/plugin/ sub-folder under running executable where it is looking for caching_sha2_password.dll

@ebyrob It should be as easy as setting the MARIADB_PLUGIN_DIR environment variable to C:\Path\To\HeidiSQL\plugins (see https://github.com/MariaDB/mariadb-connector-c/blob/9e1fef0bf2c47b6f75b68fc6ce3ed15495222da9/client/ma_plugin_info.c#L114)

@mlocati Does that mean I have to set a system environment variable to tell libmariadb.dll where it has to look for plugins? Sounds strange. Or does getenv look elsewhere as well?

@mlocati

This comment has been minimized.

mlocati commented Dec 10, 2018

@ansgarbecker It should be enough to set the environment variabile for the process that's loading the DLLs.
I'm sorry but I don't know Pascal (last time I wrote something in it was about 20 years ago), so I can't give you a function name.

BTW you can check where the app is looking for the DLLs by using Process Monitor.

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 10, 2018

I only wonder why it has to be an environment variable, and why it cannot be set via mysql_options, like I do that for other things:

mysql_options(FHandle, MYSQL_PLUGIN_DIR, PAnsiChar(PluginDir));
@mlocati

This comment has been minimized.

mlocati commented Dec 10, 2018

@ansgarbecker yes, also setting the MYSQL_PLUGIN_DIR option should work

@9EOR9

This comment has been minimized.

9EOR9 commented Dec 10, 2018

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 10, 2018

Setting MYSQL_PLUGIN_DIR via mysql_options() only works with the 32bit version of libmariadb.dll, while in 64 bit mode I always get:

Plugin caching_sha2_password could not be loaded: Das angegebene Modul wurde nicht gefunden. Library path is 'lib/mariadb/plugin/caching_sha2_password.dll'

I'll go and try with mysql_optionsv instead, but I suspect it does nothing else than mysql_options.

Also, I just see they removed the v3.0.8 connector from the download page: https://downloads.mariadb.com/Connectors/c/

@9EOR9

This comment has been minimized.

9EOR9 commented Dec 10, 2018

@9EOR9

This comment has been minimized.

9EOR9 commented Dec 10, 2018

@mlocati

This comment has been minimized.

mlocati commented Dec 11, 2018

In the meanwhile I recompiled the auth_gssapi_client.dll and caching_sha2_password.dll plugins against the 3.1 branch of https://github.com/MariaDB/mariadb-connector-c

It required these patches:

You can build these 2 DLLs by using https://github.com/mlocati/auth_gssapi_client but for your convenience I published the compiled binaries at https://github.com/mlocati/auth_gssapi_client/releases/download/1.0.2/binaries.zip

@ansgarbecker

This comment has been minimized.

Collaborator

ansgarbecker commented Dec 11, 2018

Thanks, @mlocati !
I'm afraid your 64bit binary of caching_sha2_password.dll still has that hardcoded lib/mariadb/plugin directory. At least it does not care if I set it via mysql_options.

@mlocati

This comment has been minimized.

mlocati commented Dec 12, 2018

@ansgarbecker I'm not an expert of the MariaDB source code, I can only compile it.

BTW I tried to inspect what's happening with Process Monitor.
Open Process Monitor and set these two filters:

  1. Process Name -> contains -> heidisql then include
  2. Path -> contains -> plug then include

That way you'll only see the operations of heidisql that involve paths containing plug.

Enable the Process Monitor capture, open latest HeidiSQL (heidisql64.r5391.exe) and try to connect to a MySQL 8 server.

Here's the only row I can see in Process Monitor:
immagine

It means that the application is trying to load caching_sha2_password.dll from C:\Program Files\MariaDB 10.2\lib\plugin, and it fails because the DLL is not there.
So it seems that caching_sha2_password.dll is not even loaded.

But please don't ask me more details, I really don't know the source code of MariaDB...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment