Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to mongodb via SSH dosn't work in new update #307

Closed
Mai-Lapyst opened this issue Nov 29, 2021 · 27 comments
Closed

Connect to mongodb via SSH dosn't work in new update #307

Mai-Lapyst opened this issue Nov 29, 2021 · 27 comments

Comments

@Mai-Lapyst
Copy link

Same issue as in #286: Mingo errors when trying to connect to a mongodb database via SSH just with the password set. (No keyfile)

This errors pop open when you try to open an a collection in a already configured connection:
image

Also this error appears when using the "Test" button inside the connection settings:
image

Mingo Version: 1.6.4 (PRO)

@tothradoslav
Copy link
Contributor

Thanks for this. I found the issue and corrected it. Will let you know here when a new build is ready.

@dmolin
Copy link

dmolin commented Dec 14, 2021

@tothradoslav any update on this? I know the fix has been already applied 2 weeks ago but no release yet. This is pretty much a game breaking issue for an audience that intends to use this in prod environments. Is there any ETA for the next release?

@tothradoslav
Copy link
Contributor

It's about to be released in the upcoming days. Saw your twitter post, uploading the linux RC so you can test it.

https://github.com/mingo-app/mingo/releases/tag/v1.6.5

@dmolin
Copy link

dmolin commented Dec 14, 2021

Thanks a lot @tothradoslav !! Mingo.io it's a fantastic Mongo client and my colleagues and I are already loving it! ssh support is the only bit preventing us from purchasing a license and I'm sure the early access to the RC will make everyone happy here!

@tothradoslav
Copy link
Contributor

Please let me know if it worked for you.

@dmolin
Copy link

dmolin commented Dec 14, 2021

@tothradoslav so, we're still having issue even with RC1.
The error is now the following one (when testing the connection):
"Error: MongoServerSelectionError: Server selection timed out after 5000 ms"

Worth noting:

  1. we're connecting through ssh
  2. we're connecting to a server with a replica set (the error is the same whether we try to connect to the master or one of the slaves)

Hope it helps!

@tothradoslav
Copy link
Contributor

Thanks! Going to test it properly, my connection works.

Are you using ssh with privateKey, password or agent?

Will let you know when I find something.

@dmolin
Copy link

dmolin commented Dec 14, 2021

I'm using ssh with a privateKey file.
Though, there's something that seems to be missing. When using a privateKey file, I should also be required to provide a "passphrase" that the server will use to parse the private key. this element seems to be missing in the Mingo connection configuration dialog (it asks either for a password or a privateKey)

@tothradoslav
Copy link
Contributor

Just uploaded a new pre-release, could you please try this? We added the passphrase option in SSH.

https://github.com/mingo-app/mingo/releases/tag/v1.6.5-rc.2

Thank you!

@dmolin
Copy link

dmolin commented Dec 15, 2021

Hey @tothradoslav , thanks a lot for the hard work!

We're definitely "getting there"!
So, with the passphrase I can successfully "test" the connection. (the test succeeds).

Though, I see these issues:

  • the passphrase is written in clear (it should be showing up as a password field, with "*")
  • the passphrase is not persisted. if I close the configuration panel and reopen it, it's gone and I've to re-enter it again
  • Not sure if connected to the previous point, but when I try to switch to the connection I just created my DB shows up as completely empty (like, no collections are there). Not sure if this is a failure to actually connect to the DB itself. I'm using the mongodb connection string with this format "mongodb://uid:pwd@url:port/dbname" and I assume everything is fine since the "test connection" doesn't report any error.

Another thing that might interest you, feature wise: there's no option for connecting through TLS/SSL (something we use for some of our DBs). Take NoSqlBooster as an example here:

image

They allow you to setup a configuration for your connection that accounts for all these possibilities.

@tothradoslav
Copy link
Contributor

Great, thanks for this. I will go through all your points and make sure it all works, including the SSL / TLS connections.

@tothradoslav
Copy link
Contributor

Just 2 questions:

  1. point 2, I can't reproduce it, it remembers the passphrase. Could you tell me more about the situation? If you could try to create a blank new connection and add it there? Just looking for some clues. Tried many options, can't reproduce.
  2. Have you tried the refresh connection button in the left sidebar? Or if you close / open Mingo? (I know these are dummy solutions, just trying to find a way to debug this).

Thanks for all your help!

@dmolin
Copy link

dmolin commented Dec 15, 2021

So, good news on the passphrase :)
Creating a new connection seems to fix it and it's persisted correctly, so that's a win :)
The issue with the empty database remains though: I tried hitting the "refresh collection" button: it spins for a wile but then the result is still "0" at the end.
Let me stress: this happens when I try to connect to a production DB; if I connect to my local instance of Mongo I've no issues at all.
I do wonder if this can be related to the fact I'm connecting to a slave instance of a replica-set. Tentative question: are replica set supported in Mingo?

Happy to help! I want to see this product succeed. I already love it and prefer it much more than NoSQLBooster!

@tothradoslav
Copy link
Contributor

Well, that's great to hear.

The database connection is bugging me. I am using replica sets with secondaryPreferred option and even SSH tunnel and connects properly. My connection looks something like this:

mongodb://uns:@mongo1:9000,mongo2:9000/uns?authSource=admin&replicaSet=rs0&readPreference=secondaryPreferred

Maybe the authSource option would be missing?

BTW, the TSL/SSL connection is ready, just need to release it. It'd be great to find the db connection bug before I do so...

@dmolin
Copy link

dmolin commented Dec 15, 2021 via email

@dmolin
Copy link

dmolin commented Dec 16, 2021

Good news :)

So, I had to try out a couple of times but it seems to be working if I specify the list of replica set servers! that's the difference compared to what I was used to with NoSQLBooster. In NoSQLBooster I have setup 2 separate connections: one for the slave and one for the master in the replicaSet; I actually liked that, since I could reliably connect mainly to the slave, being sure that my connection was exclusively read-only, while I used the "main" connection only when in need of making changes to the prod database.

Is there any chance that something like this is possible in Mingo too?
I see that by default the connection is read-only too in Mingo, but if I confirm that I want to make changes then it becomes permanently read-write, so I wonder if there's a way to obtain the same functionality I currently get with NoSQLBooster (1 connection with read-only access, 1 with read-write).

The only thing that comes to mind right now is to "clone" the connection (and name the clone "RO") and just use the cloned one (where I never give the permission to "unlock" the database) as a read only access; I think that might work and essentially give me the same functionality I get with the other client.

Now, apart from the passphrase in clear, the only feature that remains to be added is the support for TSL/SSL. I tried adding "?ssl=true" to my connection string but I then get the error:

"Error: MongoServerSelectionError: self signed certificate"

One last thing: is there a way to "defer" the opening of the connections until I actually want to use a DB? In my connection list now I've a couple of configurations that are not working right now (like the one that requires SSL) but I want to keep them there until we find a way to make them work; Though, this means that even when I use another connection (like the one one my local dev env) I keep getting periodic popups "SSH Tunnel Error"... I assume there's a background process that constantly scans the entire list of my connections and check them out; This is not ideal; is there a way to disable this and let it happen only when I actually use a connection?

Excellent work anyway; I really appreciate!

@tothradoslav
Copy link
Contributor

Great, and thanks for all the info :)

I think it should be possible in Mingo, too, so will take a look. We will try upload a new build tomorrow. Will keep you posted.

We will also take a look at the ssh tunnel errors.

Thanks again!

@tothradoslav
Copy link
Contributor

tothradoslav commented Dec 16, 2021

I'd like to know what format of mongo URI do you use in NoSQLBooster to connect directly to the slave?

I'm using the following format for example:
mongodb://uns:@mongo2:9000/uns?authSource=admin&replicaSet=rs0&readPreference=secondaryPreferred

Where mongo2 is the secondary. It works partially, but can't read the list of databases, shows an error "fetchDatabases error MongoServerError: not master and slaveOk=false"

Could you please:

  1. send me the format of your URI you'd like to use
  2. take a look in dev tools console for any error reports when you click "refresh connection". I'd like to know if your error is the same as mine.

Thank you!

BTW, looking at documentation, master / slave setup is not supported since MongoDB 4.0, so just a reminder to prevent confusion. It's now a replicaSet and masters are elected.

@dmolin
Copy link

dmolin commented Dec 16, 2021

In NoSQLBooster you don't need to use a fully qualifier URI. The connection editor panel has a "Basic" tab where you can just enter the "server name" and the port.

Then in the "Authentication" tab you can enter the Auth-Db name and the username and pwd for DB authentication.
And further down, in the "SSH" tab, you can enter the information for configuring the SSH tunnel (server address, port, username, auth mode, private key and passphrase).
Same thing for SSL/TLS, (the "SSL" tab), where I just check "Use SSL/TLS protocol to connect

So, in the case of our DB with replica, I just configured 2 separate connections; one using the replica server address and another identical one using the master server address

Mingo follows a different approach, that is "closer to the metal", so to speak and I'm absolutely fine with that.

My expectation was to compose the uri as you did, probably without the "replicaSet=xxx&readPreference=xxx" part (options that I don't provide even when I use the entire servers list in the URI).

Right now I'm just listing all the servers in the URI and just avoid "unlocking" the DB. (and then I've cloned the connection and called the copy "PROD RW" and there I unlock the DB). Not ideal, since in a perfect world I'd like to avoid hitting the master DB at all, but I'm probably being picky there 😄

I'll take a look at the console (I always forget that Mingo is an Electron based app!) and report back as soon as I solve an issue I'm having right now in getting access to our prod servers 😅

@tothradoslav
Copy link
Contributor

tothradoslav commented Dec 16, 2021

The SSL options will be added in next release and SSH is already there.

The only difference is, that in Mingo you will have to define the database name in the uri and set the proper readPreference. That's the /uns in my case and readPreference=secondaryPreferred in the options. Could you try that?

This is due to how Nodejs Javascript driver works.

The READ ONLY is the actual setup of your servers for the slave, right?

@dmolin
Copy link

dmolin commented Dec 16, 2021

yes (for the read only).

One SUPER critical thing though:

You REALLY need to defer connecting to the DB until the connection is actually used. The reason why I was unable to connect to my PROD environment a few hours ago was that Mingo actually killed our SSH gateway. I had a couple of configurations in Mingo that were not working (because of the connections issue you were trying to solve for me 😄 ). Though, it seems Mingo tried ALL DAY connecting/reconnecting to those connections, even if I was not even trying to use them. This resulted in a flood of connections to the SSH gateway that equated to an effective DDOS attack, with the gateway crashing under the weight of the repeated connection attempts.

So please, avoid trying opening a connection unless:

  1. I actively Try it out (when configuring a new connection)
  2. I decide to use it (selecting it in the dropdown of the available connection)
    Except for those 2 cases, a connection should never be attempted (and/or retried over and over)

On this account: do you know where I can find the connections I've configured in Mingo? are they stored somewhere in the file system? I'd like to remove the ones I made to my prod env before I even open it... otherwise as soon as I open it, it will probably start flooding the SSH gateway again

@tothradoslav
Copy link
Contributor

Wow, that behaviour is not expected definitely and will work on it. Really sorry about that.

Regarding the configurations, they are stored in a config file in JSON format, but with all the other settings of Mingo.

If you are on a mac, it is under

~/Library/Application Support/Mingo/config.json

This file could be quite large actually.

Sorry, again :(

@dmolin
Copy link

dmolin commented Dec 16, 2021

No worries :)
I'm actually on Linux but I found it under ~/.config/mingo
Since right now I'm mainly using Mingo for my local dev env, I'll just remove the entire folder so it will get recreated from scratch next time I fire it up and I'll reconfigure my local dev connection :)

@tothradoslav
Copy link
Contributor

Hi @dmolin, we have completely rethought the way connections and SSH tunnels are managed in the background to make sure:

  • only the connection necessary for the currently viewed connection is kept alive
  • same applies to SSH tunnels
  • connections are cached and never recreated
  • when testing a new / updated connection, the connection is closed right after
  • you can manually disconnect from all open connections

We will be testing this now and will release an RC soon. Will let you know. I hope you'll be willing to try it out after all the hassle. Thank you.

@dmolin
Copy link

dmolin commented Dec 17, 2021

Hey @tothradoslav , thanks a bunch for all your hard work! It's definitely appreciated and it's all great news for my ears!
I'll definitely give it a try once you guys release the next RC :)

In the meantime I've just purchased my yearly license ;)

@tothradoslav
Copy link
Contributor

Hi @dmolin, the new RC is published, please take a look when you get a moment. SSL support is also included.

https://github.com/mingo-app/mingo/releases/tag/v1.6.5-rc.3

Thanks!

@tothradoslav
Copy link
Contributor

We have just released a new pre-release of Mingo, please check it out and let us know if your issue has been corrected. Thanks!

https://github.com/mingo-app/mingo/releases/tag/v1.6.5-rc.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants