Fixed SQL Server tables helpers

[pnetherwood]

Fixed SQL Server table helpers for foreign keys and references.

[pnetherwood]

I've sorted it out now. Even though sql server uses the information schema it does not match that of Postgress as first thought. I noticed the References query was not pulling in all of the table references so I've updated that to.

[kristijanhusak]

It works now, thanks!

[kristijanhusak]

@pnetherwood I added support for schemas and jumping to foreign key from the results buffer. I would appreciate if you would test it with your big databases just to make sure everything works fine.

[pnetherwood]

I've just tried it and it worked fine on sql server.
I had a couple of problems on a mysql db. Rather than have my password in the DB URL I was waiting for a password prompt. Unfortunately it then failed to load the schemes and the only way to fix was to include the password in the URL. I then kept getting the message from mysql about it being insecure to use password on the command line. This extra message then caused the queries to return nothing in the preview window. I was all fixed once I went to using --login-path. The second issue is that the table helpers didn't work on schemas that were not the name of the selected database. For example once connected to my database I went to the INFORMATION_SCHEME and select List on COLUMN_NAMES and it came back with an error saying it couldn't find COLUMN_NAMES in my database.

[kristijanhusak]

All of these issues are on mysql?
Did you have them before introducing schemas?

[kristijanhusak]

I found an issue, and fixed it. Please do another test if possible.

[pnetherwood]

That's fixed the password issue.
The schema issue remains. Say I have a URL of mysql://user@localhost/prod. When I log on I see prod listed in the schemas as well as another database e.g. dev. If I open List on dev.testdata I get an error saying Table prod.testdata does not exist.
I only have this problem with mysql and not sql server.

[pnetherwood]

I got this when I tried the foreign key jump in sqlserver:

Error detected while processing function db_ui#dbout#jump_to_foreign_table[11]..db_ui#schemas#query:
line    1:
E716: Key not present in Dictionary: conn, 'interactive')
E116: Invalid arguments for function db#adapter#dispatch(a:db.conn, 'interactive')

Also, when I do a Refresh (R) tables the tree does not change to reflect the addition and deletion of tables.

[kristijanhusak]

I think i fixed both issues. Now schema is appended to the queries where applicable, and i introduced two variables, {schema}, and {optional_schema}.
Would appreciate if you would give it a test.
Also, for jump foreign key issue, i passed down wrong key, should be working now.

[pnetherwood]

On the foreign key jump I get:

[DBUI] No valid foreign key found.

I was wondering if my foreign key query is returning the results in the form you are expecting.

Also the the refresh is still not updating the tree.

[kristijanhusak]

I'm using your query, just limiting results to 1 and getting two columns only.
Do you get the output for this query:

      SELECT TOP 1
         c2.table_name as foreign_table_name,
         kcu2.column_name as foreign_column_name
      from   information_schema.table_constraints c
             inner join information_schema.key_column_usage kcu
               on c.constraint_schema = kcu.constraint_schema
                  and c.constraint_name = kcu.constraint_name
             inner join information_schema.referential_constraints rc
               on c.constraint_schema = rc.constraint_schema
                  and c.constraint_name = rc.constraint_name
             inner join information_schema.table_constraints c2
               on rc.unique_constraint_schema = c2.constraint_schema
                  and rc.unique_constraint_name = c2.constraint_name
             inner join information_schema.key_column_usage kcu2
               on c2.constraint_schema = kcu2.constraint_schema
                  and c2.constraint_name = kcu2.constraint_name
                  and kcu.ordinal_position = kcu2.ordinal_position
      where  c.constraint_type = 'FOREIGN KEY'
      and kcu.column_name = '{column_name}'"

Just add your {column_name} in the query. If answer is yes, can you give me the full output of the query? Values itself are not important, so you can mask them if needed, I just need to know exact structure of output. Thanks.

[pnetherwood]

I had to modify the query to include the table name and schema:

 and c.TABLE_NAME = '{table}' and c.TABLE_SCHEMA = '{schema}'

but then when I did I got a two column result.

[kristijanhusak]

Why you had to include those?

[pnetherwood]

Because without it it was not unique. It was returning all of the foreign keys in the database with a column of that name. I ran out without the TOP 1 to check.

…

On Fri, 1 May 2020, 18:15 Kristijan Husak, ***@***.***> wrote: Why you had to include those? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2VNB3SXSEJZWJHIVXVF33RPL7TDANCNFSM4MWUB54Q> .

[kristijanhusak]

I don't really know which schema and table is used in dbout buffer, that's why I rely only on column name. I don't expect it to return unique data, that's why I use top 1. Does it return anything for you only with column name and top 1 ?

[pnetherwood]

Top 1 is not going to work unless the foreign key column name is unique across the whole database which is unlikely.
When you do the foreign key query, don't you have four columns: the FK name, original table, foreign table and foreign column which can then parse to do find the table name or am I missing something?

[kristijanhusak]

Foreign key column is not unique, but the target key should be unique, at least in a sane database structure.
So with this query i'm getting first item where foreign key is what you are jumping from, and then getting the foreign column. So if you have a PersonID in a Posts table, when you call the mapping from Posts results person id cell, It will call above query that should return:

foreign_table_name | foreign_column_name
Persons            | ID 

From this I generate query select * from Persons where ID = 'cell value that was clicked from'
Here's a gif how it works for me, but I have only sample mssql database, not with the real data.

[pnetherwood]

Ah, I see how it works now. I was bringing up the foreign key constraint list and then trying to jump to the referenced table from the foreign key constraint. When I go to a table and jump from the from the row/column item it works great. Very nice. Apologies. I thought it was to jump from relationships not cells which explains why I was confused about the query. The gif really helped. Nice work.

Did you have chance to look at the refresh?

[kristijanhusak]

Yeah it's a bit hard to explain how it exactly works. This is how it's done in DBeaver, so i tried to simulate that.

What's exactly wrong with refresh? What is not updating?

[kristijanhusak]

Found it and fixed it. Should be good now.

[pnetherwood]

I can't open the database from the tree. After the last update I'm getting:

[DBUI] Error connecting to db report: Vim(let):E688: More targets than List items

when I try to connect to the database. A normal dadbod DB command works fine with the same URL. My URL is in the form: sqlserver://user:password@host/db

[kristijanhusak]

I pushed a fix, but i'm confused how it happens.
If you run this query:

SELECT table_schema, table_name FROM information_schema.tables

Do you have some results where table_schema or table_name is empty?

[kristijanhusak]

I did few changes, hopefully there won't be any more issues. Let me know if you have any.

[pnetherwood]

It logs on now and populates the schemas but the schemas are not populated with tables. The above query correctly returns the schemas and tables with no missing entries.

[pnetherwood]

Ok. I've figured out what it is. With sqlcmd you can specify an ini file by exporting en env var SQLCMDINI=. (The ini file is quite handy as you cannot specify any additional parameters to sqlcmd in dadbod). In the file I had changed the separator to be a vertical line and I had specified mine as a unicode vertical line character. When I replaced it with a normal bar character it worked fine. So disregard the last message. It all works well now. Many thanks.

[pnetherwood]

The refresh also all works fine too now. Thanks.
I really like that foreign key jumping feature (now that I understand it).

[kristijanhusak]

In latest commit I introduced using custom separator (pipe) for easier parsing, so maybe now it should work for you even with custom ini file. Not sure which one has precedence, sqlcmd argument or the ini file.

[pnetherwood]

The ini file appears to have precedence and I have set it to the pipe character too.
What I'm using it for is to make the columns in the results pane look nicer e.g.

name           |description                                                 |countryCode|marketIdentifierCode|iceberg|crossTrade|supportsCancelReplace|MIC 
---------------|------------------------------------------------------------|-----------|--------------------|-------|----------|---------------------|----
BMF            |Bolsa de Mercadorias e Futuros                              |BRL        |BVMF                |N      |NO        |N                    |NULL
CBOE           |Chicago Board Options Exchange                              |USA        |XCBO                |N      |0         |Y                    |NULL
CBOT           |Chicago Board of Trade                                      |USA        |XCBT                |Y      |5         |Y                    |NULL
CFFEX          |China Financial Futures Exchange                            |CHN        |???                 |N      |0         |N                    |NULL
CME            |Chicago Mercantile Exchange                                 |USA        |XCME                |Y      |5         |Y                    |NULL

But now that messes with your foreign query jump functionality because you are expecting a space delimiter on the column name in dbout s:get_cell_range. Any suggestions? I could remove the separator in the ini file but I rather like it. Could you rely on naming conventions instead of a delimiter?

[kristijanhusak]

I changed the way i'm getting the range. As long as delimiter is not set to -, which is the same char used for the table line, it should work.

[pnetherwood]

Nice. That works great.