Add better documentation for the use of a dictionary in data_diff.connect and data_diff.connect_to_table

[geo909]

Is your feature request related to a problem? Please describe.

As a technical person who however is not a developer, I've had an extremely hard time understanding how to use a dict in db_conf, as described in the docs here.

Describe the solution you'd like

After a lot of digging in the code I think I realized that this depends on the type of database but, still, I feel that some kind of explanation should be provided about what this dict is. I'm not talking about something comprehensive that covers all databases, but some place to start. Maybe mention that dictionary items are arguments from the corresponding connections and provide one example?

Maybe this is something obvious to more experienced users, but keep in mind that nowdays ETL is used by a broader audience, and I'm sure that many people will have the same question.

In any case, thanks for the great work on data-diff!

[erezsh]

Hi @geo909 ,

Thanks for pointing it out! I agree it's not very clear.

The dictionary is equivalent to the database definition of the TOML config file, in terms of parameters and their values.

Here's an example connection dict for Snowflake:

SNOWFLAKE_CONN_INFO = {
    "driver": "snowflake",
    "user": "erez",
    "account": "whatever",
    "database": "TESTS",
    "warehouse": "COMPUTE_WH",
    "role": "ACCOUNTADMIN",
    "schema": "PUBLIC",
    "key": "snowflake_rsa_key.p8",
}

Does that answer your question?

[geo909]

Thanks for the quick reply @erezsh, yes it does make sense now. Do you think it would be a good idea to include something like your answer in the docs? Personally I would have found it very helpful.

[erezsh]

Yes, I'm going to add it to the docs.

[erezsh]

Added info to docstring and docs - https://github.com/datafold/datafold-docs/pull/37