Enhances MSSQL connection with pyodbc support

[mattiasthalen]

Adds support for pyodbc as an alternative driver for MSSQL connections, providing users with more flexibility and authentication options.

• Introduces a driver option to choose between pymssql and pyodbc.
• Implements connection string generation for pyodbc based on connection parameters.
• Includes validation for different authentication methods when using pyodbc.
• Updates dependencies to include pyodbc.

[mattiasthalen]

@erindru hey, I just got around to do this after some onboarding at my new employer.

I also had some time to play around with the SQL Database object in fabric, and... I think just adding pyodbc allows us to connect to it like any other mssql. Dare I say it? It feels like it's really close to have some initial Fabric support! 👯

I know, I know, it's not the lakehouse, nor the warehouse objects. But it's something!

[erindru]

Thanks @mattiasthalen !

Yes we were waiting for Fabric Lakehouse to become a sane product before trying to support Fabric. Interesting to see if this actually works enough to hit the SQL Warehouse endpoint or if there are some Fabric-specific nuances.

Can you please add a section to the docs on how to set up pyodbc to work with Microsoft's ODBC driver?

Also, can you please rebase onto main and run make style? A bunch of test failures were related to code linting

(if this is still a WIP I can hold off reviewing until it leaves draft status)

[mattiasthalen]

@erindru I tried hitting the Warehouse endpoint, but there are several differences compared to MSSQL that doesn't work the same. E.g., information_schema is case sensitive.

Shouldn't be that hard to support it though. I'll probably have a look on that next.

Still refreshing to see that the SQL Database object was supported by just being able to login.

Ok, I was scratching my head on how to solve those failing tests 😅

Will fix asap and send for review 🎉

[erindru]

Thanks, this is looking pretty good!

Just one more thing - there are no tests. Can you please add some tests to test_connection_config that show:

• The parameter validations you added are working
• The _connection_kwargs_keys are as expected for each variant (pymssql and pyodbc)
• pyodbc.connect gets invoked with the correct ODBC string

[mattiasthalen]

Hey sorry for being slow to progress. It's been hectic at work, but I hopens have time this weekend and write the tests

[erindru]

No problem, there is no expectation / time pressure on community contributions! Thanks for the update

[crazy-treyn]

@erindru I tried hitting the Warehouse endpoint, but there are several differences compared to MSSQL that doesn't work the same. E.g., information_schema is case sensitive.

FYI, Fabric Warehouses can now be created with a case insensitive collation: Create Warehouse with Case Insensitive Collation

The new Fabric CLI makes it really easy to create a new Warehouse with this case insensitive collation without messing with the REST APIs.

It is possible that your implementation would work with a Warehouse created with a case insensitive collation!

[mattiasthalen]

@erindru I tried hitting the Warehouse endpoint, but there are several differences compared to MSSQL that doesn't work the same. E.g., information_schema is case sensitive.

FYI, Fabric Warehouses can now be created with a case insensitive collation: Create Warehouse with Case Insensitive Collation

The new Fabric CLI makes it really easy to create a new Warehouse with this case insensitive collation without messing with the REST APIs.

It is possible that your implementation would work with a Warehouse created with a case insensitive collation!

Thanks for that! It doesn't work, but now it's due to other things. One slow step at a time :)

[erindru]

Closing in favour of #4686