Databricks -> PowerBi (Fabric) support for COLLATE UTF8_LCASE type

[hb0313]

We currently are implementing Unity catalog tables with UTF8_LCASE collation for STRING types, it looks like if using ADBC connector for Power BI through Fabric using implementation 2.0 beta collation to UTF8_LCASE is not supported. But if switching to default version it is supported. Wanted to check if their are any plan to have support for collation UTF8_LCASE in 2.0 version of ADBC?

[CurtHagenlocher]

Can you clarify what you mean by "support"? The driver itself isn't typically doing anything for which a particular collation might need specific support. The query generation layer might (depending on what exactly the desired result is), but that lives in the Power BI connector and not in the ADBC driver.

[hb0313]

So I went ahead and did some more digging on my end and looks like the default implementation uses ODBC which why the UTF8_LCASE collated data type is supported, but when switching to 2.0 which is actually ADBC 2.0 beta version, their is no support of collated data type.

Source - https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi-adbc#adbc

Fabric introduced support for collation recently - https://blog.fabric.microsoft.com/en-us/blog/announcing-full-collation-support-for-sql-databases-in-microsoft-fabric-preview/#

[CurtHagenlocher]

Again, what do you mean by "support"? Are you getting an error message from the driver?

I can assure you that the support for collation in the Fabric SQL database has nothing to do with loading data from Databricks. Fabric SQL is a relational database inside Fabric, and that announcement is related to tables inside that database. It is possible, of course, to load data from Databricks into Fabric SQL but then the collation would match whatever was used to create the Fabric SQL table and be independent of the collation in Databricks.

[xborder]

maybe showing the error you're getting will help. Is this what you're seeing?

[CurtHagenlocher]

If so, that seems to have been fixed in adbc-drivers/databricks#285 and I imagine the fix will soon appear in Power BI.

[hb0313]

Right this is the same error I use to get and now it is not

[hb0313]

Thanks for helping out, this seems to be fixed now in adbc-drivers/databricks#285